Bug 1248330

Summary: update RegistrarBoundaries.pm
Product: Red Hat Enterprise Linux 6 Reporter: William Yardley <wby+redhat>
Component: spamassassinAssignee: Jakub Jelen <jjelen>
Status: CLOSED NOTABUG QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: high    
Version: 6.7CC: cww, mteixeira, pbhoot, plautrba, psklenar, wby+redhat
Target Milestone: rcFlags: wby+redhat: needinfo-
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1261646 (view as bug list) Environment:
Last Closed: 2016-06-03 20:57:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1217621, 1269194    

Description William Yardley 2015-07-30 05:51:32 UTC
Description of problem:
As described at http://uribl.com/, SpamAssassin before version 3.4.1 doesn't look at 20_aux_tlds.cf, and so misses new TLDs like .xyz and .science.

From their site:


If you are using SpamAssassin, and are currently receiving spam with new TLDs such as .LINK, .SCIENCE, .CRICKET, .NINJA, .XYZ, .CLICK (and others), it is because those URLs are not being checked against URIBL. In order to lookup these URLs, you will need to upgrade your RegistrarBoundaries.pm.

If you are running a recent version of SpamAssassin such as 3.3.2 or 3.4, simply download the RegistrarBoundaries.pm from SpamAssassin SVN and replace the one you currently have installed. Restart spamd (if you run it) for the change to take effect.
Link to SVN:
http://svn.apache.org/repos/asf/spamassassin/trunk/lib/Mail/SpamAssassin/Util/RegistrarBoundaries.pm

Version-Release number of selected component (if applicable):
3.3.1-3.el6

How reproducible:
very

Comment 2 Jakub Jelen 2015-07-30 09:06:28 UTC
If I understand well the note on your linked url, it would require only update the one file, even if we have older version in RHEL6. Do you have some simple test case to verify the update of the file fix the issue?

And can you reach your regular RHEL support to evaluate this request and to give it proper prioritization?

Comment 3 William Yardley 2015-07-30 15:26:31 UTC
I did already install via:

curl -O http://svn.apache.org/repos/asf/spamassassin/trunk/lib/Mail/SpamAssassin/Util/RegistrarBoundaries.pm
sudo install -o root -g root -m 444 RegistrarBoundaries.pm /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util/RegistrarBoundaries.pm

So the ticket is mostly in the interest in helping other folks, and in avoiding having the file get clobbered in any future updates. We've been getting a lot of snowshoe spam with some of the referenced TLDs.

So far, no problems or obvious errors via spamassassin --lint -D
so this is mostly

I will try to reach out to the support team under our work account to make the business case for this enhancement.

Comment 4 Mauricio Teixeira 2015-08-13 12:31:18 UTC
This issue is also present in RHEL7 with package spamassassin-3.4.0-1.el7.x86_64. Should we open a new bug?

Comment 5 William Yardley 2015-08-13 16:33:54 UTC
FWIW, if you can go to 3.4.1 in EL7, I think the TLDs come from updates.
http://uribl.com/ has a note about it.

Comment 11 Jakub Jelen 2015-10-30 16:08:48 UTC
Petr,
Basically, it should be possible to check example spam message from tld that is not in the old list and is blacklisted on uribl.com. It should not add the score for blacklisted domain in old version, but it should in the current version.

Testing email (can be extended or used some of the linked further):
Sender mail: sender
Message content: http://www.mrp41eur.link

I read through this same problem discussed on [1] and the thing is, that although we have old domain list in 

/usr/share/spamassassin/20_aux_tlds.cf

there is updated list (after running sa-update) with current domains in 

/var/lib/spamassassin/3.003001/updates_spamassassin_org/20_aux_tlds.cf

and it is read by spamassassin according to my observation (in contrast to original report).


After fresh install, without update (or removal of /var/lib/spamassassin/), running test link in the body is not matched:

$ spamassassin -dt spam.eml
Content analysis details:   (-0.0 points, 5.0 required)

After pdating spamassassin (sa-update) it will match the url in body:

$ spamassassin -dt spam.eml
 1.7 URIBL_BLACK            Contains an URL listed in the URIBL blacklist
                            [URIs: mrp41eur.link]

(but not the sender address based on this blacklist).

Removing the results of sa-update and installing only new RegistrarBoundaries.pm makes it match the url in the body again

$ spamassassin -dt spam.eml
 1.8 URIBL_BLACK            Contains an URL listed in the URIBL blacklist
                            [URIs: mrp41eur.link]

Doing both have the same results as only sa-update. This is quite different story than in the original report. This would be great to get in touch with reporter, what is exactly what they want. But with updating the file, we will not break anything and will have fresh install working out of the box.

William, Piyush,
do you have similar expectation from the fix?

[1] http://bugs.contribs.org/show_bug.cgi?id=8980

Comment 12 William Yardley 2015-10-30 16:59:41 UTC
FWIW, we *were* running sa-update via cron, and SA not seem to pick up listed domains within TLDs on that list, even though 20_aux_tlds.cf was in the updates directory. I didn't do extensive testing, but that's what I recall. Maybe SA has made some changes since July that resolved the issue of aux_tlds.cf not being loaded?

I've moved on since this bug was filed, and we also updated RegistrarBoundaries.pm by hand as mentioned above. So not a big deal for me, and I don't have access to test in that environment.

I built a clean Centos 6.5 VM and got the following:

[vagrant@satest ~]$ spamassassin -dt < test.mbox  | grep analysis
Content analysis details:   (-0.0 points, 5.0 required)
[vagrant@satest ~]$ sudo sa-update
[vagrant@satest ~]$ spamassassin -dt < test.mbox  | grep analysis
Content analysis details:   (2.7 points, 5.0 required)

Comment 13 Piyush Bhoot 2015-11-27 12:56:06 UTC
Yes Jakub,

Customer issue was fixed with help of
http://bugs.contribs.org/show_bug.cgi?id=8980

BR,
Piyush

Comment 14 Jakub Jelen 2015-11-30 12:45:34 UTC
As I read through the latest comments in the attached case, I see the current problem is URIBL_BLOCKED and not missing domains.

William, can you make sure there is still the problem in your environment? It is not clear from your last comment if it works as expected for you or not. If not, can you formulate what is the problem and not what is the solution (update RegistrarBoundaries.pm?). Otherwise I will close this bug. The same way should probably end also the cloned one #1261646.

Comment 15 Chris Williams 2016-06-03 20:57:57 UTC
Closing per comment 14.
If this is still an issue in your environment please open a support case in the Customer Portal and request this BZ be re-opened.

Comment 16 William Yardley 2016-08-15 16:43:30 UTC
Just posting to clear NEEDINFO