Bug 219078 - [RFE] sa package should setup sa-update
Summary: [RFE] sa package should setup sa-update
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: spamassassin
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Warren Togami
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2006-12-10 15:37 UTC by Nicolas Mailhot
Modified: 2007-11-30 22:11 UTC (History)
9 users (show)

Fixed In Version: spamassassin-3.1.8
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-04-17 17:49:05 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
add sa-update cron (spec file) (3.55 KB, patch)
2006-12-11 09:21 UTC, Takanori MATSUURA
no flags Details | Diff
logrotate setup file (69 bytes, text/plain)
2006-12-11 09:22 UTC, Takanori MATSUURA
no flags Details
cron entry file (55 bytes, application/octet-stream)
2006-12-11 09:23 UTC, Takanori MATSUURA
no flags Details
cron script (199 bytes, application/octet-stream)
2006-12-11 09:29 UTC, Takanori MATSUURA
no flags Details
cron entry file (89 bytes, text/plain)
2006-12-11 10:21 UTC, Takanori MATSUURA
no flags Details
cron script (265 bytes, text/plain)
2006-12-11 10:23 UTC, Takanori MATSUURA
no flags Details
cron entry file (211 bytes, text/plain)
2006-12-14 08:31 UTC, Takanori MATSUURA
no flags Details
cron entry file (92 bytes, patch)
2006-12-14 08:32 UTC, Takanori MATSUURA
no flags Details | Diff
cron entry file (92 bytes, text/plain)
2006-12-14 08:33 UTC, Takanori MATSUURA
no flags Details
cron script (211 bytes, text/plain)
2006-12-14 08:36 UTC, Takanori MATSUURA
no flags Details

Description Nicolas Mailhot 2006-12-10 15:37:44 UTC
A good sa setup includes sa-update nowadays. The Fedora package could make it
easier on users :

1. create the directory sa-updates saves keys in
2. prepopulate it with the official upstream key
3. drop an update cron in /etc/cron*, possibly commented and possibly including
SARE channel. I use this one :

0  */4 * * * root  /usr/bin/sa-update --channel updates.spamassassin.org --chann
el saupdates.openprotect.com --gpgkey 265FA05B --gpgkey BDE9DC10 && /etc/init.d/
spamassassin restart

Comment 1 Takanori MATSUURA 2006-12-11 09:21:21 UTC
Created attachment 143268 [details]
add sa-update cron (spec file)

Comment 2 Takanori MATSUURA 2006-12-11 09:22:07 UTC
Created attachment 143269 [details]
logrotate setup file

Comment 3 Takanori MATSUURA 2006-12-11 09:23:07 UTC
Created attachment 143270 [details]
cron entry file

Comment 4 Takanori MATSUURA 2006-12-11 09:29:44 UTC
Created attachment 143271 [details]
cron script

This script accesses to the network, so actual start time of sa-update is
randomized.

Is the daemon restart is requied?

Comment 5 Nicolas Mailhot 2006-12-11 09:51:45 UTC
1. please use correct mime types for your attachments (ie text/plain) that makes
viewing them oh-so-much-easier

2. the daemon restart bit comes from the official sa wiki. That's why you should
use the sa-update return value so it only occurs when the actual rule set was
updated. (the system is designed so "is there an update" checks are wickedly
light and can happen more often than actual updates)

3. IMHO if you really want to log sa-updates the traces belong in maillog

4. it seems you totally ignored the gpg handling in your files -- we want to do
it and IIRC it will bite you at the first update attempt. That means creating
the key directory with the right permissions in the spec and putting the
official sa key in there

5. (style related, you can ignore it) can you use $() instead of `' squiggles in
your script ?

Comment 6 Takanori MATSUURA 2006-12-11 10:19:34 UTC
(In reply to comment #5)

Thank you for reviewing my idea.

> 1. 

Sorry for my foolish that I trusted the "auto-detect" in Content Type section.


> 2-5
is reflected to the following attachments.

Comment 7 Takanori MATSUURA 2006-12-11 10:21:22 UTC
Created attachment 143276 [details]
cron entry file

output of sa-update is both /var/log/sa-update.log and stdout (is mailed.)

Comment 8 Takanori MATSUURA 2006-12-11 10:23:51 UTC
Created attachment 143278 [details]
cron script

* `` is replaces to $().
* add GPG key check
* add spamassassin daemon restart

Comment 9 Nicolas Mailhot 2006-12-11 12:02:19 UTC
(In reply to comment #6)
> (In reply to comment #5)
> 
> Thank you for reviewing my idea.
> 
> > 1. 
> 
> Sorry for my foolish that I trusted the "auto-detect" in Content Type section.

bugzilla sucks, I know :(

> > 2-5
> is reflected to the following attachments.

1. You still have a `date' 
2. part of the gpg stuff requires spec changes
3. I don't like the dedicated log file
4. I'm not too sure about adding saupdates.openprotect.com by default

That being said, most of it could be described as policy decisions IE Warren
should tell us what he likes best



Comment 10 Steven Pritchard 2006-12-12 05:53:17 UTC
A few comments:

1) Shouldn't that be "service spamassassin condrestart", not "service
spamassassin restart"?
2) Adding SARE by default is a great idea IMHO, but it might be best to come up
with some generic way of adding additional update channels.  (One of my clients
is considering setting up a private update channel for their local SA rules, for
example.)
3) What about restarting amavisd?  Or any other SA-using process, for that
matter.  It might be worth making that a configurable thing as well.

Comment 11 Nicolas Mailhot 2006-12-12 08:12:33 UTC
(In reply to comment #10)
> A few comments:
> 
> 1) Shouldn't that be "service spamassassin condrestart", not "service
> spamassassin restart"?

+1 Sure, if the sa service supports it

> 2) Adding SARE by default is a great idea IMHO, but it might be best to come up
> with some generic way of adding additional update channels.  (One of my clients
> is considering setting up a private update channel for their local SA rules,
for example.)

The way sa-update is setup it really wants a single command with all the
channels stuffed in it, unless you want cascading service restarts after a long
unsynchronization.

But then, I'm sure a perl guru can come up with a smart solution

> 3) What about restarting amavisd?  Or any other SA-using process, for that
> matter.  It might be worth making that a configurable thing as well.

I think all the modern sa users communicate with sa through smamc/spamd, so they
should pick up changes as soon as spamd is restarted

Comment 12 Warren Togami 2006-12-14 05:57:26 UTC
I am extremely limited on time, and this must go in Thursday morning.  Please
quickly get these tasks done so I can quickly review and include this.

1) How does this differ from the current behavior of running sa-update with the
package as is?
2) Could you please attach a spec.patch unidiff of how you suggest this be added
to the package?
3) Then be sure that all files you suggest to be included are also attached.
4) Set the other attachments to Obsolete so they don't confuse me.


Comment 13 Takanori MATSUURA 2006-12-14 08:31:15 UTC
Created attachment 143598 [details]
cron entry file

updated cron entry file.

Comment 14 Takanori MATSUURA 2006-12-14 08:32:47 UTC
Created attachment 143599 [details]
cron entry file

updated cron entry file

Comment 15 Takanori MATSUURA 2006-12-14 08:33:57 UTC
Created attachment 143600 [details]
cron entry file

updated cron entry file

Comment 16 Takanori MATSUURA 2006-12-14 08:36:40 UTC
Created attachment 143601 [details]
cron script

updated cronscript

use only updates.spamassassin.org.
If anyone want to use other sites, please add the channel into this file.

Comment 17 Takanori MATSUURA 2006-12-14 08:44:15 UTC
(In reply to comment #12)
> 1) How does this differ from the current behavior of running sa-update with the
> package as is?

New function: sa-update works once a day as a cron job.


> 2) Could you please attach a spec.patch unidiff of how you suggest this be added
> to the package?

attachment 143268 [details] is unidiff-ed spec.patch.

> 3) Then be sure that all files you suggest to be included are also attached.

attachment 143269 [details] (spamassassin.logrotate)
attachment 143600 [details] (sa-update.cron_entry)
attachment 143601 [details] (sa-update.cron)


> 4) Set the other attachments to Obsolete so they don't confuse me.

confirmed.


As I noticed at the comment #16, only updates.spamassassin.org is set as the
channel. I think this channel is enough as the default. If other SpamAssassin
rule is needed, users can edit sa-update.cron script.

Comment 18 Kenneth Porter 2006-12-14 16:33:01 UTC
Re comment #11:

Not all SA clients use spamc/spamd. Some are Perl-based and use the underlying
Mail::SpamAssassin classes directly. MIMEDefang is the one I use. It spawns its
own Perl-based slaves that invoke SA.

I'd recommend that the restart script not chain services that need to be
restarted on one command line with &&.

Is there some package-oriented producer/consumer scheme such that multiple
services can register interest in an event like this? Perhaps the thing to do is
to provide a directory somewhere where other packages can drop scripts that will
be invoked when sa-update reports that new content has arrived. If spamd is
enabled, a script to restart it can be dropped in this directory.

This machinery should all be pushed upstream.

See also http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5003 where the
idea of breaking sa-update out into its own subpackage is being discussed.

Comment 19 Warren Togami 2006-12-14 17:58:33 UTC
Comment #18 makes good points.  For these reasons I believe we should hold off
on adding this to the RPM package.

Comment 20 Nicolas Mailhot 2006-12-14 18:29:14 UTC
(In reply to comment #19)
> Comment #18 makes good points.  For these reasons I believe we should hold off
> on adding this to the RPM package.

Given that sa is increasingly useless unless the rules are updated regularly,
that would be grounds to ship the cron commented by default (or noreplace so
people can comment it at need)

(And I run rawhide, so I get the package updates faster than most users)


Comment 21 Warren Togami 2006-12-14 18:39:34 UTC
Red Hat is not fully comfortable with sa-update just yet, because it relies on
an external data source.  For this reason we don't want to officially endorse
its use.

How hard is it to make your own one-liner cron entry to run sa-update?

Comment 22 Nicolas Mailhot 2006-12-14 18:51:30 UTC
(In reply to comment #21)
> Red Hat is not fully comfortable with sa-update just yet, because it relies on
> an external data source. For this reason we don't want to officially endorse
> its use.

It's the same source that released the rules the initial package bundles, and
last I've seen people trusted the Apache foundation

> How hard is it to make your own one-liner cron entry to run sa-update?

It wasn't hard for me, just annoying (create the key directory the package
forget, locate the right gpg key, make it available for sa, find the right
update command…)

For your run-of-the-mill user though, it's way over the hassle ceiling, he'll
just notice sa performance degrades quickly and fedora does not "just work" as
it should.

Given how spam is evolving if Red Hat is not comfortable with direct upstream
rule feeds it will have to set up its own update server sooner or later (same
applies for pyzor BTW)

Comment 23 Theo Van Dinter 2006-12-14 18:58:41 UTC
(In reply to comment #22)
> > How hard is it to make your own one-liner cron entry to run sa-update?
> 
> It wasn't hard for me, just annoying (create the key directory the package
> forget, locate the right gpg key, make it available for sa, find the right
> update command…)

Not to be picky, but running "sa-update" does all of this for you.  The key is
included in the standard distro, and sa-update will automatically do all the
right things if its gpghome dir doesn't already exist.

Generally, if people are starting out using sa-update, I suggest that they run
"sa-update -D" to see what's going on for the first couple of times.  Once used
to the process, it's easy to just put in the cronjob.

Comment 24 Warren Togami 2006-12-14 19:03:06 UTC
felicity, it might be helpful to add -D to the sa-update --help output? =)

Comment 25 Theo Van Dinter 2006-12-14 19:05:04 UTC
(In reply to comment #24)
> felicity, it might be helpful to add -D to the sa-update --help output? =)

Such as where it has:

$ sa-update --help
[...]
      -D, --debug [area=n,...]      Print debugging messages

?  :)

Comment 26 Kenneth Porter 2006-12-14 19:13:09 UTC
Re comment #21:

WRT policy, how is this different from freshclam in the clamav package in
Extras? That program does much the same thing, checking a DNS record to see if
updates are available and downloading new virus data files when available.

I do think that other SA update sources (eg. SARE) should be separate packages,
to allow separation of policy based on update source. If the sources have to be
listed on the sa-update command line, then the script should assemble the
command line from package files in an appropriate directory, one per source. The
potential SARE package can then drop its update source argument as a file in
that directory.

Comment 27 Warren Togami 2006-12-14 19:17:42 UTC
spamassassin is Core and RHEL with possible liability concerns.
clamav is Extras, maintained entirely by community.


Comment 28 Nicolas Mailhot 2006-12-14 19:26:55 UTC
(In reply to comment #23)

> Not to be picky, but running "sa-update" does all of this for you.

You just proved I'm as confused as everyone else by sa-update :). Which rather
makes my point.

Anyway apart from the fact it didn't just work here (bad user, no naked
sa-update run at first, whatever) is it good policy not to create /etc content
in the package and let upstream commands do it themselves post-install ?

(In reply to comment #27)
> spamassassin is Core…

but Core and Extras are supposed to be merged, right?

Comment 29 Jason Tibbitts 2006-12-14 19:50:20 UTC
Isn't it completely possible to put all of this sa-update functionality in a
separate package that could go into extras, so no core changes are required?



Comment 30 Warren Togami 2006-12-14 20:04:05 UTC
I will include the commented out cron and cron script.  It will use "service
spamassassin condrestart" or some equivalent...

Comment 31 Warren Togami 2006-12-14 21:31:22 UTC
http://people.redhat.com/wtogami/temp/spamassassin-3.1.7-0.test.el4.src.rpm
I did it slightly differently.  Please review these changes.

I have no idea why you wanted to move the updates to /var/spool/spamassassin
when upstream has defaulted for /var/lib/spamassassin for a long time now.

Comment 32 Takanori MATSUURA 2006-12-15 01:11:41 UTC
(In reply to comment #31)
> I have no idea why you wanted to move the updates to /var/spool/spamassassin
> when upstream has defaulted for /var/lib/spamassassin for a long time now.

See Bug 187974.


Note You need to log in before you can comment on or make changes to this bug.