Bug 1477777 - RFE: please provide a fresh anti-virus database in a sub-package
RFE: please provide a fresh anti-virus database in a sub-package
Status: NEW
Product: Fedora
Classification: Fedora
Component: clamav (Show other bugs)
25
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Sergio Monteiro Basto
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 1477130
  Show dependency treegraph
 
Reported: 2017-08-02 19:02 EDT by Roman Joost
Modified: 2017-11-16 14:20 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
clamav-data-nightly.spec (832 bytes, text/plain)
2017-09-08 07:57 EDT, Kamil Páral
no flags Details

  None (edit)
Description Roman Joost 2017-08-02 19:02:30 EDT
Description of problem:

As discussed on the mailing list for Bug 1477130, tools invoking the clamav tool to scan packages for virus signatures need a fresh database. The database is usually updated with a cron job or a systemd timer, but for tools running in CI environments the approach is not viable. Furthermore, downloading the database for each job adds quite an overhead.

So I'm requesting that a separate package ships an updated database which tools in Fedora can depend on. That way, the database is already up-to-date and no additional overhead to update it is needed.

Version-Release number of selected component (if applicable):
latest

How reproducible:
100%

Steps to Reproduce:
1. Install clamav
2. Need to run freshclam to update the anti-virus database

Expected results:
e.g. dnf install clamav-antivirus-db
Comment 1 Jason Tibbitts 2017-08-02 19:37:17 EDT
This is an odd request.  If someone wants that separate package, someone needs to create it and get it reviewed.  That doesn't need to be the clamav maintainers, though once your package is reviewed imported, the clamav package could add dependencies on it.

The clamav package itself provides clamav-data, but of course updating that requires updating the entire package which isn't really something that can be done particularly often.

Also, I'm not sure that making a separate package can actually satisfy the requirements that you have.  Updates to this package would still have to go through the regular updates system and hence that package could never be less than seven days past its prime (except for rawhide, where it might be a day or so old if rawhide is composing properly, and for EPEL where it could never be any less than two weeks old).

I think that, really, another solution is called for to fix 1477130.
Comment 2 Kamil Páral 2017-08-03 03:29:48 EDT
Have you seen Dan's argument comparing this to hwdata package?
https://lists.fedoraproject.org/archives/list/qa-devel@lists.fedoraproject.org/message/RDU7M7ILO6TQWSC2GD75X3AU3ORPBTNE/

The problem is not about having a week old database, or even a month old database (that's still a massive improvement), but 13 months old database (that's the current default state). A monthly update to the database would be great.

Another solution would be to make freshclam work faster. Currently it connects to servers which are timing out. Is there something we can do about this? Are there more reliable servers elsewhere it could connect to? Maybe change the order of default servers and put the more reliable ones to the top?
Comment 3 Robert Scheck 2017-08-06 14:34:03 EDT
(In reply to Kamil Páral from comment #2)
> Another solution would be to make freshclam work faster. Currently it
> connects to servers which are timing out. Is there something we can do about
> this? Are there more reliable servers elsewhere it could connect to? Maybe
> change the order of default servers and put the more reliable ones to the
> top?

How about running an own mirror (e.g. hosted by Fedora) for ClamAV updates?
Comment 4 Jan ONDREJ 2017-08-07 02:49:19 EDT
(In reply to Robert Scheck from comment #3)
> How about running an own mirror (e.g. hosted by Fedora) for ClamAV updates?

ftp.upjs.sk is an official mirror of Fedora/EPEL and clamav too.
You can use it. It's accessible as clamav.upjs.sk and other names (as clamav sk mirror too).

Adding a new clamav mirror is a bit difficult. It took me about half year to add ftp.upjs.sk (CISCO infrastructure admins are very busy).
Comment 5 Jason Tibbitts 2017-08-07 15:14:36 EDT
Yes, running an official clamav mirror is really quite difficult.  One of their requiremsnts is that they be able to connect to your machine to initiate a mirror sync.  You can't just run it from cron or monitor some external service.

I don't know if there's any possibility of running an unofficial mirror.  Of course, for the purposes of Fedora infrastructure, the best thing would be to just have a machine that keeps an up to date virus database using freshclam, and for these files to just get copied into the container for virus checking.  That way there's no delay at all.

If some infrastructure or testing component depends on clamav and the people involved with that component want to lend a hand with package maintenance, that would be great.  Or if they want to submit a separate package for just the virus database that updates frequently, I'm sure nobody would complain and the base clamav package could be fixed up to accommodate.

However, to be completely honest I'm not sure what actually happens if you have a machine that runs freshclam to stay up to date, and then it gets an updated clamav-data package.  If it walks you back to an older virus database then that wouldn't be particularly good.  Does anyone actually know?  The files in /var/lib/clamav aren't versioned or anything so I think they'd just get overwritten with the older versions and if so that would make a constantly-updating clamav-data package rather not a good thing.
Comment 6 Miro Hrončok 2017-09-07 16:51:25 EDT
(In reply to Jason Tibbitts from comment #5)
> However, to be completely honest I'm not sure what actually happens if you
> have a machine that runs freshclam to stay up to date, and then it gets an
> updated clamav-data package.  If it walks you back to an older virus
> database then that wouldn't be particularly good.  Does anyone actually
> know?  The files in /var/lib/clamav aren't versioned or anything so I think
> they'd just get overwritten with the older versions and if so that would
> make a constantly-updating clamav-data package rather not a good thing.

From clamav.spec:

%files data
# use %%config to keep files which were updated by 'freshclam'
# already. Without this tag, they would be overridden with older
# versions whenever a new -data package is installed.
%config %verify(not size md5 mtime) %homedir/*.cvd


I think that would work quite fine here as well.
Comment 7 Miro Hrončok 2017-09-07 17:14:00 EDT
I've created a copr demo at https://copr.fedorainfracloud.org/coprs/churchyard/clamav-data-nightly/

(The db is fetched trough build).

All that has to be done is a cron job for hitting resubmit every night and it should work like a charm.

spec: https://paste.fedoraproject.org/paste/mpGLImHrx54a4jZMyMEFPg
Comment 8 Kamil Páral 2017-09-08 07:48:00 EDT
COPR is good enough solution for our Taskotron task provided somebody is willing to volunteer to maintain it.
Comment 9 Kamil Páral 2017-09-08 07:57 EDT
Created attachment 1323708 [details]
clamav-data-nightly.spec

Spec file reupload from comment 7 so that it doesn't expire.
Comment 10 Sergio Monteiro Basto 2017-09-11 15:12:48 EDT
(In reply to Miro Hrončok from comment #6)

> From clamav.spec:
> 
> %files data
> # use %%config to keep files which were updated by 'freshclam'
> # already. Without this tag, they would be overridden with older
> # versions whenever a new -data package is installed.
> %config %verify(not size md5 mtime) %homedir/*.cvd
> 
> 
> I think that would work quite fine here as well.

So clamav does not need any modification , isn't it ? 

(In reply to Miro Hrončok from comment #7)
> I've created a copr demo at
> https://copr.fedorainfracloud.org/coprs/churchyard/clamav-data-nightly/
> 
> (The db is fetched trough build).
> 
> All that has to be done is a cron job for hitting resubmit every night and
> it should work like a charm.
> 
> spec: https://paste.fedoraproject.org/paste/mpGLImHrx54a4jZMyMEFPg

This is a  new package we should transform it in package review request , but 
IMHO the cron should run freshclam if we have update build it in koji , something like  [1]

[1]
https://pkgs.rpmfusion.org/cgit/nonfree/lpf-spotify-client.git/tree/check_new_version.py
Comment 11 Miro Hrončok 2017-09-12 04:41:25 EDT
(In reply to Sergio Monteiro Basto from comment #10)
> This is a  new package we should transform it in package review request

Well, currently, this is impossible in Fedora (build fetches from the Internet). But if somebody wants, it can be fetched and source created, than used. I've just created it a s a demo, because I was curious how hard it would be.
Comment 12 Fedora End Of Life 2017-11-16 14:20:13 EST
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Note You need to log in before you can comment on or make changes to this bug.