Bug 1477777 - RFE: please provide a fresh anti-virus database in a sub-package
Summary: RFE: please provide a fresh anti-virus database in a sub-package
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: clamav
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Sergio Basto
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1477130
TreeView+ depends on / blocked
 
Reported: 2017-08-02 23:02 UTC by Roman Joost
Modified: 2019-11-25 08:33 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-12 10:18:54 UTC
Type: Bug


Attachments (Terms of Use)
clamav-data-nightly.spec (832 bytes, text/plain)
2017-09-08 11:57 UTC, Kamil Páral
no flags Details

Description Roman Joost 2017-08-02 23:02:30 UTC
Description of problem:

As discussed on the mailing list for Bug 1477130, tools invoking the clamav tool to scan packages for virus signatures need a fresh database. The database is usually updated with a cron job or a systemd timer, but for tools running in CI environments the approach is not viable. Furthermore, downloading the database for each job adds quite an overhead.

So I'm requesting that a separate package ships an updated database which tools in Fedora can depend on. That way, the database is already up-to-date and no additional overhead to update it is needed.

Version-Release number of selected component (if applicable):
latest

How reproducible:
100%

Steps to Reproduce:
1. Install clamav
2. Need to run freshclam to update the anti-virus database

Expected results:
e.g. dnf install clamav-antivirus-db

Comment 1 Jason Tibbitts 2017-08-02 23:37:17 UTC
This is an odd request.  If someone wants that separate package, someone needs to create it and get it reviewed.  That doesn't need to be the clamav maintainers, though once your package is reviewed imported, the clamav package could add dependencies on it.

The clamav package itself provides clamav-data, but of course updating that requires updating the entire package which isn't really something that can be done particularly often.

Also, I'm not sure that making a separate package can actually satisfy the requirements that you have.  Updates to this package would still have to go through the regular updates system and hence that package could never be less than seven days past its prime (except for rawhide, where it might be a day or so old if rawhide is composing properly, and for EPEL where it could never be any less than two weeks old).

I think that, really, another solution is called for to fix 1477130.

Comment 2 Kamil Páral 2017-08-03 07:29:48 UTC
Have you seen Dan's argument comparing this to hwdata package?
https://lists.fedoraproject.org/archives/list/qa-devel@lists.fedoraproject.org/message/RDU7M7ILO6TQWSC2GD75X3AU3ORPBTNE/

The problem is not about having a week old database, or even a month old database (that's still a massive improvement), but 13 months old database (that's the current default state). A monthly update to the database would be great.

Another solution would be to make freshclam work faster. Currently it connects to servers which are timing out. Is there something we can do about this? Are there more reliable servers elsewhere it could connect to? Maybe change the order of default servers and put the more reliable ones to the top?

Comment 3 Robert Scheck 2017-08-06 18:34:03 UTC
(In reply to Kamil Páral from comment #2)
> Another solution would be to make freshclam work faster. Currently it
> connects to servers which are timing out. Is there something we can do about
> this? Are there more reliable servers elsewhere it could connect to? Maybe
> change the order of default servers and put the more reliable ones to the
> top?

How about running an own mirror (e.g. hosted by Fedora) for ClamAV updates?

Comment 4 Jan ONDREJ 2017-08-07 06:49:19 UTC
(In reply to Robert Scheck from comment #3)
> How about running an own mirror (e.g. hosted by Fedora) for ClamAV updates?

ftp.upjs.sk is an official mirror of Fedora/EPEL and clamav too.
You can use it. It's accessible as clamav.upjs.sk and other names (as clamav sk mirror too).

Adding a new clamav mirror is a bit difficult. It took me about half year to add ftp.upjs.sk (CISCO infrastructure admins are very busy).

Comment 5 Jason Tibbitts 2017-08-07 19:14:36 UTC
Yes, running an official clamav mirror is really quite difficult.  One of their requiremsnts is that they be able to connect to your machine to initiate a mirror sync.  You can't just run it from cron or monitor some external service.

I don't know if there's any possibility of running an unofficial mirror.  Of course, for the purposes of Fedora infrastructure, the best thing would be to just have a machine that keeps an up to date virus database using freshclam, and for these files to just get copied into the container for virus checking.  That way there's no delay at all.

If some infrastructure or testing component depends on clamav and the people involved with that component want to lend a hand with package maintenance, that would be great.  Or if they want to submit a separate package for just the virus database that updates frequently, I'm sure nobody would complain and the base clamav package could be fixed up to accommodate.

However, to be completely honest I'm not sure what actually happens if you have a machine that runs freshclam to stay up to date, and then it gets an updated clamav-data package.  If it walks you back to an older virus database then that wouldn't be particularly good.  Does anyone actually know?  The files in /var/lib/clamav aren't versioned or anything so I think they'd just get overwritten with the older versions and if so that would make a constantly-updating clamav-data package rather not a good thing.

Comment 6 Miro Hrončok 2017-09-07 20:51:25 UTC
(In reply to Jason Tibbitts from comment #5)
> However, to be completely honest I'm not sure what actually happens if you
> have a machine that runs freshclam to stay up to date, and then it gets an
> updated clamav-data package.  If it walks you back to an older virus
> database then that wouldn't be particularly good.  Does anyone actually
> know?  The files in /var/lib/clamav aren't versioned or anything so I think
> they'd just get overwritten with the older versions and if so that would
> make a constantly-updating clamav-data package rather not a good thing.

From clamav.spec:

%files data
# use %%config to keep files which were updated by 'freshclam'
# already. Without this tag, they would be overridden with older
# versions whenever a new -data package is installed.
%config %verify(not size md5 mtime) %homedir/*.cvd


I think that would work quite fine here as well.

Comment 7 Miro Hrončok 2017-09-07 21:14:00 UTC
I've created a copr demo at https://copr.fedorainfracloud.org/coprs/churchyard/clamav-data-nightly/

(The db is fetched trough build).

All that has to be done is a cron job for hitting resubmit every night and it should work like a charm.

spec: https://paste.fedoraproject.org/paste/mpGLImHrx54a4jZMyMEFPg

Comment 8 Kamil Páral 2017-09-08 11:48:00 UTC
COPR is good enough solution for our Taskotron task provided somebody is willing to volunteer to maintain it.

Comment 9 Kamil Páral 2017-09-08 11:57:48 UTC
Created attachment 1323708 [details]
clamav-data-nightly.spec

Spec file reupload from comment 7 so that it doesn't expire.

Comment 10 Sergio Basto 2017-09-11 19:12:48 UTC
(In reply to Miro Hrončok from comment #6)

> From clamav.spec:
> 
> %files data
> # use %%config to keep files which were updated by 'freshclam'
> # already. Without this tag, they would be overridden with older
> # versions whenever a new -data package is installed.
> %config %verify(not size md5 mtime) %homedir/*.cvd
> 
> 
> I think that would work quite fine here as well.

So clamav does not need any modification , isn't it ? 

(In reply to Miro Hrončok from comment #7)
> I've created a copr demo at
> https://copr.fedorainfracloud.org/coprs/churchyard/clamav-data-nightly/
> 
> (The db is fetched trough build).
> 
> All that has to be done is a cron job for hitting resubmit every night and
> it should work like a charm.
> 
> spec: https://paste.fedoraproject.org/paste/mpGLImHrx54a4jZMyMEFPg

This is a  new package we should transform it in package review request , but 
IMHO the cron should run freshclam if we have update build it in koji , something like  [1]

[1]
https://pkgs.rpmfusion.org/cgit/nonfree/lpf-spotify-client.git/tree/check_new_version.py

Comment 11 Miro Hrončok 2017-09-12 08:41:25 UTC
(In reply to Sergio Monteiro Basto from comment #10)
> This is a  new package we should transform it in package review request

Well, currently, this is impossible in Fedora (build fetches from the Internet). But if somebody wants, it can be fetched and source created, than used. I've just created it a s a demo, because I was curious how hard it would be.

Comment 12 Fedora End Of Life 2017-11-16 19:20:13 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 13 Fedora End Of Life 2017-12-12 10:18:54 UTC
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 14 Sergio Basto 2017-12-23 19:38:24 UTC
still valid

Comment 15 Sergio Basto 2018-03-12 04:06:39 UTC
(In reply to Miro Hrončok from comment #7)
> I've created a copr demo at
> https://copr.fedorainfracloud.org/coprs/churchyard/clamav-data-nightly/
> 
> (The db is fetched trough build).
> 
> All that has to be done is a cron job for hitting resubmit every night and
> it should work like a charm.

I agree and I can help in separate clamav package from clamav-data , clamav-data have 170+ MB and if painful download it and upload it each time we want update clamav source package .
And I guess major of the people will use freshclam to download data and don't need clamav data , on the other hand if we want give clamav-data updated we may update frequently clamav-data package.

Comment 16 Orion Poplawski 2019-11-24 02:23:04 UTC
Do we still need this?  I've been updating the CVD files when I've been updating the clamav package for other reasons and with more active upstream development this means they are not *too* stale.  Is this sufficient or do we still want a more regularly updated file?

Comment 17 Sergio Basto 2019-11-24 04:44:41 UTC
I have some ideas for this package clamav-data and or clamav-data-nightly. One goal is just split clamav-data from clamav and make clamav src.rpm with less 100 / 150 M of data

Comment 18 Kamil Páral 2019-11-25 08:33:21 UTC
(In reply to Orion Poplawski from comment #16)
> Do we still need this?  I've been updating the CVD files when I've been
> updating the clamav package for other reasons and with more active upstream
> development this means they are not *too* stale.  Is this sufficient or do
> we still want a more regularly updated file?

We no longer need it for Taskotron in particular, but I can't speak for other CI/testing systems.


Note You need to log in before you can comment on or make changes to this bug.