Bug 1998252 - clamav-data-0.103.3-4 provides older database than can be reasonably updated via clamav-update/freshclam
Summary: clamav-data-0.103.3-4 provides older database than can be reasonably updated ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: clamav
Version: 34
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Sergio Basto
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-08-26 17:09 UTC by jimbobmcgee
Modified: 2021-09-24 20:09 UTC (History)
13 users (show)

Fixed In Version: clamav-0.103.3-5.fc34 clamav-0.103.3-5.fc33 clamav-0.103.3-5.el8 clamav-0.103.3-5.el7 clamav-0.103.3-5.fc35
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-09-04 19:31:41 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description jimbobmcgee 2021-08-26 17:09:59 UTC
Description of problem:
Recently updated via `dnf update`, which uplifted clamav-data from 0.103.3-1 to 0.103.3-4.  

Afterwards, attempts to `clamscan` reported that the database was older than 7 days, and `freshclam` would no longer update `/var/lib/clamav/daily.cvd` via cdiffs.

Deleting/renaming /var/lib/clamav/daily.cvd and re-running `freshclam` allows successful full download of daily.cvd from upstream, but this seems counter to the Clam team's intentions.

If daily.cvd is not deleted/renamed, `freshclam` appears never to be able to get daily.cvd up to the current version (26275, as of 28-Aug-21).


Version-Release number of selected component (if applicable):
clamav-0.103-3-4.fc34
clamav-data-0.103-3-4.fc34
daily.cvd(26209)


How reproducible:
Forcibly reinstalling clamav-data will (as of 26-Aug-21) daily.cvd back to a version that reports age>7d and does not update


Steps to Reproduce:
1. dnf reinstall clamav-data (reverts 
2. systemctl restart clamav-freshclam.service (reruns freshclam)
3. tail /var/log/freshclam.log (assumes /etc/freshclam.conf UpdateLogFile and LogVerbose)
4. repeat 2/3 as necessary

Actual results:

```
daily database available for update (local version: 26209, remote version: 26275)
Retrieving https://database.clamav.net/daily-26210.cdiff
cdiff_apply: Parsed 107 lines and executed 107 commands
Retrieving https://database.clamav.net/daily-26211.cdiff
cdiff_apply: Parsed 41 lines and executed 41 commands
...
Retrieving https://database.clamav.net/daily-26232.cdiff
ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed 
ERROR: downloadPatch: Can't apply patch 
Downloaded 22 patches for daily, which is fewer than the 66 expected patches. 
We'll settle for this partial-update, at least for now. 
...
[LibClamAV] ************************************************** 
[LibClamAV] ***  The virus database is older than 7 days!  *** 
[LibClamAV] ***   Please update it as soon as possible.    *** 
[LibClamAV] **************************************************
```

Expected results:
The database provided by clamav-data should be new enough to be further updated by `freshclam` without multiple attempts or manual intervention.

For instance, with no existing daily.cvd, output is...

```
check_for_new_database_version: No local copy of "daily" database. 
Retrieving https://database.clamav.net/daily.cvd 
The daily.cvd database downloaded from https://database.clamav.net is older than the version advertised in the DNS TXT record. 
daily.cvd updated (version: 26273, sigs: 1968259, f-level: 90, builder: raynman) 
Retrieving https://database.clamav.net/daily-26274.cdiff 
cdiff_apply: Parsed 212 lines and executed 212 commands 
Retrieving https://database.clamav.net/daily-26275.cdiff 
cdiff_apply: Parsed 265 lines and executed 265 commands
Properly loaded 1968559 signatures from /var/lib/clamav/tmp.a90089c94d/clamav-62939e8d18862b84ed5036998b9e9088.tmp-daily.cld 
Database test passed. 
daily.cld updated (version: 26275, sigs: 1968559, f-level: 90, builder: raynman) 
```

Additional info:
I think this may be to do with recent (Mar-21) rate-limits put in place by the Clam team, but I can't confirm this.  Errors received in freshclam.log appear to be the same as when manually running `freshclam` with an old database.

Not sure how viable this is to do in packaging, but maybe a pre-install script could/should check if daily.cvd is newer than the packaged version and skip it?

Or maybe I'm not supposed to have both clamav-data and clamav-update installed together, in which case could/should installing one uninstall the other?

Comment 1 Fedora Update System 2021-08-26 20:20:17 UTC
FEDORA-2021-b0aa983da1 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-b0aa983da1

Comment 2 Fedora Update System 2021-08-26 20:20:24 UTC
FEDORA-2021-095480171f has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-095480171f

Comment 3 Fedora Update System 2021-08-26 20:20:32 UTC
FEDORA-2021-25e865b1cb has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-25e865b1cb

Comment 4 Sergio Basto 2021-08-26 20:24:54 UTC
Thank you for the report let see if it is fixed

Comment 5 Fedora Update System 2021-08-27 17:25:38 UTC
FEDORA-2021-b0aa983da1 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-b0aa983da1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-b0aa983da1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2021-08-27 18:58:58 UTC
FEDORA-2021-25e865b1cb has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-25e865b1cb`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-25e865b1cb

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2021-08-27 19:36:58 UTC
FEDORA-2021-095480171f has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-095480171f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-095480171f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Sergio Basto 2021-08-28 22:59:48 UTC
you can read the logs with journalctl -u clamav-freshclam.service --since "4 days ago" , and as I suspected I think it was a mirror problem that was solved by upstream ... [1]


[1]
Aug 27 04:22:34 ideapad.local freshclam[3191892]: Received signal: wake up
Aug 27 04:22:34 ideapad.local freshclam[3191892]: ClamAV update process started at Fri Aug 27 04:22:34 2021
Aug 27 04:22:34 ideapad.local freshclam[3191892]: daily database available for update (local version: 26231, remote version: 26275)
Aug 27 04:22:35 ideapad.local freshclam[3191892]: ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed
Aug 27 04:22:35 ideapad.local freshclam[3191892]: ERROR: downloadPatch: Can't apply patch
Aug 27 04:22:35 ideapad.local freshclam[3191892]: WARNING: Incremental update failed, trying to download daily.cvd
Aug 27 04:22:42 ideapad.local freshclam[3191892]: Received an older daily CVD than was advertised. We'll retry so the incremental update will ensure we're up-to-date.
Aug 27 04:22:42 ideapad.local freshclam[3191892]: daily database available for update (local version: 26231, remote version: 26275)
Aug 27 04:22:42 ideapad.local freshclam[3191892]: ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed
Aug 27 04:22:42 ideapad.local freshclam[3191892]: ERROR: downloadPatch: Can't apply patch
Aug 27 04:22:42 ideapad.local freshclam[3191892]: WARNING: Incremental update failed, trying to download daily.cvd
Aug 27 04:22:48 ideapad.local freshclam[3191892]: Received an older daily CVD than was advertised. We'll retry so the incremental update will ensure we're up-to-date.
Aug 27 04:22:48 ideapad.local freshclam[3191892]: daily database available for update (local version: 26231, remote version: 26275)
Aug 27 04:22:49 ideapad.local freshclam[3191892]: ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed
Aug 27 04:22:49 ideapad.local freshclam[3191892]: ERROR: downloadPatch: Can't apply patch
Aug 27 04:22:49 ideapad.local freshclam[3191892]: WARNING: Incremental update failed, trying to download daily.cvd
Aug 27 04:22:55 ideapad.local freshclam[3191892]: Received an older daily CVD than was advertised. We'll retry so the incremental update will ensure we're up-to-date.
Aug 27 04:22:55 ideapad.local freshclam[3191892]: main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr)
Aug 27 04:22:55 ideapad.local freshclam[3191892]: bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Aug 27 06:22:55 ideapad.local freshclam[3191892]: Received signal: wake up
Aug 27 06:22:55 ideapad.local freshclam[3191892]: ClamAV update process started at Fri Aug 27 06:22:55 2021
Aug 27 06:22:55 ideapad.local freshclam[3191892]: daily database available for update (local version: 26231, remote version: 26275)
Aug 27 06:22:55 ideapad.local freshclam[3191892]: ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed
Aug 27 06:22:55 ideapad.local freshclam[3191892]: ERROR: downloadPatch: Can't apply patch
Aug 27 06:22:55 ideapad.local freshclam[3191892]: WARNING: Incremental update failed, trying to download daily.cvd
Aug 27 06:23:02 ideapad.local freshclam[3191892]: Testing database: '/var/lib/clamav/tmp.e714c161c8/clamav-c2ddac76b92ade1abb036c609fd22881.tmp-daily.cvd' ...
Aug 27 06:23:06 ideapad.local freshclam[3191892]: Database test passed.
Aug 27 06:23:06 ideapad.local freshclam[3191892]: daily.cvd updated (version: 26275, sigs: 1968559, f-level: 90, builder: raynman)
Aug 27 06:23:06 ideapad.local freshclam[3191892]: main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr)
Aug 27 06:23:06 ideapad.local freshclam[3191892]: bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Aug 27 08:23:06 ideapad.local freshclam[3191892]: Received signal: wake up
Aug 27 08:23:06 ideapad.local freshclam[3191892]: ClamAV update process started at Fri Aug 27 08:23:06 2021
Aug 27 08:23:06 ideapad.local freshclam[3191892]: daily.cvd database is up-to-date (version: 26275, sigs: 1968559, f-level: 90, builder: raynman)
Aug 27 08:23:06 ideapad.local freshclam[3191892]: main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr)
Aug 27 08:23:06 ideapad.local freshclam[3191892]: bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)

Comment 9 Fedora Update System 2021-08-29 01:53:41 UTC
FEDORA-EPEL-2021-568f2e4092 has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-568f2e4092

Comment 10 Fedora Update System 2021-08-29 01:53:57 UTC
FEDORA-EPEL-2021-01e7b83241 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-01e7b83241

Comment 11 Fedora Update System 2021-08-29 18:43:46 UTC
FEDORA-EPEL-2021-568f2e4092 has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-568f2e4092

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2021-08-29 18:54:02 UTC
FEDORA-EPEL-2021-01e7b83241 has been pushed to the Fedora EPEL 7 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-01e7b83241

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2021-09-04 19:31:41 UTC
FEDORA-2021-095480171f has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 14 Fedora Update System 2021-09-04 19:35:17 UTC
FEDORA-2021-25e865b1cb has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 15 Fedora Update System 2021-09-05 20:15:38 UTC
FEDORA-EPEL-2021-568f2e4092 has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Fedora Update System 2021-09-05 20:18:47 UTC
FEDORA-EPEL-2021-01e7b83241 has been pushed to the Fedora EPEL 7 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Fedora Update System 2021-09-24 20:09:37 UTC
FEDORA-2021-b0aa983da1 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.