Bug 1998252

Summary:	clamav-data-0.103.3-4 provides older database than can be reasonably updated via clamav-update/freshclam
Product:	[Fedora] Fedora	Reporter:	jimbobmcgee <bugzilla.redhat.com>
Component:	clamav	Assignee:	Sergio Basto <sergio>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	34	CC:	anon.amish, bennie.joubert, hanspeter.gosteli, janfrode, j, lee.jnk, ondrejj, orion, pgnet.dev, redhat-bugzilla, rh-bugzilla, sergio, steve
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	clamav-0.103.3-5.fc34 clamav-0.103.3-5.fc33 clamav-0.103.3-5.el8 clamav-0.103.3-5.el7 clamav-0.103.3-5.fc35	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-09-04 19:31:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description jimbobmcgee 2021-08-26 17:09:59 UTC

Description of problem:
Recently updated via `dnf update`, which uplifted clamav-data from 0.103.3-1 to 0.103.3-4.  

Afterwards, attempts to `clamscan` reported that the database was older than 7 days, and `freshclam` would no longer update `/var/lib/clamav/daily.cvd` via cdiffs.

Deleting/renaming /var/lib/clamav/daily.cvd and re-running `freshclam` allows successful full download of daily.cvd from upstream, but this seems counter to the Clam team's intentions.

If daily.cvd is not deleted/renamed, `freshclam` appears never to be able to get daily.cvd up to the current version (26275, as of 28-Aug-21).


Version-Release number of selected component (if applicable):
clamav-0.103-3-4.fc34
clamav-data-0.103-3-4.fc34
daily.cvd(26209)


How reproducible:
Forcibly reinstalling clamav-data will (as of 26-Aug-21) daily.cvd back to a version that reports age>7d and does not update


Steps to Reproduce:
1. dnf reinstall clamav-data (reverts 
2. systemctl restart clamav-freshclam.service (reruns freshclam)
3. tail /var/log/freshclam.log (assumes /etc/freshclam.conf UpdateLogFile and LogVerbose)
4. repeat 2/3 as necessary

Actual results:

```
daily database available for update (local version: 26209, remote version: 26275)
Retrieving https://database.clamav.net/daily-26210.cdiff
cdiff_apply: Parsed 107 lines and executed 107 commands
Retrieving https://database.clamav.net/daily-26211.cdiff
cdiff_apply: Parsed 41 lines and executed 41 commands
...
Retrieving https://database.clamav.net/daily-26232.cdiff
ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed 
ERROR: downloadPatch: Can't apply patch 
Downloaded 22 patches for daily, which is fewer than the 66 expected patches. 
We'll settle for this partial-update, at least for now. 
...
[LibClamAV] ************************************************** 
[LibClamAV] ***  The virus database is older than 7 days!  *** 
[LibClamAV] ***   Please update it as soon as possible.    *** 
[LibClamAV] **************************************************
```

Expected results:
The database provided by clamav-data should be new enough to be further updated by `freshclam` without multiple attempts or manual intervention.

For instance, with no existing daily.cvd, output is...

```
check_for_new_database_version: No local copy of "daily" database. 
Retrieving https://database.clamav.net/daily.cvd 
The daily.cvd database downloaded from https://database.clamav.net is older than the version advertised in the DNS TXT record. 
daily.cvd updated (version: 26273, sigs: 1968259, f-level: 90, builder: raynman) 
Retrieving https://database.clamav.net/daily-26274.cdiff 
cdiff_apply: Parsed 212 lines and executed 212 commands 
Retrieving https://database.clamav.net/daily-26275.cdiff 
cdiff_apply: Parsed 265 lines and executed 265 commands
Properly loaded 1968559 signatures from /var/lib/clamav/tmp.a90089c94d/clamav-62939e8d18862b84ed5036998b9e9088.tmp-daily.cld 
Database test passed. 
daily.cld updated (version: 26275, sigs: 1968559, f-level: 90, builder: raynman) 
```

Additional info:
I think this may be to do with recent (Mar-21) rate-limits put in place by the Clam team, but I can't confirm this.  Errors received in freshclam.log appear to be the same as when manually running `freshclam` with an old database.

Not sure how viable this is to do in packaging, but maybe a pre-install script could/should check if daily.cvd is newer than the packaged version and skip it?

Or maybe I'm not supposed to have both clamav-data and clamav-update installed together, in which case could/should installing one uninstall the other?

Comment 1 Fedora Update System 2021-08-26 20:20:17 UTC

FEDORA-2021-b0aa983da1 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-b0aa983da1

Comment 2 Fedora Update System 2021-08-26 20:20:24 UTC

FEDORA-2021-095480171f has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-095480171f

Comment 3 Fedora Update System 2021-08-26 20:20:32 UTC

FEDORA-2021-25e865b1cb has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-25e865b1cb

Comment 4 Sergio Basto 2021-08-26 20:24:54 UTC

Thank you for the report let see if it is fixed

Comment 5 Fedora Update System 2021-08-27 17:25:38 UTC

FEDORA-2021-b0aa983da1 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-b0aa983da1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-b0aa983da1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2021-08-27 18:58:58 UTC

FEDORA-2021-25e865b1cb has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-25e865b1cb`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-25e865b1cb

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2021-08-27 19:36:58 UTC

FEDORA-2021-095480171f has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-095480171f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-095480171f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Sergio Basto 2021-08-28 22:59:48 UTC

you can read the logs with journalctl -u clamav-freshclam.service --since "4 days ago" , and as I suspected I think it was a mirror problem that was solved by upstream ... [1]


[1]
Aug 27 04:22:34 ideapad.local freshclam[3191892]: Received signal: wake up
Aug 27 04:22:34 ideapad.local freshclam[3191892]: ClamAV update process started at Fri Aug 27 04:22:34 2021
Aug 27 04:22:34 ideapad.local freshclam[3191892]: daily database available for update (local version: 26231, remote version: 26275)
Aug 27 04:22:35 ideapad.local freshclam[3191892]: ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed
Aug 27 04:22:35 ideapad.local freshclam[3191892]: ERROR: downloadPatch: Can't apply patch
Aug 27 04:22:35 ideapad.local freshclam[3191892]: WARNING: Incremental update failed, trying to download daily.cvd
Aug 27 04:22:42 ideapad.local freshclam[3191892]: Received an older daily CVD than was advertised. We'll retry so the incremental update will ensure we're up-to-date.
Aug 27 04:22:42 ideapad.local freshclam[3191892]: daily database available for update (local version: 26231, remote version: 26275)
Aug 27 04:22:42 ideapad.local freshclam[3191892]: ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed
Aug 27 04:22:42 ideapad.local freshclam[3191892]: ERROR: downloadPatch: Can't apply patch
Aug 27 04:22:42 ideapad.local freshclam[3191892]: WARNING: Incremental update failed, trying to download daily.cvd
Aug 27 04:22:48 ideapad.local freshclam[3191892]: Received an older daily CVD than was advertised. We'll retry so the incremental update will ensure we're up-to-date.
Aug 27 04:22:48 ideapad.local freshclam[3191892]: daily database available for update (local version: 26231, remote version: 26275)
Aug 27 04:22:49 ideapad.local freshclam[3191892]: ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed
Aug 27 04:22:49 ideapad.local freshclam[3191892]: ERROR: downloadPatch: Can't apply patch
Aug 27 04:22:49 ideapad.local freshclam[3191892]: WARNING: Incremental update failed, trying to download daily.cvd
Aug 27 04:22:55 ideapad.local freshclam[3191892]: Received an older daily CVD than was advertised. We'll retry so the incremental update will ensure we're up-to-date.
Aug 27 04:22:55 ideapad.local freshclam[3191892]: main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr)
Aug 27 04:22:55 ideapad.local freshclam[3191892]: bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Aug 27 06:22:55 ideapad.local freshclam[3191892]: Received signal: wake up
Aug 27 06:22:55 ideapad.local freshclam[3191892]: ClamAV update process started at Fri Aug 27 06:22:55 2021
Aug 27 06:22:55 ideapad.local freshclam[3191892]: daily database available for update (local version: 26231, remote version: 26275)
Aug 27 06:22:55 ideapad.local freshclam[3191892]: ERROR: cdiff_apply: lseek(desc, -350, SEEK_END) failed
Aug 27 06:22:55 ideapad.local freshclam[3191892]: ERROR: downloadPatch: Can't apply patch
Aug 27 06:22:55 ideapad.local freshclam[3191892]: WARNING: Incremental update failed, trying to download daily.cvd
Aug 27 06:23:02 ideapad.local freshclam[3191892]: Testing database: '/var/lib/clamav/tmp.e714c161c8/clamav-c2ddac76b92ade1abb036c609fd22881.tmp-daily.cvd' ...
Aug 27 06:23:06 ideapad.local freshclam[3191892]: Database test passed.
Aug 27 06:23:06 ideapad.local freshclam[3191892]: daily.cvd updated (version: 26275, sigs: 1968559, f-level: 90, builder: raynman)
Aug 27 06:23:06 ideapad.local freshclam[3191892]: main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr)
Aug 27 06:23:06 ideapad.local freshclam[3191892]: bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Aug 27 08:23:06 ideapad.local freshclam[3191892]: Received signal: wake up
Aug 27 08:23:06 ideapad.local freshclam[3191892]: ClamAV update process started at Fri Aug 27 08:23:06 2021
Aug 27 08:23:06 ideapad.local freshclam[3191892]: daily.cvd database is up-to-date (version: 26275, sigs: 1968559, f-level: 90, builder: raynman)
Aug 27 08:23:06 ideapad.local freshclam[3191892]: main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr)
Aug 27 08:23:06 ideapad.local freshclam[3191892]: bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)

Comment 9 Fedora Update System 2021-08-29 01:53:41 UTC

FEDORA-EPEL-2021-568f2e4092 has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-568f2e4092

Comment 10 Fedora Update System 2021-08-29 01:53:57 UTC

FEDORA-EPEL-2021-01e7b83241 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-01e7b83241

Comment 11 Fedora Update System 2021-08-29 18:43:46 UTC

FEDORA-EPEL-2021-568f2e4092 has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-568f2e4092

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2021-08-29 18:54:02 UTC

FEDORA-EPEL-2021-01e7b83241 has been pushed to the Fedora EPEL 7 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-01e7b83241

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2021-09-04 19:31:41 UTC

FEDORA-2021-095480171f has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 14 Fedora Update System 2021-09-04 19:35:17 UTC

FEDORA-2021-25e865b1cb has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 15 Fedora Update System 2021-09-05 20:15:38 UTC

FEDORA-EPEL-2021-568f2e4092 has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Fedora Update System 2021-09-05 20:18:47 UTC

FEDORA-EPEL-2021-01e7b83241 has been pushed to the Fedora EPEL 7 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Fedora Update System 2021-09-24 20:09:37 UTC

FEDORA-2021-b0aa983da1 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.