Bug 1935465 - Cached rpms in dnf uses historic yum checksum code instead of librepo's checksum
Summary: Cached rpms in dnf uses historic yum checksum code instead of librepo's checksum
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 35
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
Assignee: Matthew Almond
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: dnf-community
TreeView+ depends on / blocked
 
Reported: 2021-03-04 23:59 UTC by Matthew Almond
Modified: 2022-12-13 15:19 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-12-13 15:19:02 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Matthew Almond 2021-03-04 23:59:58 UTC
Description of problem: dnf has two implementations of rpm checksumming. One stemming from yum written in Python, embedded in dnf. Another in librepo which is authoritative for the normal download+install+delete flow.

Version-Release number of selected component (if applicable):

How reproducible: every time

# Direct install:

Steps to Reproduce:

1. dnf install bash --installroot=/var/tmp/direct.root --releasever=33


Expected/Actual results:

dnf downloads packages, verifies using librepo's lr_checksum_compare, and when these match, installs them.


# Cached install:
Steps to Reproduce:

1. dnf install bash --installroot=/var/tmp/cached.root  --releasever=33 --downloadonly
2. dnf install bash --installroot=/var/tmp/cached.root  --releasever=33 --cacheonly

Expected results:

Step 1 will download and verify the files, adding xattr to cache checksum
Step 2 will read checksum from xattr if present, falling back to lr_checksum_compare.

Actual results:

dnf uses dnf.yum.misc.checksum() in Package.verifyLocalPkg() which doesn't use the xattr cache. This works, but is slower, and is broken with CoW (see next bit)

Additional info:
Today this is a slight perf regression, and doesn't cause any visible failures. The long term issue is that there are two pieces of code that verify checksums and they're different. In the context of https://github.com/rpm-software-management/librepo/pull/222 this means failure because the updated code in librepo isn't being used in the lesss common path.

Comment 1 Matthew Almond 2021-03-05 00:03:12 UTC
I'm motivated to fix this. My initial thoughts are to modify librepo's python bindings, exposing lr_checksum_fd_compare + LrChecksumType from librepo/checksum.c. Then I can modify dnf/package.py to use that method directly, and delete the checksum code out of dnf/yum/misc.py

Comment 2 Matthew Almond 2021-03-11 23:19:37 UTC
Main fix:

https://github.com/rpm-software-management/libdnf/pull/1164
https://github.com/rpm-software-management/dnf/pull/1743

This replaces the old code with the librepo stuff properly.

A minor bug:
https://github.com/rpm-software-management/librepo/pull/234

Comment 4 Ben Cotton 2021-08-10 13:44:19 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

Comment 5 Ben Cotton 2022-11-29 16:53:16 UTC
This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 35 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 6 Ben Cotton 2022-12-13 15:19:02 UTC
Fedora Linux 35 entered end-of-life (EOL) status on 2022-12-13.

Fedora Linux 35 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.