Bug 2178126

Summary: dnf5 download strange and non-deterministic output when packages available in multiple repos
Product: [Fedora] Fedora Reporter: Eva Mrakova <emrakova>
Component: dnf5Assignee: Nicola Sella <nsella>
Status: NEW --- QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 38CC: amatej, jkolarik, nsella, pkratoch, rpm-software-management
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eva Mrakova 2023-03-14 11:33:35 UTC 
Description of problem:
'dnf5 download rpm-ost\*' provides non-deterministic and confusing output when the same versions of packages are available in multiple repos
 
Version-Release number of selected component (if applicable):
dnf5-5.0.6-2.fc38.x86_64
dnf5-5.0.7-20230314005057.17.g160033cc.fc38.x86_64 (currently latest in dnf5-unstable)

How reproducible:
always when run several times (remove downloaded packages between runs)

Steps to Reproduce:
1. dnf5 download rpm-ost\* 
   (have rpm-ost* packages in the same versions in multiple repos)

Actual results:
various, when run several times:
e.g.
# dnf5 download rpm-ost\*
Updating and loading repositories:
Repositories loaded.
Downloading Packages:
[ 1/10] rpm-ostree-devel-0:2023.2-1.fc38.i686              100% |  46.1 KiB/s |  38.1 KiB |  00m01s
[ 2/10] rpm-ostree-libs-0:2023.2-1.fc38.i686               100% | 691.4 KiB/s |  26.3 KiB |  00m00s
[ 3/10] rpm-ostree-devel-0:2023.2-1.fc38.x86_64            100% |  43.9 KiB/s |  38.1 KiB |  00m01s
[ 4/10] rpm-ostree-libs-0:2023.2-1.fc38.x86_64             100% | 457.2 KiB/s |  26.1 KiB |  00m00s
[ 5/10] rpm-ostree-0:2023.2-1.fc38.x86_64                  100% |   3.8 MiB/s |   4.0 MiB |  00m01s
[ 6/10] rpm-ostree-devel-0:2023.2-1.fc38.i686              100% | 241.1 KiB/s |  38.1 KiB |  00m00s
[ 7/10] rpm-ostree-devel-0:2023.2-1.fc38.x86_64            100% | 356.3 KiB/s |  38.1 KiB |  00m00s
[ 8/10] rpm-ostree-libs-0:2023.2-1.fc38.i686               100% | 316.5 KiB/s |  26.3 KiB |  00m00s
[ 9/10] rpm-ostree-0:2023.2-1.fc38.x86_64                  100% |  11.4 MiB/s |   4.0 MiB |  00m00s
[10/10] rpm-ostree-libs-0:2023.2-1.fc38.x86_64             100% | 110.9 KiB/s |  26.1 KiB |  00m00s
---------------------------------------------------------------------------------------------------
[10/10] Total                                              100% |   1.0 MiB/s |   8.3 MiB |  00m08s

or:
# dnf5 download rpm-ost\*
Updating and loading repositories:
Repositories loaded.
Downloading Packages:
[ 1/10] rpm-ostree-devel-0:2023.2-1.fc38.x86_64            100% | 370.2 KiB/s |  38.1 KiB |  00m00s
[ 2/10] rpm-ostree-devel-0:2023.2-1.fc38.i686              100% | 325.6 KiB/s |  38.1 KiB |  00m00s
[ 3/10] rpm-ostree-libs-0:2023.2-1.fc38.i686               100% | 796.1 KiB/s |  26.3 KiB |  00m00s
[ 4/10] rpm-ostree-libs-0:2023.2-1.fc38.x86_64             100% | 814.4 KiB/s |  26.1 KiB |  00m00s
[ 5/10] rpm-ostree-devel-0:2023.2-1.fc38.i686              100% | 418.6 KiB/s |  38.1 KiB |  00m00s
[ 6/10] rpm-ostree-devel-0:2023.2-1.fc38.x86_64            100% | 586.6 KiB/s |  38.1 KiB |  00m00s
[ 7/10] rpm-ostree-libs-0:2023.2-1.fc38.i686               100% | 525.4 KiB/s |  26.3 KiB |  00m00s
[ 8/10] rpm-ostree-0:2023.2-1.fc38.x86_64                  100% |  10.1 MiB/s |   4.0 MiB |  00m00s
>>> Downloading successful, but checksum doesn't match. Calculated: 85a48a2d00cd82f6d43607825c6d2f6
[ 9/10] rpm-ostree-libs-0:2023.2-1.fc38.x86_64             100% | 606.0 KiB/s |  26.1 KiB |  00m00s
[10/10] rpm-ostree-0:2023.2-1.fc38.x86_64                  100% | 383.5 KiB/s |   4.0 MiB |  00m11s
>>> Interrupted by header callback: Inconsistent server data, reported file Content-Length: 4139007
---------------------------------------------------------------------------------------------------
[10/10] Total                                              100% | 490.7 KiB/s |   8.3 MiB |  00m17s

Every time, five packages were downloaded:
# ll rpm-os*
-rw-r--r--. 1 root root 4233215 Mar 14 03:55 rpm-ostree-2023.2-1.fc38.x86_64.rpm
-rw-r--r--. 1 root root   39006 Mar 14 03:55 rpm-ostree-devel-2023.2-1.fc38.i686.rpm
-rw-r--r--. 1 root root   39042 Mar 14 03:55 rpm-ostree-devel-2023.2-1.fc38.x86_64.rpm
-rw-r--r--. 1 root root   26902 Mar 14 03:55 rpm-ostree-libs-2023.2-1.fc38.i686.rpm
-rw-r--r--. 1 root root   26685 Mar 14 03:55 rpm-ostree-libs-2023.2-1.fc38.x86_64.rpm


Expected results:
what dnf provides on the same machine (so, not pretending that there are 10 files when there are only 5):
# dnf download rpm-ost\*
(1/5): rpm-ostree-devel-2023.2-1.fc38.i686.rpm  114 kB/s |  38 kB     00:00    
(2/5): rpm-ostree-libs-2023.2-1.fc38.i686.rpm   337 kB/s |  26 kB     00:00    
(3/5): rpm-ostree-devel-2023.2-1.fc38.x86_64.rp  85 kB/s |  38 kB     00:00    
(4/5): rpm-ostree-libs-2023.2-1.fc38.x86_64.rpm 207 kB/s |  26 kB     00:00    
(5/5): rpm-ostree-2023.2-1.fc38.x86_64.rpm      4.8 MB/s | 4.0 MB     00:00    


Additional info:
* on my f38 machine the packages are available in fedora and updates-testing repos
* related: dnf5 repoquery provides list of duplicates (while dnf not):
# dnf5 repoquery rpm-ostree\*
Updating and loading repositories:
Repositories loaded.
rpm-ostree-0:2023.2-1.fc38.x86_64
rpm-ostree-devel-0:2023.2-1.fc38.i686
rpm-ostree-devel-0:2023.2-1.fc38.x86_64
rpm-ostree-libs-0:2023.2-1.fc38.i686
rpm-ostree-libs-0:2023.2-1.fc38.x86_64
rpm-ostree-0:2023.2-1.fc38.x86_64
rpm-ostree-devel-0:2023.2-1.fc38.i686
rpm-ostree-devel-0:2023.2-1.fc38.x86_64
rpm-ostree-libs-0:2023.2-1.fc38.i686
rpm-ostree-libs-0:2023.2-1.fc38.x86_64