Bug 1898293

Summary: repomanage --old does not list the oldest package per module
Product: Red Hat Enterprise Linux 8 Reporter: jcastran
Component: yumAssignee: amatej
Status: CLOSED ERRATA QA Contact: Eva Mrakova <emrakova>
Severity: medium Docs Contact:
Priority: medium    
Version: 8.3CC: amatej, Christopher.Blake, henson, james.antill, kwalker, phil.seeley, pkratoch
Target Milestone: rcKeywords: Triaged
Target Release: 8.0   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: dnf-plugins-core-4.0.21-2.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-09 19:52:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description jcastran 2020-11-16 18:36:33 UTC
Description of problem:
repomanage --old should show the oldest packages in the specified directory/repository. It currently only shows 1 package, even when multiple modules are available.

# find . -name perl-5*
  ./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.26.3-416.el8.x86_64.rpm
  ./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-404.module+el8.1.0+2926+ce7246ad.x86_64.rpm
  ./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.30.1-451.module+el8.3.0+6961+31ca2e7a.x86_64.rpm

# repomanage  --old  rhel-8-for-x86_64-appstream-rpms/ | grep p/perl-5
  Added repomanage_repo repo from rhel-8-for-x86_64-appstream-rpms/
  rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-404.module+el8.1.0+2926+ce7246ad.x86_64.rpm

--old should show me the oldest version of a package per module.

Version-Release number of selected component (if applicable):
yum-utils-4.0.17-5.el8.noarch

How reproducible:
Everytime

Steps to Reproduce:
1. reposync --download-metadata -n --repo rhel-8-for-x86_64-appstream-rpms
2. repomanage  --old  rhel-8-for-x86_64-appstream-rpms/ | grep p/perl-5
3.

Actual results:
Only a single package of the lowest version is shown:

rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-404.module+el8.1.0+2926+ce7246ad.x86_64.rpm


Expected results:
The oldest package from each module AND, if applicable, non modular content should be shown. 

./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.26.3-416.el8.x86_64.rpm
./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-404.module+el8.1.0+2926+ce7246ad.x86_64.rpm
./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.30.1-451.module+el8.3.0+6961+31ca2e7a.x86_64.rpm



Additional info:
Previously addressed in bug 1804720 but not resolved as expected.

repomanage --new works correctly showing 1 package of each module.

# repomanage  --new rhel-8-for-x86_64-appstream-rpms/ | grep p/perl-5
  Added repomanage_repo repo from rhel-8-for-x86_64-appstream-rpms/
  rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-403.module+el8+2770+c759b41a.x86_64.rpm
  rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.26.3-416.el8.x86_64.rpm
  rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.30.1-451.module+el8.3.0+6961+31ca2e7a.x86_64.rpm

Comment 1 amatej 2020-11-26 14:58:42 UTC
Ok there is actually several problems here.

1. When you run the reposync with -n (newest packages only) you also download metadata, but these are metadata for the whole repository not just the newest packages. Given that repomanage needs to work with modules it uses the metadata, this basically means that it currently doesn't matter what rpms you have downloaded, if you download the repodata for the whole repository repomanage works as if you had downloaded the whole repository. For example when running with --new there is rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-403.module+el8+2770+c759b41a.x86_64.rpm but this rpm was not downloaded (its missing in the  # find . -name perl-5* output). It is not clear to me how (if) this should be fixed. There is kind of missing createrepo step.

2. Regarding the actual output
> Expected results:
> The oldest package from each module AND, if applicable, non modular content should be shown. 
>
> ./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.26.3-416.el8.x86_64.rpm
> ./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-404.module+el8.1.0+2926+ce7246ad.x86_64.rpm
> ./rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.30.1-451.module+el8.3.0+6961+31ca2e7a.x86_64.rpm

The --old option is probably not documented enough, but it is showing only older packages not oldest. Meaning if there is only one package in a stream it is not older its the newest package and it doesn't get shown. However there is a bug, it should not be showing rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-404.module+el8.1.0+2926+ce7246ad.x86_64.rpm but instead rhel-8-for-x86_64-appstream-rpms/Packages/p/perl-5.24.4-403.module+el8+2770+c759b41a.x86_64.rpm.

This should be addressed in a PR: https://github.com/rpm-software-management/dnf-plugins-core/pull/414 (together with another caching issue that may be causing problems on multiple consecutive runs of repomanage)

3. Finally I would like to note that there is a general problem of how to figure out the package order in a stream, it is not easy to tell which packages are newest and which are older. Repomanage with the mentioned PR fix is showing rpms from the latest versions of a module streams but these might not be the rpms with the latest nevras. It is a similar issue to this reposync bz: https://bugzilla.redhat.com/show_bug.cgi?id=1833074

Comment 2 Kyle Walker 2021-06-07 17:52:46 UTC
@amatej
I just noticed that the rebase from https://bugzilla.redhat.com/show_bug.cgi?id=1951414 is pulling in the upstream PR you mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1898293#c1. That should cover the majority of the issues here.

Want to track this as resolved via that rebase?

Comment 3 amatej 2021-06-08 11:26:23 UTC
I wasn't sure it sufficiently fixes the customer issue but looking at this now I made another PR: https://github.com/rpm-software-management/dnf-plugins-core/pull/431 that enhances repomanage documentation to better reflect the actual behavior (it also allows running repomanage with just repodata). 

I would prefer if we also backported this new PR to resolve this, otherwise it seems kind of incomplete to me.

Comment 4 amatej 2021-06-14 08:24:00 UTC
My new PR was merged.

Linking also some older test updates related to this: https://github.com/rpm-software-management/ci-dnf-stack/pull/913

Comment 14 errata-xmlrpc 2021-11-09 19:52:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: dnf security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4464