Description of problem: System is not picking up the latest packages when trying to fetch the update from repository whe installing "ipa-client" and "idm:DL1" module This create problem when updating system connected to local repository created using reposync with the "--newest" option as it only contains the newest version of package. Firstly to demostrate , below output collected when the system is connected to CDN which also show this problem where new version of "sssd" and "python3-libsemanage" is not picked up by RHEL 8.7 system even when it has the latest version of DNF and livsolve. While on RHEL 8 , it able to pick latest version of "sssd" but not the latest version of "python3-libsemange" ON RHEL 8.8 system (with updated dnf/libsolv packages) ------------------- * Version of dnf/libsolve ~~~ # cat /etc/redhat-release Red Hat Enterprise Linux release 8.8 (Ootpa) # rpm -qa | grep -e dnf -e livsolv | sort dnf-4.7.0-16.el8_8.noarch dnf-data-4.7.0-16.el8_8.noarch dnf-plugins-core-4.0.21-19.el8_8.noarch dnf-plugin-subscription-manager-1.28.36-2.el8.x86_64 libdnf-0.63.0-14.el8_8.x86_64 python3-dnf-4.7.0-16.el8_8.noarch python3-dnf-plugins-core-4.0.21-19.el8_8.noarch python3-libdnf-0.63.0-14.el8_8.x86_64 ~~~ * While installing "ipa-client" in RHEL 8.8 it does pickup new "sssd" , "policycoreutils-python-utils" packages but still picks old "python3-libsemange" and "krb5-pkinit" packages. ~~~ # dnf install ipa-client --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils ipa-client x86_64 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms 288 k ipa-client-common noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms 190 k krb5-pkinit x86_64 1.18.2-22.el8_7 rhel-8-for-x86_64-baseos-rpms 176 k policycoreutils-python-utils noarch 2.9-24.el8 rhel-8-for-x86_64-baseos-rpms 254 k python3-libsemanage x86_64 2.9-9.el8 rhel-8-for-x86_64-baseos-rpms 128 k python3-sssdconfig noarch 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 149 k sssd-client x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 238 k sssd-common x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 1.7 M sssd-common-pac x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 190 k sssd-dbus x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 226 k sssd-idp x86_64 2.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 128 k sssd-ipa x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 369 k sssd-krb5-common x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 198 k sssd-tools x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 255 k sssd-nfs-idmap x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 125 k ~~~ * Here, while installing "idm:DL1" module in RHEL 8.8 it does pickup new "sssd" , , "policycoreutils-python-utils" and "krb5-pkinit" packages but still picks old "python3-libsemange" packages. ~~~ ]# dnf module install idm:DL1 --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils Operation aborted. ipa-client x86_64 4.9.11-6.module+el8.8.0+19022+e8902f4b rhel-8-for-x86_64-appstream-rpms 288 k ipa-client-common noarch 4.9.11-6.module+el8.8.0+19022+e8902f4b rhel-8-for-x86_64-appstream-rpms 190 k krb5-pkinit x86_64 1.18.2-25.el8_8 rhel-8-for-x86_64-baseos-rpms 176 k policycoreutils-python-utils noarch 2.9-24.el8 rhel-8-for-x86_64-baseos-rpms 254 k python3-libsemanage x86_64 2.9-9.el8 rhel-8-for-x86_64-baseos-rpms 128 k python3-sssdconfig noarch 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 149 k sssd-client x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 238 k sssd-common x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 1.7 M sssd-common-pac x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 190 k sssd-dbus x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 226 k sssd-idp x86_64 2.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 128 k sssd-ipa x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 369 k sssd-krb5-common x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 198 k sssd-tools x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 255 k sssd-nfs-idmap x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 125 k ~~~ * Later packges available packages in the repo. ~~~ # dnf list ipa-client sssd python3-libsemanage krb5-pkinit policycoreutils-python-utils Updating Subscription Management repositories. Last metadata expiration check: 0:01:33 ago on Thu 06 Jul 2023 08:49:29 AM EDT. Available Packages ipa-client.x86_64 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms krb5-pkinit.i686 1.18.2-25.el8_8 rhel-8-for-x86_64-baseos-rpms krb5-pkinit.x86_64 1.18.2-25.el8_8 rhel-8-for-x86_64-baseos-rpms policycoreutils-python-utils.noarch 2.9-24.el8 rhel-8-for-x86_64-baseos-rpms python3-libsemanage.x86_64 2.9-9.el8_6 rhel-8-for-x86_64-baseos-rpms sssd.x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms ~~~ * If I try specifying these pacakges on the command line then they do update to latest version. ~~~ # dnf install ipa-client sssd python3-libsemanage krb5-pkinit policycoreutils-python-utils --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils ipa-client x86_64 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms 288 k krb5-pkinit x86_64 1.18.2-25.el8_8 rhel-8-for-x86_64-baseos-rpms 176 k policycoreutils-python-utils noarch 2.9-24.el8 rhel-8-for-x86_64-baseos-rpms 254 k python3-libsemanage x86_64 2.9-9.el8_6 rhel-8-for-x86_64-baseos-rpms 128 k sssd x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 115 k ipa-client-common noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms 190 k python3-sssdconfig noarch 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 149 k sssd-ad x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 307 k sssd-client x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 238 k sssd-common x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 1.7 M sssd-common-pac x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 190 k sssd-dbus x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 226 k sssd-idp x86_64 2.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 128 k sssd-ipa x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 369 k sssd-krb5 x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 160 k sssd-krb5-common x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 198 k sssd-ldap x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 247 k sssd-tools x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 255 k sssd-nfs-idmap x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 125 k sssd-proxy x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 156 k ~~~ ON RHEL 8.7 system (with updated dnf/libsolv packages rest RHEL 8.7 patch level) ------------------- * Version of dnf/libsolve ~~~ # cat /etc/redhat-release Red Hat Enterprise Linux release 8.7 (Ootpa) # rpm -qa | grep -e dnf -e libsolv | sort dnf-4.7.0-16.el8_8.noarch dnf-data-4.7.0-16.el8_8.noarch dnf-plugins-core-4.0.21-14.1.el8.noarch dnf-plugin-subscription-manager-1.28.32-1.el8.x86_64 libdnf-0.63.0-11.1.el8.x86_64 libsolv-0.7.20-4.el8_7.x86_64 python3-dnf-4.7.0-16.el8_8.noarch python3-dnf-plugins-core-4.0.21-14.1.el8.noarch python3-libdnf-0.63.0-11.1.el8.x86_64 ~~~ * While installing "ipa-client" it pick old "sssd" , " policycoreutils-python-utils" , "krb5-pkinit" and "python3-libsemanage" packages. ~~~ # dnf install ipa-client --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils Operation aborted. ipa-client x86_64 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms 288 k ipa-client-common noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms 190 k krb5-pkinit x86_64 1.18.2-21.el8 rhel-8-for-x86_64-baseos-rpms 176 k policycoreutils-python-utils noarch 2.9-20.el8 rhel-8-for-x86_64-baseos-rpms 253 k python3-libsemanage x86_64 2.9-9.el8 rhel-8-for-x86_64-baseos-rpms 128 k python3-sssdconfig noarch 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 147 k sssd-common-pac x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 188 k sssd-dbus x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 224 k sssd-idp x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-appstream-rpms 126 k sssd-ipa x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 367 k sssd-krb5-common x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 197 k sssd-tools x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 247 k ~~~ * While installing "idm:DL1" module also it pick old "sssd" , " policycoreutils-python-utils" and "python3-libsemanage" packages but it does pick new version of "krb5-pkinit" ~~~ # dnf module install idm:DL1 --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils Operation aborted. ipa-client x86_64 4.9.11-6.module+el8.8.0+19022+e8902f4b rhel-8-for-x86_64-appstream-rpms 288 k ipa-client-common noarch 4.9.11-6.module+el8.8.0+19022+e8902f4b rhel-8-for-x86_64-appstream-rpms 190 k krb5-pkinit x86_64 1.18.2-25.el8_8 rhel-8-for-x86_64-baseos-rpms 176 k policycoreutils-python-utils noarch 2.9-20.el8 rhel-8-for-x86_64-baseos-rpms 253 k python3-libsemanage x86_64 2.9-9.el8 rhel-8-for-x86_64-baseos-rpms 128 k python3-sssdconfig noarch 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 147 k sssd-common-pac x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 188 k sssd-dbus x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 224 k sssd-idp x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-appstream-rpms 126 k sssd-ipa x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 367 k sssd-krb5-common x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 197 k sssd-tools x86_64 2.7.3-4.el8_7.1 rhel-8-for-x86_64-baseos-rpms 247 k ~~~ * Available latest packages in the repo are not being picked. ~~~ # dnf list ipa-client sssd python3-libsemanage krb5-pkinit policycoreutils-python-utils Updating Subscription Management repositories. Last metadata expiration check: 0:00:18 ago on Thu 06 Jul 2023 09:02:04 AM EDT. Available Packages ipa-client.x86_64 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms krb5-pkinit.i686 1.18.2-25.el8_8 rhel-8-for-x86_64-baseos-rpms krb5-pkinit.x86_64 1.18.2-25.el8_8 rhel-8-for-x86_64-baseos-rpms policycoreutils-python-utils.noarch 2.9-24.el8 rhel-8-for-x86_64-baseos-rpms python3-libsemanage.x86_64 2.9-9.el8_6 rhel-8-for-x86_64-baseos-rpms sssd.x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms ~~~ * If I try specifying these pacakges on the command line then they do update to latest version. ~~~ # dnf install ipa-client sssd python3-libsemanage krb5-pkinit policycoreutils-python-utils --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils Operation aborted. ipa-client x86_64 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms 288 k krb5-pkinit x86_64 1.18.2-25.el8_8 rhel-8-for-x86_64-baseos-rpms 176 k policycoreutils-python-utils noarch 2.9-24.el8 rhel-8-for-x86_64-baseos-rpms 254 k python3-libsemanage x86_64 2.9-9.el8_6 rhel-8-for-x86_64-baseos-rpms 128 k sssd x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 115 k sssd-client x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 238 k sssd-common x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 1.7 M sssd-kcm x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 259 k ipa-client-common noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 rhel-8-for-x86_64-appstream-rpms 190 k python3-sssdconfig noarch 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 149 k sssd-ad x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 307 k sssd-common-pac x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 190 k sssd-dbus x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 226 k sssd-idp x86_64 2.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 128 k sssd-ipa x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 369 k sssd-krb5 x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 160 k sssd-krb5-common x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 198 k sssd-ldap x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 247 k sssd-tools x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 255 k sssd-proxy x86_64 2.8.2-2.el8 rhel-8-for-x86_64-baseos-rpms 156 k ~~~ --- Now with system connected to CDN it fine that it is able to pull the old version of pacakge. However if the system is fetching updates form local repository which is created with reposync with the "--newest --delete" option, it fails as the old version of package is not present in the repository. Create local repository using below command which will download latest packages i.e. 8.8 package and delete oldversion of packages and shared it over HTTP, # dnf reposync -n --download-metadata -p /var/www/html/scratch --repoid=rhel-8-for-x86_64-appstream-rpms --repoid=rhel-8-for-x86_64-baseos-rpms --delete Demostating the issue with local repos created like above we see that because its trying to fetch old version of packages , it fails to find it in repo. RHEL 8.8(Local repos) (8.7 is installed with latest dnf/libsolv packages) --------------------- ~~~ # yum install ipa-client Updating Subscription Management repositories. Local Repo - rhel-8-for-x86_64-baseos-rpms 144 MB/s | 62 MB 00:00 Local Repo - rhel-8-for-x86_64-appstream-rpms 155 MB/s | 58 MB 00:00 Last metadata expiration check: 0:00:09 ago on Thu 06 Jul 2023 09:11:23 AM EDT. Dependencies resolved. ======================================================================================================================== Package Arch Version Repository Size ======================================================================================================================== Installing: ipa-client x86_64 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 288 k Installing dependencies: augeas-libs x86_64 1.12.0-8.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 437 k autofs x86_64 1:5.1.4-102.el8_8.2 localrepo-rhel-8-for-x86_64-baseos-rpms 713 k avahi-libs x86_64 0.7-20.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 62 k bind-libs x86_64 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 175 k bind-libs-lite x86_64 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 1.2 M bind-license noarch 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 104 k bind-utils x86_64 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 452 k certmonger x86_64 0.79.17-2.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 686 k cyrus-sasl-gssapi x86_64 2.1.27-6.el8_5 localrepo-rhel-8-for-x86_64-baseos-rpms 50 k fstrm x86_64 0.6.1-3.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 29 k ipa-client-common noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 190 k ipa-common noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 800 k ipa-selinux noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 182 k krb5-pkinit x86_64 1.18.2-22.el8_7 localrepo-rhel-8-for-x86_64-baseos-rpms 176 k krb5-workstation x86_64 1.18.2-22.el8_7 localrepo-rhel-8-for-x86_64-baseos-rpms 957 k libicu x86_64 60.3-2.el8_1 localrepo-rhel-8-for-x86_64-baseos-rpms 8.8 M libipa_hbac x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 123 k libjose x86_64 10-2.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 64 k libkadm5 x86_64 1.18.2-22.el8_7 localrepo-rhel-8-for-x86_64-baseos-rpms 187 k libsss_simpleifp x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 115 k libwbclient x86_64 4.17.5-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 125 k nspr x86_64 4.34.0-3.el8_6 localrepo-rhel-8-for-x86_64-appstream-rpms 143 k nss x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 747 k nss-softokn x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 1.2 M nss-softokn-freebl x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 398 k nss-sysinit x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 75 k nss-tools x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 582 k nss-util x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 139 k policycoreutils-python-utils noarch 2.9-24.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 254 k protobuf-c x86_64 1.3.0-6.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 37 k python3-augeas noarch 0.5.0-12.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 31 k python3-bind noarch 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 151 k python3-dns noarch 1.15.0-11.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 253 k python3-gssapi x86_64 1.5.1-5.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 489 k python3-ipaclient noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 696 k python3-ipalib noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 764 k python3-jwcrypto noarch 0.5.0-1.1.module+el8.7.0+15888+685a878f localrepo-rhel-8-for-x86_64-appstream-rpms 66 k python3-ldap x86_64 3.3.1-2.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 230 k python3-libipa_hbac x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 116 k python3-libsemanage x86_64 2.9-9.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 128 k python3-netaddr noarch 0.7.19-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 1.5 M python3-policycoreutils noarch 2.9-24.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 2.3 M python3-pyasn1 noarch 0.3.7-6.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 126 k python3-pyasn1-modules noarch 0.3.7-6.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 110 k python3-pyusb noarch 1.0.0-9.1.module+el8.7.0+15690+c8d45780 localrepo-rhel-8-for-x86_64-appstream-rpms 88 k python3-qrcode-core noarch 5.1-12.module+el8.1.0+4107+4a66eb87 localrepo-rhel-8-for-x86_64-appstream-rpms 46 k python3-sss x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 118 k python3-sss-murmur x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 106 k python3-sssdconfig noarch 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 149 k python3-yubico noarch 1.3.2-9.1.module+el8.7.0+15690+c8d45780 localrepo-rhel-8-for-x86_64-appstream-rpms 63 k samba-client-libs x86_64 4.17.5-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 5.0 M samba-common noarch 4.17.5-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 230 k samba-common-libs x86_64 4.17.5-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 180 k sssd-client x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 238 k sssd-common x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 1.7 M sssd-common-pac x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 190 k sssd-dbus x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 226 k sssd-idp x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 128 k sssd-ipa x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 369 k sssd-krb5-common x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 198 k sssd-tools x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 255 k xmlrpc-c x86_64 1.51.0-8.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 213 k xmlrpc-c-client x86_64 1.51.0-8.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 41 k Installing weak dependencies: sssd-nfs-idmap x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 125 k Enabling module streams: idm client Transaction Summary ======================================================================================================================== Install 65 Packages Total download size: 36 M Installed size: 120 M Is this ok [y/N]: y Downloading Packages: (1/65): avahi-libs-0.7-20.el8.x86_64.rpm 1.6 MB/s | 62 kB 00:00 (2/65): cyrus-sasl-gssapi-2.1.27-6.el8_5.x86_64.rpm 1.1 MB/s | 50 kB 00:00 (3/65): python3-dns-1.15.0-11.el8.noarch.rpm 23 MB/s | 253 kB 00:00 (4/65): xmlrpc-c-1.51.0-8.el8.x86_64.rpm 12 MB/s | 213 kB 00:00 [MIRROR] krb5-pkinit-1.18.2-22.el8_7.x86_64.rpm: Status code: 404 for http://reposync.test.local/scratch/rhel-8-for-x86_64-baseos-rpms/Packages/k/krb5-pkinit-1.18.2-22.el8_7.x86_64.rpm (IP: reposync.test.local) [MIRROR] libkadm5-1.18.2-22.el8_7.x86_64.rpm: Status code: 404 for http://reposync.test.local/scratch/rhel-8-for-x86_64-baseos-rpms/Packages/l/libkadm5-1.18.2-22.el8_7.x86_64.rpm (IP: reposync.test.local) [MIRROR] krb5-pkinit-1.18.2-22.el8_7.x86_64.rpm: Status code: 404 for http://reposync.test.local/scratch/rhel-8-for-x86_64-baseos-rpms/Packages/k/krb5-pkinit-1.18.2-22.el8_7.x86_64.rpm (IP: reposync.test.local) [MIRROR] libkadm5-1.18.2-22.el8_7.x86_64.rpm: Status code: 404 for http://reposync.test.local/scratch/rhel-8-for-x86_64-baseos-rpms/Packages/l/libkadm5-1.18.2-22.el8_7.x86_64.rpm (IP: reposync.test.local) [FAILED] krb5-pkinit-1.18.2-22.el8_7.x86_64.rpm: No more mirrors to try - All mirrors were already tried without success (6-7/65): libicu-60.3-2.el8_1.x86_64.rpm 4% [=- ] 5.5 MB/s | 1.6 MB 00:06 ETA The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'yum clean packages'. Error: Error downloading packages: krb5-pkinit-1.18.2-22.el8_7.x86_64: Cannot download, all mirrors were already tried without success ~~~ RHEL 8.7(Local repos) (8.7 is installed with latest dnf/libsolv packages) --------------------- ~~~ # yum install ipa-client Updating Subscription Management repositories. Last metadata expiration check: 0:00:26 ago on Thu 06 Jul 2023 09:06:38 AM EDT. Dependencies resolved. ======================================================================================================================== Package Arch Version Repository Size ======================================================================================================================== Installing: ipa-client x86_64 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 288 k Installing dependencies: augeas-libs x86_64 1.12.0-8.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 437 k autofs x86_64 1:5.1.4-102.el8_8.2 localrepo-rhel-8-for-x86_64-baseos-rpms 713 k avahi-libs x86_64 0.7-20.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 62 k bind-libs x86_64 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 175 k bind-libs-lite x86_64 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 1.2 M bind-license noarch 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 104 k bind-utils x86_64 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 452 k certmonger x86_64 0.79.17-2.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 686 k cyrus-sasl-gssapi x86_64 2.1.27-6.el8_5 localrepo-rhel-8-for-x86_64-baseos-rpms 50 k fstrm x86_64 0.6.1-3.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 29 k ipa-client-common noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 190 k ipa-common noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 800 k ipa-selinux noarch 4.9.10-3.module+el8.7.0+15888+685a878f localrepo-rhel-8-for-x86_64-appstream-rpms 179 k krb5-pkinit x86_64 1.18.2-21.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 176 k krb5-workstation x86_64 1.18.2-21.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 957 k libicu x86_64 60.3-2.el8_1 localrepo-rhel-8-for-x86_64-baseos-rpms 8.8 M libipa_hbac x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 122 k libjose x86_64 10-2.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 64 k libkadm5 x86_64 1.18.2-21.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 187 k libsss_simpleifp x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 113 k libwbclient x86_64 4.16.4-6.el8_7 localrepo-rhel-8-for-x86_64-baseos-rpms 124 k nspr x86_64 4.34.0-3.el8_6 localrepo-rhel-8-for-x86_64-appstream-rpms 143 k nss x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 747 k nss-softokn x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 1.2 M nss-softokn-freebl x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 398 k nss-sysinit x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 75 k nss-tools x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 582 k nss-util x86_64 3.79.0-11.el8_7 localrepo-rhel-8-for-x86_64-appstream-rpms 139 k policycoreutils-python-utils noarch 2.9-20.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 253 k protobuf-c x86_64 1.3.0-6.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 37 k python3-augeas noarch 0.5.0-12.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 31 k python3-bind noarch 32:9.11.36-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 151 k python3-dns noarch 1.15.0-11.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 253 k python3-gssapi x86_64 1.5.1-5.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 489 k python3-ipaclient noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 696 k python3-ipalib noarch 4.9.11-5.module+el8.8.0+18147+84fe6ec1 localrepo-rhel-8-for-x86_64-appstream-rpms 764 k python3-jwcrypto noarch 0.5.0-1.1.module+el8.7.0+15888+685a878f localrepo-rhel-8-for-x86_64-appstream-rpms 66 k python3-ldap x86_64 3.3.1-2.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 230 k python3-libipa_hbac x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 115 k python3-libsemanage x86_64 2.9-9.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 128 k python3-netaddr noarch 0.7.19-8.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 1.5 M python3-policycoreutils noarch 2.9-20.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 2.2 M python3-pyasn1 noarch 0.3.7-6.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 126 k python3-pyasn1-modules noarch 0.3.7-6.el8 localrepo-rhel-8-for-x86_64-appstream-rpms 110 k python3-pyusb noarch 1.0.0-9.1.module+el8.7.0+15690+c8d45780 localrepo-rhel-8-for-x86_64-appstream-rpms 88 k python3-qrcode-core noarch 5.1-12.module+el8.1.0+4107+4a66eb87 localrepo-rhel-8-for-x86_64-appstream-rpms 46 k python3-sss x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 116 k python3-sss-murmur x86_64 2.8.2-2.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 106 k python3-sssdconfig noarch 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 147 k python3-yubico noarch 1.3.2-9.1.module+el8.7.0+15690+c8d45780 localrepo-rhel-8-for-x86_64-appstream-rpms 63 k samba-client-libs x86_64 4.16.4-6.el8_7 localrepo-rhel-8-for-x86_64-baseos-rpms 5.0 M samba-common noarch 4.16.4-6.el8_7 localrepo-rhel-8-for-x86_64-baseos-rpms 228 k samba-common-libs x86_64 4.16.4-6.el8_7 localrepo-rhel-8-for-x86_64-baseos-rpms 179 k sssd-common-pac x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 188 k sssd-dbus x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 224 k sssd-idp x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-appstream-rpms 126 k sssd-ipa x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 367 k sssd-krb5-common x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 197 k sssd-tools x86_64 2.7.3-4.el8_7.1 localrepo-rhel-8-for-x86_64-baseos-rpms 247 k xmlrpc-c x86_64 1.51.0-8.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 213 k xmlrpc-c-client x86_64 1.51.0-8.el8 localrepo-rhel-8-for-x86_64-baseos-rpms 41 k Enabling module streams: idm client Transaction Summary ======================================================================================================================== Install 62 Packages Total download size: 34 M Installed size: 115 M Is this ok [y/N]: y Downloading Packages: (1/62): avahi-libs-0.7-20.el8.x86_64.rpm 6.3 MB/s | 62 kB 00:00 (2/62): cyrus-sasl-gssapi-2.1.27-6.el8_5.x86_64.rpm 3.2 MB/s | 50 kB 00:00 [MIRROR] policycoreutils-python-utils-2.9-20.el8.noarch.rpm: Status code: 404 for http://reposync.test.local/scratch/rhel-8-for-x86_64-baseos-rpms/Packages/p/policycoreutils-python-utils-2.9-20.el8.noarch.rpm (IP: reposync.test.local) [MIRROR] krb5-workstation-1.18.2-21.el8.x86_64.rpm: Status code: 404 for http://reposync.test.local/scratch/rhel-8-for-x86_64-baseos-rpms/Packages/k/krb5-workstation-1.18.2-21.el8.x86_64.rpm (IP: reposync.test.local) [MIRROR] policycoreutils-python-utils-2.9-20.el8.noarch.rpm: Status code: 404 for http://reposync.test.local/scratch/rhel-8-for-x86_64-baseos-rpms/Packages/p/policycoreutils-python-utils-2.9-20.el8.noarch.rpm (IP: reposync.test.local) [MIRROR] krb5-workstation-1.18.2-21.el8.x86_64.rpm: Status code: 404 for http://reposync.test.local/scratch/rhel-8-for-x86_64-baseos-rpms/Packages/k/krb5-workstation-1.18.2-21.el8.x86_64.rpm (IP: reposync.test.local) [FAILED] krb5-workstation-1.18.2-21.el8.x86_64.rpm: No more mirrors to try - All mirrors were already tried without success (4-5/62): libicu-60.3-2.el8_1.x86_64.rpm 14% [=====- ] 19 MB/s | 4.9 MB 00:01 ETA The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'yum clean packages'. Error: Error downloading packages: krb5-workstation-1.18.2-21.el8.x86_64: Cannot download, all mirrors were already tried without success ~~~ The user is pursuving this option i.e. reposync with "--newest --delete" because their system are offline and they need to ship the repo to their offline location and want to use a small size beacuse they have restirction on size and can't ship entire repo with all packages. Version-Release number of selected component (if applicable): # rpm -aq | grep -e dnf -e libsolv | sort dnf-4.7.0-16.el8_8.noarch dnf-data-4.7.0-16.el8_8.noarch dnf-plugins-core-4.0.21-19.el8_8.noarch dnf-plugin-subscription-manager-1.28.36-2.el8.x86_64 libdnf-0.63.0-14.el8_8.x86_64 libsolv-0.7.20-4.el8.x86_64 python3-dnf-4.7.0-16.el8_8.noarch python3-dnf-plugins-core-4.0.21-19.el8_8.noarch python3-libdnf-0.63.0-14.el8_8.x86_64 How reproducible: Everytime Steps to Reproduce: 1. Install a fresh minimal install RHEL 8.7 and RHEL 8.8 system and subscribe to CDN. 2. Remove packages for "ipa-client", "sssd" , "python3-libsemanage" , "krb5-pkinit" or "policycoreutils-python-utils" if they are already present as we want to see these being pulled in dependencies when they are not installed by default. # dnf remove ipa-client sssd python3-libsemanage krb5-pkinit policycoreutils-python-utils 3. Update "dnf" and "libsolv" to the latest version. # dnf update dnf livsolv 4. Try installing "ipa-client" or "idm:DL1" module and see the dependecies it pick will be the old ones. # dnf install ipa-client --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils # dnf module install idm:DL1 --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils # dnf list ipa-client sssd python3-libsemanage krb5-pkinit policycoreutils-python-utils # dnf install ipa-client sssd python3-libsemanage krb5-pkinit policycoreutils-python-utils --assumeno | grep -i -e ipa-client -e sssd -e python3-libsemanage -e krb5-pkinit -e policycoreutils-python-utils Actual results: Can't explain the behavior of dnf as to why it pick old packages sometime. Also behavior is different in RHEL 8.7 and RHEL 8.8 with same version of dnf. We expect it pick newest available package in the repository. Expected results: We expect that it will pull the latest package available form the repository. This will help in uses cases where local repository is setup with reposync with "--newest" and "--delete" option so as to keep the size of the repo small. Additional info: Earlier we had this libsolv bug for a similar issue but it was resolved and hence I have used newest available libsolv in test on RHEL 8.7 and RHEL 8.8 but still see old package being pulled. https://bugzilla.redhat.com/show_bug.cgi?id=2150300
As I wrote in the bz2150300 you linked, there is no guarantee that dnf (libsolv) will use the latest versions of dependencies. We have the `best` option which guarantees use of the newest available version but only for the packages directly requested (not dependencies), that is why it works if you specify the packages on command line. We would like if the latest dependencies were always used so we will likely report this to libsolv but even if they fix it this is just one case. Since there is no rule for this it might happen again with different set of packages or conditions. Furthermore the users approach is not ideal. They are basically using a repository with missing packages. Even if libsolv always picked the latest possible packages it can still lead to problems. For example when some package requires older version of a library but cannot use the latest one. In general when a repository changes (packages are removed) repodata have to be updated accordingly. I would recommend running `createrepo_c --update` on the downloaded repo. It should fix the issues reported in this bz because the missing packages will not be taken into account anymore. Note that this is not a perfect solution, createrepo_c doesn't update modular metadata so there still might be problems.
###Reproducer Steps Simplified### * Install ipa-client, then perform a yum update to identify dependencies that were just pulled that have updates. ~~~ # yum install ipa-client -y # yum update python3-libsemanage ========================================================================================================== Package Architecture Version Repository Size ========================================================================================================== Upgrading: libsemanage x86_64 2.9-9.el8_6 rhel-8Server-baseos 168 k python3-libsemanage x86_64 2.9-9.el8_6 rhel-8Server-baseos 128 k Transaction Summary ========================================================================================================== Upgrade 2 Packages ~~~ ____ Using reposync to resolve this issue is not a good resolution as this undoes the entire point of reposync with --download-repodata which is a single command to build a fully functional repository. libsolv should not be grabbing older packages to resolve dependencies. This would mean that immediately after a `yum install X`, you have to run `yum update` which is not expected behaviour. This would also negatively impact negatively impact customers who build systems and deploy them. After installation, they will already have to perform a yum update. Any system that registered during installation, and then that as their deployment strategy, and also used check-update would cause a case to be opened. For Bug 2150300, it was brought up that this is valid from libsolv's point of view, and then it was determined that we needed to revert the patch as it was causing problems with customers deploying images. > The proposed workaround does not seem to me like a viable solution to the problem. Currently many Red Hat layered products are blocked from releasing updated images, including updates with new CVE fixes. > The regression here causes non-default dependencies to be included in the installation path. This results in container builds that (at compose) will have a failing letter grade. The impact is already being seen by the internal teams, and anticipated to grow dramatically if not resolved as soon as possible. It seems it was reintroduced but we need some way to prevent this from happening.