RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1865803 - [RFE] allow dnf to run with solv* files only
Summary: [RFE] allow dnf to run with solv* files only
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: dnf
Version: 8.2
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 8.0
Assignee: amatej
QA Contact: Eva Mrakova
Mariya Pershina
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-08-04 08:26 UTC by Michael Mráka
Modified: 2021-05-18 15:00 UTC (History)
4 users (show)

Fixed In Version: dnf-4.4.2-9.el8
Doc Type: Enhancement
Doc Text:
.New `fill_sack_from_repos_in_cache` function is now supported in DNF API With this update, the new DNF API `fill_sack_from_repos_in_cache` function has been introduced which allows to load repositories only from the cached `solv`, `solvx` files, and the `repomd.xml` file. As a result, if the user manages `dnf` cache, it is possible to save resources without having duplicate information (`xml` and `solv`), and without processing `xml` into `solv`.
Clone Of:
Environment:
Last Closed: 2021-05-18 15:00:36 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Michael Mráka 2020-08-04 08:26:33 UTC 
Description of problem:
When using python3-dnf API in cloud environment we would save a lot of time and disk space if we can pre-build solv* files (from repo xml metadata) once and then spin multiple 'workers" which use just solv* (without trying to rebuild them from original xmls).

Version-Release number of selected component (if applicable):
dnf-4.2.17-7.el8_2.noarch
dnf-4.2.23-1.git.9175.126d861.el8.noarch

How reproducible:


Steps to Reproduce:
E.g.
1. dnf list --disablerepo=* --repofrompath centos-base,http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/ 
2. rm -rf /var/cache/dnf/centos-base-7ce9369cf52b8f9b/
3. dnf list -C --disablerepo=* --repofrompath centos-base,http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/

Actual results:
Added centos-base repo from http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/
Error: Cache-only enabled but no cache for 'centos-base'


Expected results:
List available packages because solv* file actually are in cache.

Additional info:
Comment 5 amatej 2020-11-03 13:23:31 UTC 
I have made a PR: https://github.com/rpm-software-management/libdnf/pull/1076 and tests: https://github.com/rpm-software-management/ci-dnf-stack/pull/905

It should work according to comment 3, If the repomd.xml and solv files are kept dnf can run without downloading the xml metadata.

If only the solv files are kept and repomd is also deleted we have to download the whole repository, this could be fixed so that only repomd is downloaded but it would require big change in librepo and since comment 3 mentions its ok to keep repomd I left it like this.

filenames.solvx is optional, dnf will work without it (so it works only with repomd and main solv file) but the dependency resolution may fail in some cases where it would succeed with the filenames.solvx present. In general running without the filenames.solvx is problematic (more details here: https://bugzilla.redhat.com/show_bug.cgi?id=1619368#c3) and I am not sure we should allow it but since it has to be explicitly deleted its probably fine.
Comment 6 Michael Mráka 2020-11-03 18:32:53 UTC 
That's great.
Thank you Ales.
Comment 7 Neal Gompa 2020-11-04 19:05:52 UTC 
So this is somewhat flawed for cloud images, especially if zck repodata is used, since there's a significant performance improvement if the existing zck repodata files are still preserved. This doesn't currently matter for RHEL, but I could see this becoming a useful optimization in the future.
Comment 8 amatej 2020-12-10 14:24:37 UTC 
After some internal discussion we decided that my original PR is risky in the sense that it changes core dnf workflow and it could affect a lot of users, instead I made PRs with different possibly even more suitable approach.

https://github.com/rpm-software-management/libdnf/pull/1106
https://github.com/rpm-software-management/dnf/pull/1703

These PRs only add a new API function fill_sack_from_repos_in_cache to python3-dnf which is capable of loading repositories without xml metadata present (repomd.xml is still required). It also doesn't check if the metadata are up to date in any way and it doesn't download anything.
The idea is to call fill_sack_from_repos_in_cache instead of regular fill_sack.
Comment 9 amatej 2021-01-14 11:35:33 UTC 
Created a PR with some tests and documentation update: https://github.com/rpm-software-management/dnf/pull/1711
Comment 10 amatej 2021-02-08 08:19:50 UTC 
Here is a fixup for the tests: https://github.com/rpm-software-management/dnf/pull/1721 without this they may fail on some systems.
Comment 12 amatej 2021-02-09 12:30:22 UTC 
Apparently I can't get the tests right, here is another hopefully final fix: https://github.com/rpm-software-management/dnf/pull/1723
Comment 26 errata-xmlrpc 2021-05-18 15:00:36 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (dnf bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1657
Note You need to log in before you can comment on or make changes to this bug.