Bug 1940895

Summary: [RFE] When files are in "(replaced)" status (in 'rpm -V output), they cannot query their history using the 'rpm --dbpath <db> -qf <file>' command, and need to be able to do this
Product: Red Hat Enterprise Linux 8 Reporter: Marion Levine <malevine>
Component: rpmAssignee: Michal Domonkos <mdomonko>
Status: CLOSED ERRATA QA Contact: Jan Blazek <jblazek>
Severity: high Docs Contact: Mariya Pershina <mpershin>
Priority: high    
Version: 8.3CC: jason.mccallister, jcastran, kwalker, lmiksik, malevine, mbanas, mdomonko, pmatilai
Target Milestone: rcKeywords: FutureFeature, Patch, Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: rpm-4.14.3-23.el8 Doc Type: Enhancement
Doc Text:
.A new `--path` CLI option is added to RPM With this update, you can query packages by a file that is currently not installed using a new `--path` CLI option. This option is similar to the existing `--file` option, but matches packages solely based on the provided path. Note that the file at that path does not need to exist on disk. The `--path` CLI option can be useful when a user excludes all documentation files at install time by using the `--nodocs` option with `yum`. In this case, by using the `--path` option, you can display the owning package of such an excluded file, whereas the `--file` option will not display the package because the requested file does not exist.
Story Points: ---
Clone Of:
: 2072176 (view as bug list) Environment:
Last Closed: 2022-05-10 15:28:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2072176    
Bug Blocks:    

Description Marion Levine 2021-03-19 14:00:40 UTC
2. Who is the customer behind the request?

Account: Wells Fargo/1455359
TAM customer: no
CSM customer: no
Strategic: yes

3. What is the nature and description of the request?

They would like to be able to query packages using 'rpm --dbpath <db> -qf <file>' as they were able to in RHEL 6.

They stated that when files are in "(replaced)" status (in 'rpm -V output), they cannot query their history using the following command, and need to be able to do this:

    # rpm --dbpath <db> -qf <file>

Example:

    $ rpm --dbpath /src/tlketl1/dev/anthill/rpmdb -qf /src/tlketl1/dev/abinitio/tlk/tlk_practice/sql/BladeEntitlementContract_ATOM_3569.sql
MISSING 1406 <--------------------------------- SHOWS 'MISSING' INSTEAD OF A PACKAGE NAME
tlk-tlk_practice-20210219093029-1447.x86_64

4. Why does the customer need this? (List the business requirements here)

We have automated build and deploy processes for ~150 enterprise applications.

    - These applications may have a dozen to a few hundred named RPM packages each.
    - Applications can chose a 'full' or 'incremental' build.
    - A 'full' build consists of all objects in a named RPM package.
    - An 'incremental' build consists of a subset of objects in an RPM package.
    - Most Applications create 'incremental' builds that consist of only a few files that is effectively a 'patch'.
    - 'Full' builds use rpm -U while 'incremental' builds use rpm -i.

Over time an application’s deployed base of files may consist of a full build plus dozens of patches.

    - Sometimes they need to rollback a patch.
    - When we rollback, we erase (rpm -e) the RPM that they want to roll back.
    - However this leaves missing files.

In order to recover the missing files, we run rpm -qf to find the previous package that contained that missing file. This loops for each file:

     - The previous package is retrieved from Artifactory.
     - We then restore the file from the identified package using "rpm2cpio <rpm> | cpio -idvm '<file>".

5. How would the customer like to achieve this? (List the functional requirements here)

We would like to request a switch that would allow this functionality

6. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.

We can repeat the following test:

# rpm --dbpath=/test/rpmdb -ivh --force rpm_test-1-0.x86_64.rpm --prefix /test/installdir/
# rpm --dbpath=/test/rpmdb -ivh --force rpm_test-2-0.x86_64.rpm --prefix /test/installdir/

Whichever one is installed second, will overwrite the first one. The first one now shows (replaced)

# rpm --dbpath=/test/rpmdb -V rpm_test-1-0
  .........    /files/x (replaced)
# rpm --dbpath=/test/rpmdb -V rpm_test-2-0

7. Is there already an existing RFE upstream or in Red Hat Bugzilla?

No

8. Does the customer have any specific timeline dependencies and which release would they like to target (i.e. RHEL5, RHEL6)?

RHEL 8

9. Is the sales team involved in this request and do they have any additional input?

No

10. List any affected packages or components.

rpm

11. Would the customer be able to assist in testing this functionality if implemented?

yes

Comment 2 Kyle Walker 2021-04-07 15:01:36 UTC
Just to add additional context, the behaviour change in question is around this commit:

commit 9ad57bda4a82b9847826daa766b4421d877bb3d9
Author: Panu Matilainen <pmatilai>
Date:   Mon Aug 29 16:19:32 2011 +0300

    Use RPMDBI_INSTFILENAMES on file queries as well
    
    - This changes query behavior quite a bit as files with non-installed
      status are no longer returned as matches. The rationale is that
      it makes figuring out dependency issues with the "new" behavior
      more obvious (but perhaps we should have a switch to enable
      former behavior). Besides not satisfying dependencies, files with
      non-installed status are not really owned by the package, as they
      would not be removed on package removal.

diff --git a/lib/query.c b/lib/query.c
index 73d64e34a..1e429548b 100644
--- a/lib/query.c
+++ b/lib/query.c
@@ -409,7 +409,8 @@ static rpmdbMatchIterator initQueryIterator(QVA_t qva, rpmts ts, const char * ar
            fn = xstrdup(arg);
        (void) rpmCleanPath(fn);
 
-       mi = rpmtsInitIterator(ts, RPMDBI_BASENAMES, fn, 0);
+       /* XXX Add a switch to enable former BASENAMES behavior? */
+       mi = rpmtsInitIterator(ts, RPMDBI_INSTFILENAMES, fn, 0);
        if (mi == NULL)
            mi = rpmtsInitIterator(ts, RPMDBI_PROVIDENAME, fn, 0);


Essentially, the request is to add the above mentioned switch.

Comment 3 Panu Matilainen 2021-04-08 08:09:39 UTC
Oh, that. I had completely forgotten such a thing...

Comment 5 Michal Domonkos 2021-04-08 16:06:28 UTC
Thank you, Kyle and John, for the thorough analysis of this issue in the associated customer ticket and not only hunting down the root cause, but also finding the corresponding commit(s).  Great job!  It's going to be that much easier and quicker for us to resolve this!

Comment 8 Michal Domonkos 2021-08-16 16:42:11 UTC
Upstream PR:
https://github.com/rpm-software-management/rpm/pull/1755

Comment 9 Panu Matilainen 2021-09-01 11:41:53 UTC
Funny how sometimes the obvious gets lost in the details...

This can be achieved with a one-liner Python script that will work on basically any rpm version, replace <file> with the file you're looking to find:

    python -c "import sys,rpm;[print(h['nevra']) for h in rpm.ts().dbMatch('basenames', sys.argv[1])]" <file>

Alternative db paths can be specified by adding rpm.addMacro('_dbpath', '/some/where') before the list comprehension, and of course if one writes it out as a real script you can do far more.

Comment 37 errata-xmlrpc 2022-05-10 15:28:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (rpm bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2082