Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2161993 - wrong metadata if uploaded rpm have different name than name in rpm
Summary: wrong metadata if uploaded rpm have different name than name in rpm
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.12.0
Hardware: Unspecified
OS: Unspecified
high
high with 9 votes
Target Milestone: 6.14.0
Assignee: satellite6-bugs
QA Contact: Shweta Singh
URL:
Whiteboard:
: 2161992 2162591 2166093 2177108 2192717 2211967 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-18 14:25 UTC by Jan Jansky
Modified: 2024-04-12 04:25 UTC (History)
30 users (show)

Fixed In Version: pulp-rpm-3.18.13
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2184122 2190472 (view as bug list)
Environment:
Last Closed: 2023-11-08 14:18:13 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Hotfix RPM for Satellite 6.12.2 (402.79 KB, application/x-rpm)
2023-04-03 19:45 UTC, wclark
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 35952 0 Normal New RPM packages renamed when manually uploaded leading to 404s on dnf 2023-02-27 08:30:04 UTC
Github pulp pulp_rpm issues 3039 0 None closed Location href and relative_path can be mismatched for uploaded rpms 2023-03-30 15:55:57 UTC
Red Hat Issue Tracker SAT-16126 0 None None None 2023-02-23 15:49:30 UTC
Red Hat Issue Tracker SAT-18850 0 None None None 2023-07-23 09:06:26 UTC
Red Hat Knowledge Base (Solution) 7006135 0 None None None 2023-04-04 14:15:42 UTC
Red Hat Knowledge Base (Solution) 7009800 0 None None None 2023-04-27 06:52:30 UTC
Red Hat Product Errata RHSA-2023:6818 0 None None None 2023-11-08 14:19:08 UTC

Description Jan Jansky 2023-01-18 14:25:59 UTC
Description of problem:

If i will upload rpm for example xxx-1.2.3.rpm into repository, but inside that rpm the name is for example yyy, repository metadata will point into yyy even though physically the file is named xxx and pulpcore db data pointing correctly to xxx-1.2.3.rpm.


Version-Release number of selected component (if applicable):
Satellite 6.12.0-4

How reproducible:
Always

Steps to Reproduce:
1. Create custom repository
2. Upload rpm
3. Install rpm on client

Actual results:
[Errno 14] HTTPS Error 404 - Not Found

Expected results:
Successful installation

Additional info:
As workaround repository can be removed and manually removed records in pulpcore database, after that physically rename rpm to correspond to its metadata and after that file will be pointed correctly to actual location.

More details internally.

Comment 3 Andrea Perotti 2023-01-20 11:54:10 UTC
*** Bug 2161992 has been marked as a duplicate of this bug. ***

Comment 6 Daniel Alley 2023-02-23 05:42:06 UTC
*** Bug 2166093 has been marked as a duplicate of this bug. ***

Comment 7 Daniel Alley 2023-02-23 05:43:49 UTC
Copying over info from duplicate: https://bugzilla.redhat.com/show_bug.cgi?id=2166093

"Uploading RPMS seems to have name mismatches as reported in the community. This seems to be some behavior where the names are not entirely driven by pulp which seems to break dnf installs due to wrong package name.
https://community.theforeman.org/t/rpm-packages-renamed-when-manually-uploaded/31776

Created from redmine issue https://projects.theforeman.org/issues/35952"

Comment 8 Daniel Alley 2023-02-23 06:21:25 UTC
*** Bug 2162591 has been marked as a duplicate of this bug. ***

Comment 9 Daniel Alley 2023-02-23 06:23:26 UTC
Lots of discussion on https://bugzilla.redhat.com/show_bug.cgi?id=2162591, marked as duplicate of this one.

Comment 12 Amit Agarwal 2023-03-21 18:38:25 UTC
My customer is not happy about the issue and wanted a workaround / fix ASAP

Comment 13 Hao Chang Yu 2023-03-22 06:41:32 UTC
This seems to be the mismatch of package relative_path and location_href. The mismatch seems to happen when relative_path parameter is provided to the API when uploading the rpm.

As we can see below, relative_path will only set to match the location href if it is not provided.
~~~
In "app/serializers/package.py"

    def deferred_validate(self, data):
        <snip>
        new_pkg["location_href"] = (
            format_nevra_short(
                new_pkg["name"],
                new_pkg["epoch"],
                new_pkg["version"],
                new_pkg["release"],
                new_pkg["arch"],
            )
            + ".rpm"
        )
        if not data.get("relative_path"):
            data["relative_path"] = new_pkg["location_href"]  <====================== 
~~~



Previously in Satelite 6.10 and 6.11, location_href is matching the relative_path
~~~
In app/serializers/package.py:

new_pkg = _prepare_package(data["artifact"], data["relative_path"]) <============


In app/shared_utils.py:

def _prepare_package(artifact, filename):
    <snip>
    package = Package.createrepo_to_dict(cr_pkginfo)

    package["location_href"] = filename  <=============================
    artifact_file.close()
    return package
~~~

Comment 19 Paul Donohue 2023-03-30 17:47:40 UTC
Yum repositories are only supposed to contain RPMs named $name-$version-$release.$arch.rpm
(Note: No epoch!  I'm not entirely sure why the epoch isn't included, maybe for historical reasons, but this seems to be a convention that is hard-coded in lots of things.)

Unfortunately, some vendors that provide RPMs do not correctly name their RPM files.  For example, IBM provides RPMs for Tivoli, but does not include the version/release in the filename: TIVsm-API64.x86_64.rpm

In Satellite versions that used Pulp 2, the filename of an uploaded RPM file was ignored, and RPMs served by Satellite/Pulp used an automatically-generated filename based on the name/version/release/arch extracted from the metadata within the RPM.

However, in Satellite 6.10/6.11, the filename on uploaded RPMs is used as-is.  Among other problems, this can break things if multiple RPMs are uploaded with the same (improper) filename.  See RedHat case 03309188 and https://bugzilla.redhat.com/show_bug.cgi?id=2151657 for example.

In Satellite 6.12, it appears that uploaded RPMs might now be renamed automatically.
However, the generated name in the <location> tag in the relevant repodata file seems to includes the epoch.  For example: <location href="Packages/o/openssl-1:1.0.2k-26.el7_9.x86_64.rpm"/>
But the name that is served by Pulp does not include the epoch, which breaks things for any package name that includes an epoch.  (I haven't tested whether Pulp is correctly renaming the file without the epoch, or whether it is still using the uploaded filename as-is which happened to be correct in this test.)  For example:
# yum update
...
openssl-1:1.0.2k-26.el7_9 FAILED                                            65% [=============================================-                        ]  30 MB/s |  56 MB  00:00:00 ETA 
https://satellite/pulp/repos/ORG/Env/RHEL7_COMP_VIEW/custom/Test/RHEL7_Test/Packages/o/openssl-1%3A1.0.2k-26.el7_9.x86_64.rpm: [Errno 14] HTTPS Error 404 - Not Found
# curl --silent --cacert /etc/pki/ca-trust/source/anchors/katello-server-ca.pem --cert "/etc/pki/entitlement/$EntSerial.pem" --key "/etc/pki/entitlement/$EntSerial-key.pem" "https://$SatServer/pulp/repos/ORG/Env/RHEL7_COMP_VIEW/custom/Test/RHEL7_Test/Packages/o/" | grep openssl-1
<li><a href="openssl-1.0.2k-26.el7_9.x86_64.rpm">openssl-1.0.2k-26.el7_9.x86_64.rpm</a></li>

At first glance, it looks like the above location_href code in "app/serializers/package.py" that is using the epoch is probably the issue here.

Comment 20 Paul Donohue 2023-03-30 18:06:11 UTC
With a little more digging, it looks like Satellite 6.12 only generates the name in the <location> tag, and does not rename the files that are imported into / served by Pulp.

So, basically, this breaks an even wider range of use cases than the Satellite 6.10/6.11 behavior.  (6.10/6.11 only broke uploads of RPMs if both the filename was incorrect and two different RPMs with the same filename were uploaded; 6.12 now breaks any upload with an incorrect filename or with an epoch used in the RPM.)

Comment 22 wclark 2023-04-03 19:45:31 UTC
Created attachment 1955577 [details]
Hotfix RPM for Satellite 6.12.2

INSTALL INSTRUCTIONS:

NOTE: This hotfix will also work with Satellite 6.12.3 when it is released

1. Take a complete backup or snapshot of Satellite 6.12.2 server

2. Download the hotfix RPM attached to this BZ and copy it to Satellite server

3. # dnf install ./python39-pulp-rpm-3.18.9-2.HOTFIXRHBZ2161993.el8pc.noarch.rpm --disableplugin=foreman-protector

4. # satellite-maintain service restart

Comment 23 Sayan Das 2023-04-04 06:04:54 UTC
(In reply to wclark from comment #22)
> Created attachment 1955577 [details]
> Hotfix RPM for Satellite 6.12.2
> 
> INSTALL INSTRUCTIONS:
> 
> NOTE: This hotfix will also work with Satellite 6.12.3 when it is released
> 
> 1. Take a complete backup or snapshot of Satellite 6.12.2 server
> 
> 2. Download the hotfix RPM attached to this BZ and copy it to Satellite
> server
> 
> 3. # dnf install
> ./python39-pulp-rpm-3.18.9-2.HOTFIXRHBZ2161993.el8pc.noarch.rpm
> --disableplugin=foreman-protector
> 
> 4. # satellite-maintain service restart

Hello William,

Thanks for having the hotfix prepared.

For the sake of clarity, I wanted to get a confirmation if we need to republish the metadata of individual custom repos as well as related CV versions ( where the repo resides ) to fix the existing contents here.

The new rpm uploads may work but to fix existing contents, is that step additionally required ?

( Ref one of the related BZ https://bugzilla.redhat.com/show_bug.cgi?id=2151657#c17 )



-- Sayan

Comment 25 Daniel Alley 2023-04-05 05:39:31 UTC
*** Bug 2177108 has been marked as a duplicate of this bug. ***

Comment 29 Paul Donohue 2023-04-06 21:04:05 UTC
The hotfix does not fix previously uploaded RPMs, only newly uploaded RPMs.

Here is what I did to fix the previously uploaded RPMs:
https://bugzilla.redhat.com/show_bug.cgi?id=2151657#c18

Comment 30 Jens Rudolf 2023-05-10 13:24:52 UTC
This bug is still active in Satellite 6.12.4.

Comment 31 Paul Donohue 2023-05-11 17:38:07 UTC
The fixes are present in Satellite 6.13.0

(I have not yet tested to verify that everything is working as expected for me in Satellite 6.13.0, I simply confirmed that the fixes are present in the installed codebase.)

Comment 32 Daniel Alley 2023-05-23 04:59:45 UTC
*** Bug 2192717 has been marked as a duplicate of this bug. ***

Comment 34 Shweta Singh 2023-07-28 07:53:45 UTC
Verified.

Version Tested: Satellite 6.14.0 Snap 8.0

Verification Steps:
1. Register a client to satellite.
2. Create a custom repo on satellite and upload an rpm into it which will have different name from the name in its metadata.
3. Create CV, add the repo into it and publish it.
4. Make the content available to the client and install the rpm on it.

Result:
rpm will be installed on client successfully even the name of the rpm is different from the name in its metadata.
pulp-rpm: 
python39-pulp-rpm-3.19.9-1.el8pc.noarch

Comment 35 Daniel Alley 2023-09-04 22:46:07 UTC
*** Bug 2211967 has been marked as a duplicate of this bug. ***

Comment 36 Daniel Alley 2023-09-05 21:35:28 UTC
@jens.rudolf Did you see comment #29?  Does that help?

Comment 37 Daniel Alley 2023-09-05 21:41:50 UTC
On second thought, while the diagnosis and instructions are correct - it would be best to work with support to rectify any previously-existing instances of this issue which the hotfix doesn't resolve.

They may already have a better process that I'm not aware of, or, be able to produce one.

Comment 38 Jens Rudolf 2023-09-06 11:50:18 UTC
@dalley I can confirm that the hotfix works on newly uploaded RPMs. We manually removed the RPMs with the wrong metadata from the Satellite. Meanwhile we updated to Satellite v6.13.x which fixes the problem for us.

Comment 41 errata-xmlrpc 2023-11-08 14:18:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.14 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6818

Comment 42 Red Hat Bugzilla 2024-04-12 04:25:13 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.