Bug 2211967

Summary:	Pulp_rpm publishes incorrect location_href for manually uploaded rpm's in custom repositories.
Product:	Red Hat Satellite	Reporter:	Caleb Denney <caleb.denney>
Component:	Pulp	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED DUPLICATE	QA Contact:	Satellite QE Team <sat-qe-bz-list>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	6.12.3	CC:	dalley, rlavi
Target Milestone:	Unspecified	Keywords:	Triaged
Target Release:	Unused
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-09-04 22:46:07 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Caleb Denney 2023-06-02 19:00:04 UTC

Description of problem:
Problem appears to be similar to 2135608, but affects 6.12.3 which is newer than the version it was closed under.

When manually uploading packages to a custom repository, the published_at url uses the same filename as the packages that were uploaded. However, the location_href of the packages may include epoch, release, or other rpm metadata as part of the filename. This results in yum/dnf attempting to download available packages from a different filename than what is actually published and thus returns 404 errors. Pulp also retains metadata and associates it with package hashes, so uploading rpm's after renaming them does not solve the issue without first purging the packages from pulp's content.

Version-Release number of selected component (if applicable):

How reproducible:
Always with packages named differently than how Satellite/pulp_rpm generates repomd.xml for manually uploaded custom repositories.

Steps to Reproduce:
1. Upload rpm (noteable example jdk-11.0.19_linux-x64_bin.rpm)
2. View packages in the custom repository, jdk-11-2000:11.0.19-9.x86_64 is available
3. View published at url (Packages/j/jdk-11.0.19_linux-x64_bin.rpm exists)
4. Attempt to install jdk-11 package from content host with this repository enabled

Actual results:
Host fails to install jdk-11 due to 404 error, could not download .../Packages/j/jdk-11-2000:11.0.19-9.x86_64.rpm

Expected results:
Host downloads jdk-11 rpm from correct url.

Additional info:
Either the published_at filename is incorrect or the repodata/* files are incorrectly generated.

If the custom repo url points to file:///var/lib/pulp/custom_rpm_repo and you use the createrepo tool to generate the repodata for you custom rpms, then the location_href will reflect the local filenames and sync correctly. This is my current workaround until this bug impacting manually uploaded files can be fixed. Custom rpm repos seem to work fine as long as they're syncing from an upstream url, only local custom repositories with no upstream are affected.

From what I can tell, this seems to be a bug in the pulp_rpm plugin. Unless we're following an upstream repository, the locally generated repodata's primary.xml.gz should contain a <location href="Packages/j/jdk-11.0.19_linux-x64_bin.rpm"/> to match the file that was uploaded, but instead templates the href as "Packages/j/<name>-<epoch>:<ver>-<rel>.<arch>.rpm". This behavior would be fine if pulp_rpm also renamed the published filename to match the templated href in primary.xml.gz

Comment 1 Caleb Denney 2023-06-02 19:18:56 UTC

Looking at pulp_rpm in github and in their docs, it appears that the location_href for manually uploaded files follows their NEVRA naming scheme (name, epoch, version, release, arch), which is what we're seeing in the metadata as intended.

Therefore, the package names appearing in the published_at url are incorrect and should be following the NEVRA naming scheme. Not sure if this is pulp issue or a katello issue.

Comment 2 Daniel Alley 2023-09-04 22:46:07 UTC


*** This bug has been marked as a duplicate of bug 2161993 ***

Comment 3 Daniel Alley 2023-09-04 22:49:46 UTC

Keep in mind the repositories need to be republished for any effect to be taken. I believe there were also some instructions here regarding fixing data that is already in Pulp which may be "wrong" with respect to the intended behaviors.

https://bugzilla.redhat.com/show_bug.cgi?id=2151657#c18

Comment 4 Daniel Alley 2023-09-04 23:14:17 UTC

Actually @caleb, it looks like this was fixed in 6.12.4, not 6.12.3.  Were you looking at the version for which there was a hotfix delivered?

Comment 5 Caleb Denney 2023-09-05 14:02:22 UTC

Thank you for the 2151657 link, I thought the pulp data was supposed to be immutable to the point that the href relative paths weren't editable.

As for 2161993, this does appear to be a duplicate of that bug, but the bug is still present in 6.12.4 according to Jens Rudolf. Fixes present in 6.13.0 and 6.14.0 snap 8 according to Paul Donohue and Shweta Singh respectively. 

We are on build 6.12.4.1 now. I will try to verify if the behavior is fixed and report back. Otherwise, we plan to do a 6.13 upgrade in Q4.

Comment 6 Daniel Alley 2023-09-06 00:34:52 UTC

It should be fixed going forwards, but it doesn't automatically fix packages which were already saved with the wrong value.  You should work with support to resolve that issue for any existing saved content - I think there is already (or at least there can be) some automated way to do this rather than relying on the manual method described in that comment.

>> I thought the pulp data was supposed to be immutable to the point that the href relative paths weren't editable

In terms of the operation of the app, yes, but of course if there's incorrect data that can be fixed in the database.