Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1627372

Summary: [RFE] Satellite 6 is unable to sync third party repository if upstream provides primary.xml only
Product: Red Hat Satellite Reporter: Sunayana <suarora>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED WONTFIX QA Contact: Kersom <koliveir>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.3.2CC: bkearney, bmbouter, daviddavis, dhjoshi, dkliban, fperalta, ggainey, ipanova, m, mvanderw, rchan, satellite6-bugs, ttereshc
Target Milestone: UnspecifiedKeywords: FutureFeature, Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-03-20 16:19:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sunayana 2018-09-10 15:30:42 UTC
1. Title:

Satellite 6 is unable to sync third party repository if upstream doesn't provide filelists metadata.

2. What is the nature and description of the request?\

Satellite 6 is unable to sync third party repository if upstream doesn't provide filelists metadata. 

Whereas it is possible to sync the repository if upstream doesn't provide filelists metadata on Satellite 5.

Below is the details of third party repository which cu is trying:

Repository link:

https://dl.packager.io/srv/rpm/pkgr/gogs/pkgr/el/7/x86_64

repomd.xml file provided by customer:

https://dl.packager.io/srv/rpm/pkgr/gogs/pkgr/el/7/x86_64/repodata/repomd.xml

Getting below error while syncing:

------------------
Yum Metadata: Malformed repository: "filelists" metadata is not found in repomd.xml
------------------

3. Why does the customer need this? (List the business requirements here)

Cu wants to synchronize third party repositories if filelists metadata is not present for repository due to security parameters.

4. How would the customer like to achieve this? (List the functional requirements here)

N/A

5. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.

N/A

6. Is there already an existing RFE upstream or in Red Hat Bugzilla?

No

7. Does the customer have any specific timeline dependencies and which release would they like to target (i.e. RHEL5, RHEL6)?

ASAP

8. Is the sales team involved in this request and do they have any additional input?

No

9. List any affected packages or components.

N/A

10. Would the customer be able to assist in testing this functionality if implemented?

Yes

Additional Information:

Below link for repository is not publicly accessible bit still it is possible to configure locally with yum and able to sync successfully on satellite 5.

Issue is with satellite 6 only:

https://dl.packager.io/srv/rpm/pkgr/gogs/pkgr/el/7/x86_64

Comment 1 Tanya Tereshchenko 2018-09-11 16:49:02 UTC
It's not only about filelists.xml, the same requirement is for other.xml. The repomd.xml in the description has reference to the primary.xml only.

It was discussed before and it was advised by the yum devs that such repository is treated as malformed since not all functionality will be available in that case, e.g. files search.

See BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1267579#c8
And the upstream discussion: https://pulp.plan.io/issues/1287#note-15

Satellite6/Pulp relies on the remote metadata, one of the main use cases is on_demand download, so all search and content management happens based on metadata, without access to the rpms themselves.

There are more complexities to this request, e.g. one day user decides to create a repo without filelists and the other day with, and the expectation would be that rpms are updated accordingly in Satellite6 and now contain filelists information.
RPMs are identified by NEVRA and the way currently the de-duplication works those rpms will stay without filelists after update.

Just to set proper expectations: it's not a small and quick change and it's a question whether to do it or not.

Let's discuss it here.

Comment 2 Bryan Kearney 2018-09-27 17:23:49 UTC
Tanya, does yum and|or dnf support this repo? if so, pulp should.

Comment 3 Tanya Tereshchenko 2018-10-08 12:34:21 UTC
Bryan,

yum is able to sync such repos and install content from them. The file search functionality won't work.

I haven't tried dnf but here is a reply from them:
"DNF could eventually work with primary.xml only assuming there are no file dependencies.
filelists.xml is typically required for deps and necessary for file queries.
At this moment filelists are required by DNF."

It seems tome that it's not safe to accept a repo without filelists.xml. I'll ask DNF team for more details and if there is a way to identify in primary.xml whether we should require filelists or not.

Comment 4 Bryan Kearney 2018-10-08 14:10:31 UTC
Tanya:

Thanks for the response. I think the customers POV is that if yum works, then pulp should handle it.

Comment 5 Tanya Tereshchenko 2018-10-09 08:11:14 UTC
That's probably fair. If yum/dnf doesn't work properly with the upstream repo and with a repo from Satellite in a same way, it's ok.

David, should we create a new upstream issue for that? Or any other opinions?

Comment 6 David Davis 2018-10-09 12:24:23 UTC
Adding upstream issue.

https://pulp.plan.io/issues/4070

Comment 7 pulp-infra@redhat.com 2018-10-09 13:31:49 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 8 pulp-infra@redhat.com 2018-10-09 13:31:52 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 13 David Davis 2020-02-13 16:27:53 UTC
I talked with RPM team and the general feeling is that Pulp shouldn't support this. While yum may support this, it introduces data integrity problems that yum probably doesn't have to contend with. For example, RPMs are immutable in Pulp and Pulp de-duplicates RPMs by NEVRA so if a package gets synced without its filelist, it will remain that way even if another source has a filelist for the package. We're hesitant to support this feature because of the potential bugs it'll create for users.

Comment 14 Bryan Kearney 2020-03-20 16:19:47 UTC
Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this, please do not reopen. Instead, feel free to contact Red Hat Technical Support. Thank you.

Comment 15 pulp-infra@redhat.com 2020-12-10 17:14:58 UTC
The Pulp upstream bug status is at CLOSED - WONTFIX. Updating the external tracker on this bug.

Comment 16 Red Hat Bugzilla 2024-01-06 04:25:41 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days