Bug 1326363
| Summary: | Capsule sync fails with 'some distro file downloads failed' | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Kedar Bidarkar <kbidarka> | ||||
| Component: | Foreman Proxy | Assignee: | John Mitsch <jomitsch> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Kedar Bidarkar <kbidarka> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 6.2.0 | CC: | bbuckingham, bcourt, bmbouter, daviddavis, dkliban, ehelms, ggainey, ipanova, jcline, mhrivnak, mmccune, pcreech, rchan, ttereshc | ||||
| Target Milestone: | Unspecified | Keywords: | Triaged | ||||
| Target Release: | Unused | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | pulp-rpm-2.8.1.3-1 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2016-07-27 11:28:33 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Kedar Bidarkar
2016-04-12 14:21:32 UTC
Apr 12 01:02:20 katello1 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed
----------------------------------------------------------------------
Tried provisioning both RHEL7 and RHEL6 repos,
RHEL7: provisioning was successful ( boot files could be fetched from isolated capsule by the HOST).
RHEL6 : It was unable to fetch the boot files { vmlinuz and initrd )
May be the above "ERROR: some distro file downloads failed" is being referred to RHEL6 distro files.
Looking into this with Justin, we found that the kickstart repo structure is different between 6.7 and 7.2 rhel kickstart repos on the cdn. On the capsule sync, the 7.2 kickstart repo succeeded, while the 6.7 kickstart repo failed. The rhel 6.7 kickstart repo contains a PULP_DISTRIBUTION file that points to a repodata/productid file.[1] The 7.2 kickstart repo does not contain this same file [2] Pulp is presumably publishing the distribution into the publish directory including the ‘/repodata/productid’ file. It then is generating the yum metadata for the repository wiping out the ‘/repodata/productid’ file. This results in a missing distribution file that leads to the issue. I'll attach the pulp error from dynflow where you can see that its looking for productid [1]http://cdn.rcm-internal.redhat.com/content/dist/rhel/server/6/6.7/x86_64/kickstart/PULP_DISTRIBUTION.xml [2]http://cdn.rcm-internal.redhat.com/content/dist/rhel/server/7/7.2/x86_64/kickstart/ Created attachment 1146669 [details]
pulp error from dynflow
This is caused by these two lines, which specifically exclude any distribution file that starts with "repodata/" https://github.com/pulp/pulp_rpm/blob/pulp-rpm-2.8.2-1/plugins/pulp_rpm/plugins/distributors/yum/publish.py#L796-L797 Barnaby, do you recall why that change was made? Michael, Those lines were added to prevent a distribution from overwriting any of the generated metadata files. The productid file should be in the repomd.xml and be included separately from the distribution. However, if it would be more beneficial to blacklist the files the pulp_distribution does not get to include at a more fine-grained level then go for it. Looking at this, my guess is Pulp won't be able to sync from other Pulp servers if the original repository contained a PULP_DISTRIBTION.xml file _and_ that PULP_DISTRIBUTION.xml file references anything in 'repodata/'. The reason is the files are stripped out at publish time (which seems inappropriate to me) as mhrivnak noticed. Pulp does not create PULP_DISTRIBUTION.xml: it just republishes whatever it found in the upstream repository[0]. Therefore, PULP_DISTRIBUTION.xml references files Pulp intentionally doesn't publish and the Pulp that tries to sync from that published repository fails just like it does here. Therefore, in my opinion, this is a bug with Pulp. It would be possibly (I think) to work around this issue on this particular repository by modifying the CDN's version of PULP_DISTRIBUTION.xml, but Pulp should not be getting itself into this state at all. [0] https://github.com/pulp/pulp_rpm/blob/master/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py#L437-L441 I've filed an issue on the upstream Pulp bug tracker: https://pulp.plan.io/issues/1843 The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. The Pulp upstream bug priority is at High. Updating the external tracker on this bug. The Pulp upstream bug status is at POST. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. VERIFIED With sat62-snap9.0
Capsule sync now successful with ks-rhel{6, 7} and rhel{6, 7} repos.
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug. The Pulp upstream bug status is at VERIFIED. Updating the external tracker on this bug. The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501 |