Bug 1920511
| Summary: | 3rd party repository sync fails with 'InvalidStringData: strings in documents must be valid UTF-8' | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Jan Senkyrik <jsenkyri> |
| Component: | Pulp | Assignee: | satellite6-bugs <satellite6-bugs> |
| Status: | CLOSED ERRATA | QA Contact: | Lai <ltran> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.8.0 | CC: | dalley, ddrozdow, dhjoshi, ggainey, gpayelka, jjeffers, osousa, qfz769, rcarrier, rchan, R.Eggermont, satellite6-bugs, saydas, snemeth, ttereshc, wpinheir |
| Target Milestone: | 6.9.7 | Keywords: | Triaged |
| Target Release: | Unused | Flags: | jsenkyri:
needinfo-
jsenkyri: needinfo- |
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | pulp-rpm-2.21.5.2-1 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-11-10 16:23:39 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Jan Senkyrik
2021-01-26 12:46:02 UTC
*** Bug 1965595 has been marked as a duplicate of this bug. *** I have been dealing with this issue since March, and can maybe add a few references to this bug. This issue is also open at nvidia: https://github.com/NVIDIA/yum-packaging-precompiled-kmod/issues/19 There are no obvious invalid UTF-8 in the yaml-file refered in the module section of http://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/repodata/repomd.xml This seems to be a pulp issue. Hope this bug gets visible sometime - it is not 'just another 3rd party malformed repo'. @qfz769, Have you encountered this recently? Some other users who have reported this issue have reported the following: "Either Nvidia has updated the repo, or something has changed on my end which I am unaware of, because suddenly I am able to sync the repo the same as you. This was not the case during last week." We have also been unable to reproduce this ourselves. It's possible that there are environmental factors like particular geographic locations hitting different CDNs (one of which could be corrupted), or perhaps specific libraries on a host system causing problems. The fact that the problem seems to have occurred spontaneously (without changes to Pulp / Satellite) and potentially been resolved spontaneously (at least for some users) does make it difficult to explain in the context of "it's a Satellite bug". Since this now has the attention of someone at Nvidia I will go discuss it in the threads you've provided (thank you for bringing them up). If you could also try again and let us know how it went, that would be great. @qfz769, I believe I may have discovered the issue. Will post an update next week. I just did a fresh sync - still the same error. (In reply to qfz769 from comment #18) > I just did a fresh sync - still the same error. Thanks to @dalley we know what's going on here. See attahced pulp issue for details! The Pulp upstream bug status is at NEW. Updating the external tracker on this bug. The Pulp upstream bug priority is at High. Updating the external tracker on this bug. The Pulp upstream bug status is at POST. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. The Pulp upstream bug status is at CLOSED - DUPLICATE. Updating the external tracker on this bug. The Pulp upstream bug status is at POST. Updating the external tracker on this bug. The Pulp upstream bug status is at POST. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST. I allow myself to add a workaround that worked for me, undtil this issue hopefully is fixed in a minor 6.9.z
# cp /usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/modules.py /usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/modules.py-backup
Edit:
# vim /usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/modules.py
profile_defaults = {}
for stream, defaults in module.peek_profile_defaults().items():
profile_defaults[stream] = defaults.get()
return bson.BSON.encode(profile_defaults) <-----
Change to :
return bson.binary.Binary(bson.BSON.encode(profile_defaults)) <---
Restart the services and try to sync the repo:
# satellite-maintain service restart
@qfz769.ku.dk That's not just a workaround, it's the proper patch :) Yes, I'm hoping that it can be slipped into the next z-stream, I think it barely missed the window for the last one. The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. Steps to test 1. Create a custom repo using the following repos: [0] https://developer.download.nvidia.com/compute/cuda/repos/rhel8/ppc64le/ [1] http://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64 2. Sync repos Expected result: Syncing should sync successfully Actual result: Syncing does sync successfully Verified on 6.9.7_01 with pulp-rpm-plugins-2.21.5.2-1.el7sat.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Satellite 6.9.7 Async Bug Fix Update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:4612 |