Bug 2070168

Summary:	Error 'modulemd-yaml-error-quark' while synchronizing fedora modular repository on Satellite 6.10.
Product:	Red Hat Satellite	Reporter:	Akshay Kapse <akapse>
Component:	Pulp	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Lai <ltran>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	6.10.3	CC:	dalley, iballou, matthew.lesieur, osousa
Target Milestone:	Unspecified	Keywords:	Regression, Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	libmodulemd2-2.12.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	2105047 2105048 (view as bug list)		Environment:
Last Closed:	2022-10-06 16:04:50 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Akshay Kapse 2022-03-30 14:24:11 UTC

Description of problem:
- While synchronizing fedora modular repository : https://dl.fedoraproject.org/pub/fedora/linux/updates/35/Modular/x86_64/

Observing below error:
~~~
modulemd-yaml-error-quark: Unexpected YAML event in document stream [line 3305 col 1] (3)
~~~

- This error can be seen only on Satellite 6.10 but not on Satellite 6.9 or 6.8
- Able to synchronize this repository on Satellite 6.9/6.8.

Version-Release number of selected component (if applicable):
- Pulp3


How reproducible:
- Easily.

Steps to Reproduce:
1. Create a custom product.

2. Create a custom repository with below upstream url:
~~~
https://dl.fedoraproject.org/pub/fedora/linux/updates/35/Modular/x86_64/
~~~

3. Try to sync it now.


Actual results:
-On Satellite 6.10 error will be:
~~~
modulemd-yaml-error-quark: Unexpected YAML event in document stream [line 3305 col 1] (3)
~~~


Expected results:
- Should be able to sync as on Satellite 6.9.


Additional Info about repository setting and error seen:

# hammer repository info --id 9
Id:                      9
Name:                    fedora-modular
Label:                   fedora-modular
Description:             
Organization:            <org>
Red Hat Repository:      no
Content Type:            yum
Mirror on Sync:          no
Url:                     https://dl.fedoraproject.org/pub/fedora/linux/updates/35/Modular/x86_64/
Publish Via HTTP:        yes
Published At:            https://<Satellite 6.10>/pulp/content/RedHat/Library/custom/custom/fedora-modular/
Relative Path:       	 <org>/Library/custom/custom/fedora-modular
Download Policy:         immediate
Ignorable Content Units: 
HTTP Proxy:              
    HTTP Proxy Policy: global_default_http_proxy
Product:                 
    Id:   317
    Name: custom
GPG Key:                 

Sync:                    
    Status:         Warning
    Last Sync Date: about 1 hour
Created:                 2022/03/30 12:30:16
Updated:                 2022/03/30 12:30:18
Content Counts:          
    Packages:       0
    Source RPMS:    0
    Package Groups: 0
    Errata:         0
    Module Streams: 0


# hammer repository synchronize --id 9
[......................................................................................................................] [100%]
No content added.
Total steps: 6/6
--------------------------------
Associating Content: 0/0
Downloading Artifacts: 0/0
Downloading Metadata Files: 6/6
Error: modulemd-yaml-error-quark: Unexpected YAML event in document stream [line 3305 col 1] (3)

Comment 1 Matthew LeSieur 2022-05-18 16:36:24 UTC

This same error is returned when synchronizing Fedora 36 Modular releases and Fedora 36 Modular updates.

https://dl.fedoraproject.org/pub/fedora/linux/releases/36/Modular/x86_64/os/
https://dl.fedoraproject.org/pub/fedora/linux/updates/36/Modular/x86_64/

Comment 2 Matthew LeSieur 2022-05-19 15:38:01 UTC

I think I may have found the cause of the "Unexpected YAML" error when processing the Fedora Modular repositories.  I took a closer look at the modules.yaml file in the repodata directory for each repository in question, and found Pulp is stopping on the first instance of "modulemd-obsoletes" document type.  The code below is from https://dl.fedoraproject.org/pub/fedora/linux/updates/36/Modular/x86_64/repodata/e3de024b0b47bdbec20a8bbe79e879538099cee9db3af3f00731f6ab92bf9e34-modules.yaml.gz line 2438 (downloaded 2022-05-19 ~10:55 EDT):

---
document: modulemd-obsoletes
version: 1
data:
  modified: 2022-03-03T15:28Z
  module: perl-IO-Socket-SSL
  stream: "2.068"
  eol_date: 2022-06-01T00:00Z
  message: Module stream perl-IO-Socket-SSL:2.068 is no longer supported. Please switch
    to perl-IO-Socket-SSL:2.074
  obsoleted_by:
    module: perl-IO-Socket-SSL
    stream: "2.074"
...

It seems the "modulemd-obsoletes" document type was introduced into Fedora as per BZ 1834844.  The BZ calls out libmodulemd-2.10.0, but Satellite 6.10.5 only includes version libmodulemd2-2.9.3-1.el7pc.x86_64.  So, at a minimum, a newer version of libmodulemd2 is required before Satellite can synchronize repositories using modulemd-obsoletes.

Comment 3 Daniel Alley 2022-06-08 02:33:55 UTC

That's exactly the issue - and it definitely needs to be addressed because I believe EPEL 8 may be impacted by a similar issue (it doesn't outright fail but it doesn't do exactly the right thing either)

Will keep you updated

Comment 5 Daniel Alley 2022-07-05 16:23:15 UTC

On EL8, the package name is just libmodulemd rather than libmodulemd2

Comment 7 Daniel Alley 2022-07-07 23:07:43 UTC

It seems like at some point in the past month or two Fedora and EPEL removed the modular-obsoletes metadata from their repos, so maybe this has been "resolved" by that.  But we still need the upgrade anyway to prevent further issues from https://bugzilla.redhat.com/show_bug.cgi?id=2105107

Comment 8 Matthew LeSieur 2022-07-08 14:46:30 UTC

I still see the "modulemd-obsoletes" document type in the Fedora 36 Release Modular repository.  Satellite 6.10.6 is still unable to synchronize this repository.  Satellite can synchronize the Update Modular repository, which was failing last I checked a month or two ago.

https://dl.fedoraproject.org/pub/fedora/linux/releases/36/Modular/x86_64/os/repodata/

---
document: modulemd-obsoletes
version: 1
data:
  modified: 2022-01-24T08:54Z
  module: perl
  stream: "5.30"
  eol_date: 2021-06-01T00:00Z
  message: Module stream perl:5.30 is no longer supported. Please switch to perl:5.32
  obsoleted_by:
    module: perl
    stream: "5.32"
...
---

Comment 9 Daniel Alley 2022-07-08 15:38:09 UTC

You're right, I may have gotten the files mixed up.  Anyway this is moving forwards and ought to be resolved in the next release(s)

Comment 10 Matthew LeSieur 2022-09-14 18:47:59 UTC

I verified syncing the Fedora repositories is fixed after installing Satellite 6.11.2.  6.11 was fixed in 2105047.

Comment 11 Daniel Alley 2022-09-14 19:15:55 UTC

Great!

Comment 13 Brad Buckingham 2022-10-06 16:04:50 UTC

At this time, there isn't another 6.10.z planned.  As this is resolved in 6.11.z, I am closing this bugzilla as 'currentrelease'.