Bug 1693683

Summary: Port libdnf to libmodulemd 2.x+
Product: Red Hat Enterprise Linux 8 Reporter: Stephen Gallagher <sgallagh>
Component: libdnfAssignee: amatej
Status: CLOSED ERRATA QA Contact: Radek Bíba <rbiba>
Severity: medium Docs Contact:
Priority: high    
Version: 8.0CC: amatej, lberton, mdomonko, mprahl, ngompa13, pkratoch, swm-qe
Target Milestone: rcKeywords: FutureFeature, Triaged
Target Release: 8.1   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: libdnf-0.47.0-1.el8 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-04 01:52:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1693680, 1752511, 1797749, 1797750    
Bug Blocks: 1825061    

Description Stephen Gallagher 2019-03-28 13:27:58 UTC
Description of problem:
Upstream libmodulemd has deprecated the 1.x API as unmaintainable. The 2.x API is vastly simplified and is where all fixes and new functionality will be added.


Version-Release number of selected component (if applicable):
libdnf-0.22.5-4.el8

How reproducible:
N/A


Additional info:

The full API specification for libmodulemd 2.2 is available at https://fedora-modularity.github.io/libmodulemd/2.2/ and a basic API tutorial is included in https://fedora-modularity.github.io/libmodulemd/2.2/modulemd-2.0-Modulemd.html (including a recommended DNF usage).

Comment 8 Matt Prahl 2019-09-26 15:29:56 UTC
Module Build Service (MBS) builds fail on non-RHEL7 systems because MBS uses DNF to find the list of NEVRAs in a repository, and this fails because MBS uses libmodulemd v2 and libdnf uses libmodulemd v1. This blocks local builds from working correctly since the majority of users use Fedora for those.

Here is a reproducer Python script that demonstrates the issue:
```
import logging

import dnf
import gi
# If this is 1.0, this script works just fine
gi.require_version('Modulemd', '2.0')
from gi.repository import Modulemd

logging.getLogger().setLevel(logging.DEBUG)


repo_url = 'https://kojipkgs.fedoraproject.org/repos/f30-build/latest/$arch/'
base = dnf.Base()
dnf_conf = base.conf
dnf_conf.metadata_expire = 0
dnf_conf.cachedir = '/tmp/dnf-reproducer'
base.reset(repos=True, goal=True, sack=True)
repo_name = 'f30_x86_64'
repo_arch_url = 'https://kojipkgs.fedoraproject.org/repos/f30-build/latest/x86_64/'
base.repos.add_new_repo(repo_name, dnf_conf, baseurl=[repo_arch_url])
base.repos[repo_name].load()
# This stays stuck with the error
# `Module yaml error: Document type was unspecified or unknown`
# when `gi.require_version('Modulemd', '2.0')` is set
base.fill_sack(load_system_repo=False)
print(len(base.sack.query().available()))
```

Comment 10 Jaroslav Mracek 2020-01-06 14:08:44 UTC
We investigated the code of libmodulemd-2 and we found significant differences that will brake our API. Or probably we are looking at it by incorrect way therefore I would like to ask for help with porting. Please could anyone familiar with libmodulemd-2 API help us?

Comment 11 Stephen Gallagher 2020-01-06 19:42:20 UTC
(In reply to Jaroslav Mracek from comment #10)
> We investigated the code of libmodulemd-2 and we found significant
> differences that will brake our API. Or probably we are looking at it by
> incorrect way therefore I would like to ask for help with porting. Please
> could anyone familiar with libmodulemd-2 API help us?

You are correct, it is absolutely an ABI break (because the API and ABI of v1 is terrible and difficult to maintain and even more difficult to *use*)

I'd be very happy to help with any porting efforts. I tried to look into it myself a few months ago, but I don't have a sufficient understanding of the way it works currently to get anywhere. It would be best if I could sit down for a few hours with one of the DNF developers and work through it.

Comment 13 Neal Gompa 2020-02-01 13:58:10 UTC
I know this is a RHEL bug, but from the point of view of the maintainer of the DNF package manager in Mageia, OpenMandriva, openSUSE, and now ROSA, I'd *really* like to see this done sooner rather than later. It's a rather annoying complication to ship both versions and the bugs resulting from differing behaviors in module processing due to the different libraries is rather insane.

Comment 14 amatej 2020-02-03 07:27:02 UTC
Yes, I am actively working on it but its going to take some time.

Comment 16 amatej 2020-02-04 08:11:40 UTC
Yes of course, thank you.
Sorry if I have been unresponsive on GitHub lately, but I am just slowly trying to get it to work and pass our tests. I will surely let you know if there's anything needed.

Comment 17 amatej 2020-02-10 13:48:48 UTC
I have created a PR: https://github.com/rpm-software-management/libdnf/pull/894

I didn't use any of the new functions yet, I am not sure how jmracek wanted to use them, I will ask about it.

Comment 30 errata-xmlrpc 2020-11-04 01:52:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (yum bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4510