Bug 1547041 - [RFE] Modular aware repoclosure [NEEDINFO]
Summary: [RFE] Modular aware repoclosure
Keywords:
Status: ASSIGNED
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf-plugins-core
Version: rawhide
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-20 12:04 UTC by Tomas Mlcoch
Modified: 2023-12-04 15:36 UTC (History)
9 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:
jmracek: needinfo? (lruzicka)
lberton: needinfo-
lruzicka: needinfo? (jmracek)


Attachments (Terms of Use)

Description Tomas Mlcoch 2018-02-20 12:04:52 UTC
Please provide a repoclosure tool with modularity support that is able to run on hybrid repositories, repositories with both modules and bare rpms.

DNF Behavior for hybrid repositories should be already defined, so the repoclosure should follow that behavior.

Comment 1 Daniel Mach 2018-02-20 12:56:57 UTC
This will also need proper work with modular $context.

Comment 4 Jaroslav Mracek 2020-02-13 21:10:00 UTC
I would like to enhance repoclosure with interaction with modules.

At the present time repoclosure tests only available packages (without excluded packages and packages filtered out by modularity).

I am thinking about two additional tests:
1. testing all packages without modules - like all modules were disabled
2. Testing all module combinations with nonmodular content. At the present time there are about 6000 module combinations in Fedora 31 that can be tested.

Comment 5 Jaroslav Mracek 2020-02-14 13:57:56 UTC
I am sorry but I was too optimistic with module combinations. With 93 consumable modules:streams we have following number of combination

number of modules consumed in parallel | number of combinations
        1                              |       93
        2                              |     2787
        3                              |    53084
        4                              |   745538
        5                              |  8269283

Anyway we can test it.

Still I am thinking about tests:
1. Testing all packages without modules - like all modules were disabled
2. Testing all broken packages with also all modular packages to discover if they depends on modules.
3. Testing all module depend combinations with nonmodular content. Like when module has no dependencies test it like only the particular module was enabled. Or enabled with all its dependencies.
4. Discover all modules with broken dependency tree.

Any suggestions?

Comment 6 Jaroslav Mracek 2020-02-14 17:13:23 UTC
I created a pull-request (https://github.com/rpm-software-management/dnf-plugins-core/pull/386) that provides basic support for modules in repoclosure. I would like to ask for help with forming requirements and testing. Could anyone prove a feedback?

Comment 7 Tomas Mlcoch 2020-02-21 08:57:51 UTC
> 1. Testing all packages without modules - like all modules were disabled

Not sure if viable, in RHEL we have packages that depends on packages from _default_ modules.

> 2. Testing all broken packages with also all modular packages to discover if they depends on modules.

You will never ever have all modular packages available. Some modules are conflicting with each other. Thus this test doesn't make a sense.


What I suggest is:

* To test DNF like behavior - when testing packages allow them to "enable" default modules if they need them because of dependency.

* When testing modules, test them one by one. Let's try to install a module = enable all its deps and if necessary also enable other default modules if they provide deps needed by the module.

I believe these two tests I just mentioned could work, however this is based on my previous experience with modules and DNF that may changed since then as I'm not in touch with modules for several months. If there are any doubts/unclarity about what I suggested, please investigate with subject matter experts, this is just my brain dump in order to help, but I have no capacity to work on this these days.

Comment 8 Lukas Ruzicka 2020-02-26 12:39:58 UTC
(In reply to Jaroslav Mracek from comment #5)

> 1. Testing all packages without modules - like all modules were disabled

Yes, I think that this makes sense for Fedora. Currently, no modules should have default streams for Fedora 32, so I do not expect that modular only content should interfere with this test. 

> 2. Testing all broken packages with also all modular packages to discover if
> they depends on modules.

Does it mean that if I get broken packages from the above test, I could test whether their dependency can be satisfied by a module? I would like to know that there is such a dependency and
if the required module would really satisfy it. 

3. Test that one particular module has all dependencies met in the non-modular repository, and if not, what other modules are needed to saturate the dependencies. I should be able to select the one particular module on the command line.

4. Test that that one (or more, even all) modules can be installed in at least one stream without breaking dependencies for the rest of the packages. If modular content should be usable, it is crucial you can combine any viable module combination, install it and still have a fully working system with all the dependencies satisfied.

5. Also test, that an installed module (or modules) can be safely uninstalled without breaking any of the dependencies of the remaining system.

6. Test that a modular stream can be switched to a different stream without breaking any of the dependencies of the remaining system.

7. Providing, dnf has access to modular metadata, would it be possible to test that the module has all the required metadata (sanity) ?

> 3. Testing all module depend combinations with nonmodular content. Like when
> module has no dependencies test it like only the particular module was
> enabled. Or enabled with all its dependencies.

I think that this is covered by 4, just want to be sure.

> 4. Discover all modules with broken dependency tree.

Yes, totally. 

I would also totally love when the results could be obtained both on a terminal and as a machine readable (json) file.

Please, let me know, Jaroslav, if you need more information from me. We can talk about the details later. If some of the proposals do not make any sense, let me know, too.


> 
> Any suggestions?

Comment 9 Johnny Hughes 2021-05-29 11:24:58 UTC
I am not sure what a possible fix is, but not being able to test the repoclosure of released repos for CentOS Stream is very troubling.  Basically, we just have to guess if a repo works at this point.  Is ther any method to test for the closure of repos with default modules?


Note You need to log in before you can comment on or make changes to this bug.