This issue is still present and is causing module dependency errors to be emitted on all dnf operations once a conflict is detected.

Eg.

# dnf module install perl:5.24/minimal
...
Complete!
# yum update
Last metadata expiration check: 0:00:31 ago on Thu Feb 14 13:46:32 2019.
Modular dependency problem:

 Problem: module freeradius:3.0:820190131191847:fbe42456-0.x86_64 requires module(perl:5.26), but none of the providers can be installed
  - module perl:5.26:820181219174508:9edba152-0.x86_64 conflicts with module(perl:5.24) provided by perl:5.24:820190207164249:ee766497-0.x86_64
  - module perl:5.24:820190207164249:ee766497-0.x86_64 conflicts with module(perl:5.26) provided by perl:5.26:820181219174508:9edba152-0.x86_64
  - conflicting requests
Dependencies resolved.
Nothing to do.
Complete!
# 

Ie. even when dnf has nothing to do and is being asked to perform a non-modular, normal update, we are getting module dependency errors regarding a non-enabled module.

Pinning a NEEDINFO onto poor Jaroslav to understand a bit better why this is happening.

This is similar to a pre-Beta bug that actually produced these errors _and_ prevented module enable/install.  That is, it is a non-cosmetic version of this same behavior:

https://bugzilla.redhat.com/show_bug.cgi?id=1640711

Given that this doesn't actually prevent anything, I don't envision this rising to the level of a release blocker, but it would be nice to get it sorted as quickly as possible.

(In reply to Ian McLeod from comment #2)

> This is similar to a pre-Beta bug that actually produced these errors _and_
> prevented module enable/install.  That is, it is a non-cosmetic version of
> this same behavior:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1640711

I think that the issue in 1640711 is at least partially fixed (I have to check with reporter).

Summary of the problem:

DNF reports always modular errors to user for any operation that requires available packages.

- WHY?
-- Modular issues have a dramatic consequences on content of available packages for the user. Example:

I have two systems one that shows error message from example in Description, and other without error. 
module freeradius:3.0:820181213135451:fbe42456-0.x86_64 requires module(perl:5.26), but none of the providers can be installed.

Output from ``dnf module list`` is identical on both systems (only error message differs)
freeradius           3.0 [d]    server [d]                      High-performance and highly configurable free RADIUS server

Then let use ``dnf repolist freeradius`` command. The command has nothing to do with modularity, therefore we can suggest that showing modular error messages is not required, but see output:
System without error message shows:
freeradius-0:3.0.15-18.module+el8+2441+0808aa29.x86_64

On the system with error message there is no package freeradius at all.

What it says, that if any problems appears during modular solving we have to provide an information about it. Otherwise how to explain the difference to user?

- Are error messages from modular solver always relevant
-- Yes, they are because they effect content of available packages. We cannot predict if packages from modules or excluded packages based on modular rules will effect particular operation (direct or indirect - dependency).

- How to solve the issue?
-- Change a definition of defaults? Defaults will not provide module packages only will mark preferred streams for enablement
--- It does not solve the issue but makes it less visible

-- Disable defaults module with broken dependency
--- We cannot distinguish if missing module dependency is caused by incorrect platform or missing dependency.
--- Missing dependency could be due to module stream conflict or missing dependency (available in disabled repository or something else)
--- We cannot predict if problems are permanent therefore permanent solution in disabling modules should be handled with care (manually by end user)

I agree with what Petr Pisar said elsewhere. In case of ordinary (non-modular) packages yum doesn't filter the listed content based on the fact whether the package can or cannot be installed. It lists available content in the repository and actual depsolving happens once the user issues a transaction that is relevant to that package.

Example:
[root@host-8-251-70 ~]# dnf list libcurl-minimal
Available Packages
libcurl-minimal.i686                                      7.61.1-8.el8                                    rhel
libcurl-minimal.x86_64                                    7.61.1-8.el8                                    rhel
[root@host-8-251-70 ~]# dnf -y install libcurl-minimal
Error: 
 Problem: problem with installed package libcurl-7.61.1-8.el8.x86_64
  - package libcurl-minimal-7.61.1-8.el8.x86_64 conflicts with libcurl provided by libcurl-7.61.1-8.el8.x86_64
  - cannot install the best candidate for the job
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

For a user perspective the current behaviour for modular content is confusing. As a user I would expect the same behaviour as above, i.e. have the modular content available in enabled/default streams visible (despite possibly conflicting) and the conflict should arise only after dnf/libdnf needs that content in a transaction.

To implement Commend 5 it requires complete redesign of modularity especially multi-context in one streams.

(In reply to Jaroslav Mracek from comment #6)
> To implement Commend 5 it requires complete redesign of modularity
> especially multi-context in one streams.

Right, we have many aspects of modular updates tied to the behaviour of filtering then depsolving; eg. hotfix behaviour also depends heavily on this.

However, just because a package is filtered out does not mean that that filtering is even remotely relevant.  If I'm just doing a "yum update" on an already-uptodate system, then the dependency collision in filtering is not relevant to me.

The *only* time that the modular dependency conflict matters is if I'm about to enable a module in order to install a package, and I cannot due to modular conflicts.

So I suggest our two options here are either:

- Emit a modular depsolver error only when we actually attempt to enable a default module which has a conflict with another enabled module; or (as described before)
- Disable the conflicting modules up-front when a new module is enabled

Emitting the error for every single yum operation really makes no sense.  A useful comparison might be what happens in the presence of rpm conflicts in a repo; eg. if I have coreutils installed then I can never install coreutils-single, but I do not expect dnf to complain "error, coreutils-single is available but can't be installed" every time I update a system with coreutils installed.  Rather, I only get the conflict error when I actually try to install one on top of the other.

I propose following pull request as a possible solution of the issue https://github.com/rpm-software-management/dnf/pull/1338. Please can you provide a feedback?

For those who are not familiar with dnf code and can't think of all the implications of the proposed patch, would it be possible to describe the proposed/expected behaviour in a human language?

Karel please don't panic. The patch is not a final solution, but only a preview for the reporter. We are testing an user interface in cases with modular errors and your feedback is also valuable.

I am sorry, but from the patch I am not able to tell in which situations it would apply and in which it won't. Moreover, if you won't describe your intentions no one is able to tell whether the implementation is actually correct, i.e. patch can show what you did, not what you wanted to do.

(In reply to Jaroslav Mracek from comment #8)
> I propose following pull request as a possible solution of the issue
> https://github.com/rpm-software-management/dnf/pull/1338. Please can you
> provide a feedback?

Glad to give it a try; could you possibly do a scratch build (either rhel8 or f30 would be fine) and I'll see how it looks?

Thanks!

Stephen, please could you provide a feedback?

(In reply to Jaroslav Mracek from comment #15)
> Stephen, please could you provide a feedback?

I've tried the copr build but I don't see any change in behaviour: installing perl:5.24 still results in the same 

 Problem: module freeradius:3.0:820190131191847:fbe42456-0.noarch requires module(perl:5.26), but none of the providers can be installed
  - module perl:5.26:820181219174508:9edba152-0.noarch conflicts with module(perl:5.24) provided by perl:5.24:820190207164249:ee766497-0.noarch
  - module perl:5.24:820190207164249:ee766497-0.noarch conflicts with module(perl:5.26) provided by perl:5.26:820181219174508:9edba152-0.noarch
  - conflicting requests

errors being reported, both on initial install of the module and on subsequent "yum update" operations.

This is with the following packages installed from the copr:

Upgraded:
  dnf-4.2.0-1.git.8737.d43785b.el8.noarch                                                                        
  dnf-data-4.2.0-1.git.8737.d43785b.el8.noarch                                                                   
  libdnf-0.27.2-1.git.2501.73b0de0.el8.x86_64                                                                    
  python3-dnf-4.2.0-1.git.8737.d43785b.el8.noarch                                                                
  python3-dnf-plugins-core-4.0.5-1.git.777.fe9b836.el8.noarch                                                    
  python3-hawkey-0.27.2-1.git.2501.73b0de0.el8.x86_64                                                            
  python3-libdnf-0.27.2-1.git.2501.73b0de0.el8.x86_64                                                            
  yum-4.2.0-1.git.8737.d43785b.el8.noarch

I am sorry, I provided incorrect build. Please try again. The correct version is dnf-4.2.0-1.git.8736.1029e0b.el8 .

What it changed?
If the modular problem is only with defaults, error messages are only visible with the "-v" option or in the dnf logger for command like repoquery, upgrade. If you run the "module enable" command, errors are always visible.

Is it working according your expectation?

This is improved but I'm not sure it's quite there yet.  With the new build, after installing perl:5.24 I no longer get errors on "yum update"; but I still get problems reported when I install the perl:5.24 module in the first place:

# yum module install perl:5.24/minimal
...
Problems in request:
Modular dependency problem with Defaults:

 Problem: conflicting requests
  - module freeradius:3.0:820190131191847:fbe42456-0.noarch requires module(perl:5.26), but none of the providers can be installed
  - module perl:5.26:820181219174508:9edba152-0.noarch conflicts with module(perl:5.24) provided by perl:5.24:820190207164249:ee766497-0.noarch
  - module perl:5.24:820190207164249:ee766497-0.noarch conflicts with module(perl:5.26) provided by perl:5.26:820181219174508:9edba152-0.noarch
Dependencies resolved.

I don't think that this situation should be reported as a "problem", when this is in fact an intended and expected outcome when the perl:5.24 stream is enabled.

This is the preferred behavior, as articulated by various members of the module team on the April 17th interlock call.

I'm going to NEEDINFO Adam, Langdon and SCT to confirm that I am accurately reflecting
what was discussed.

I'll describe it in terms of some real modules in RHEL8 that produce this behavior.

The perl module has two streams: 5.26 and 5.24.
The freeradius module has a single stream (3.0), that is also a default that requires perl:5.26.

I will now outline actions and their expected outputs:


# dnf module enable perl:5.24

The key point here is that once perl:5.24 is enabled, the freeradious default stream will become un-enable-able
and essentially unable to be used.

For this action, dnf _may_ emit an informational message to clarify this fact.  The message should be very clearly
a non-error message.  It may say, essentially "I'm going to enable this stream.  Please be advised that you will
no longer be able to enable or install content from the freeradius default stream."

The existing messages read very much like errors and we are very concerned that they will be confusing to end users.

This informational message must only occur in response to the module enable command above.  It must not repeat
during unrelated dnf actions.



All further examples assume that the command above has already been executed and accepted.  It is our belief that the behaviors described below already occur.


# dnf module enable freeradius
# dnf module install freeradius

Either of these should produce a module dependency _error_ indicating that the requirement for perl:5.26 cannot be met.


# dnf install freeradius

This should return an _error_ indicating that the package is not available.

(In reply to Ian McLeod from comment #21)
> This is the preferred behavior, as articulated by various members of the
> module team on the April 17th interlock call.
> 
> I'm going to NEEDINFO Adam, Langdon and SCT to confirm that I am accurately
> reflecting
> what was discussed.

This looks correct.  

The only comment I'd add is that we expect that in your example above, once the perl:5.24 stream is enabled, dnf commands which do _not_ involve the freeradius module or package[s] must not emit any information (error or not) about the freeradius module dependency.  (That's kind-of assumed given the prior discussion but is worth calling out explicitly.)

Did I capture that correctly?

Thanks!

(In reply to Stephen Tweedie from comment #22)
> (In reply to Ian McLeod from comment #21)
> > This is the preferred behavior, as articulated by various members of the
> > module team on the April 17th interlock call.
> > 
> > I'm going to NEEDINFO Adam, Langdon and SCT to confirm that I am accurately
> > reflecting
> > what was discussed.
> 
> This looks correct.  
> 
> The only comment I'd add is that we expect that in your example above, once
> the perl:5.24 stream is enabled, dnf commands which do _not_ involve the
> freeradius module or package[s] must not emit any information (error or not)
> about the freeradius module dependency.  (That's kind-of assumed given the
> prior discussion but is worth calling out explicitly.)
> 
> Did I capture that correctly?
> 
> Thanks!

You did, and it is what I was getting at when I said the following in the summary:

"This informational message must only occur in response to the module enable command above.  It must not repeat
during unrelated dnf actions."

The issue also requires to answer a fundamental question about originally masked packages.

Background:
pkg-5-1.nonmodular
pkg-2.1-2.modular-stream1 (installed)
pkg-2.2-2.modular-stream2

module modular:stream1 require module m-second:1

Example1:
module modular:stream1 is enabled, but cannot be resolved due to dependencies

Present behaviour:
visible pkg-5-1.nonmodular =>  pkg-2.1-2.modular-stream1 will be upgraded to pkg-2.1-2.modular-stream1

Expected behaviour:
What you suggest as an expected behaviour?

(In reply to Jaroslav Mracek from comment #29)
> Present behaviour:
> visible pkg-5-1.nonmodular =>  pkg-2.1-2.modular-stream1 will be upgraded to
> pkg-2.1-2.modular-stream1

I believe you wanted to write:

Present behaviour:
visible pkg-5-1.nonmodular =>  pkg-2.1-2.modular-stream1 will be upgraded to pkg-5-1.nonmodular

My (user) expectations would be that packages are masked based on enabled and default stream content irrespectively whether it can be actually resolved or not.
It would probably have implications for multiple contexts: If I am not able to identify the proper stream context for enabled stream (as none of them can be resolved) I would do the masking utilizing data from all the contexts
(I am curious whether it would make any real harm to do such filtering every time).

If I should find a _similar_ situation with non-modular packages, it would be something like not being able to install available package due to missing dependencies. I still see the package I am not able to install which also allows me to troubleshoot the situation.

Just my 2 cents...

We have to also consider that conflict in module result in unavailability of updates (even critical one). 

Module stream is similar and presented as a virtual repository.

In case that repository is unavailable dnf stop to work due to skip_if_unavailable setting.

Conflict in module stream results in unavailability of all packages from virtual repository - module stream. The system is unable to consume any updates but dnf produce only a warning. But now it should even produce a warning that doesn't look as an error because it could scare users.

I am sorry but I see here an inconsistency that must be clarified.

(In reply to Jaroslav Mracek from comment #29)
> The issue also requires to answer a fundamental question about originally
> masked packages.
> 
> Background:
> pkg-5-1.nonmodular
> pkg-2.1-2.modular-stream1 (installed)
> pkg-2.2-2.modular-stream2
> 
> module modular:stream1 require module m-second:1
> 
> Example1:
> module modular:stream1 is enabled, but cannot be resolved due to dependencies
> 
> Present behaviour:
> visible pkg-5-1.nonmodular =>  pkg-2.1-2.modular-stream1 will be upgraded to
> pkg-2.1-2.modular-stream1
> 
> Expected behaviour:
> What you suggest as an expected behaviour?

I'm not sure I understand.  Surely it would not be permitted to enable modular:stream1 in the first place if it has dependency problems?  So this is just asking about edge conditions where there's a configuration/repository problem, isn't it?

I would agree that we should upgrade to pkg-2.1-2.modular-stream1 in this scenario.  If that package fails to install due to an unsatisfied RPM dependency because of some other missing module dependency. then that's fine.

But I think this issue goes beyond the scope of simply limiting the times at which module dependency errors are emitted.

(In reply to Jaroslav Mracek from comment #32)
> We have to also consider that conflict in module result in unavailability of
> updates (even critical one). 
> 
> Module stream is similar and presented as a virtual repository.
> 
> In case that repository is unavailable dnf stop to work due to
> skip_if_unavailable setting.
> 
> Conflict in module stream results in unavailability of all packages from
> virtual repository - module stream. The system is unable to consume any
> updates but dnf produce only a warning. But now it should even produce a
> warning that doesn't look as an error because it could scare users.
> 
> I am sorry but I see here an inconsistency that must be clarified.

I'm somewhat confused: in https://bugzilla.redhat.com/show_bug.cgi?id=1666770#c29 you wrote that dnf *will* update packages to a modular stream that is enabled but cannot be resolved due to dependencies.  The scenario you talk about here is that dnf will *not* update to packages from a stream that is unavailable due to conflicts.

I think we need to be a lot more specific about the scenarios here.

For example, the main issue in this BZ is that we're getting spurious conflicts about modules which are not installed.  Eg. when perl:5.24 is enabled, we get conflicts for "freeradius:3.0:820181213135451:fbe42456-0.x86_64 requires module(perl:5.26)".

In that case, we don't have any issue about missing updates, because the conflict is arising from a module which is not enabled or installed.

Can you give an example where we would be unable to apply genuine updates?

So far the main discussion has been that we would silently hide the packages from conflicting modules; eg. "yum install freeradius" might show no packages available because the freeradius module had conflicts.  But that behaviour should never break updates for modules which have already been installed successfully.

Before we can resolve this issue we need to resolve https://bugzilla.redhat.com/show_bug.cgi?id=1757460 first. The proper solution of bz#1757460 we need additional metadata that will describe context upgrade path and end of life. When upgrade path will be know, most of the errors will disappear therefore the primary issue of this bug will be also be resolve. Critical question is: can we change metadata or shall we recognize related issues as unresolvable?

(In reply to Jaroslav Mracek from comment #38)
> Before we can resolve this issue we need to resolve
> https://bugzilla.redhat.com/show_bug.cgi?id=1757460 first. The proper
> solution of bz#1757460 we need additional metadata that will describe
> context upgrade path and end of life. 

I think we may be over-complicating things here.

The minimum requirement for addressing bz#1757460 may well be addressed
simply within the dependency resolution (eg. using best=0 for the module
dependency resolver step.)

Adding an extra feature to guide the user about when, and how, to upgrade
between streams, is a useful additional RFE but the core modularity
design does not require that.

> When upgrade path will be know, most
> of the errors will disappear therefore the primary issue of this bug will be
> also be resolve.

I don't think that's true; the upgrade path will always have to be 
opt-in, we cannot just upgrade users forcibly when an old stream goes EOL.
So we will need to be able to cleanly handle EOL streams even after we add
helpers for stream switches; we cannot rely on the upgrade path to 
eliminate these warnings.

So I think that we can, and should, keep the stream switching part of 
bz#1757460 separate from this BZ.  (The dep-solving part of bz#1757460
still does relate to this BZ, of course, but that's a simpler part of the
problem, and I don't think that it requries any additional metadata.)

I received modularity conflicting requests when trying to update rawhide to f33 today:

# dnf --refresh upgrade 
Fedora - Modular Rawhide - Developmental packages for the next Fedo  25 kB/s |  15 kB     00:00    
Fedora - Rawhide - Developmental packages for the next Fedora relea  53 kB/s |  15 kB     00:00    
google-chrome                                                       8.5 kB/s | 1.3 kB     00:00    
Modular dependency problems:

 Problem 1: conflicting requests
  - nothing provides module(platform:f32) needed by module eclipse:latest:3220191219031631:14c70fc4-0.x86_64
 Problem 2: conflicting requests
  - nothing provides module(platform:f32) needed by module gimp:2.10:3220191106095052:43bbeeef-0.x86_64
Dependencies resolved.
Nothing to do.
Complete!

Just for kicks, I tried "resetting" the modules, whatever that does; and it seems to remove the "Problem:"

# dnf module reset eclipse

dnf --refresh upgrade 
Fedora - Modular Rawhide - Developmental packages for the next Fedo  24 kB/s |  15 kB     00:00    
Fedora - Rawhide - Developmental packages for the next Fedora relea  28 kB/s |  15 kB     00:00    
google-chrome                                                       9.8 kB/s | 1.3 kB     00:00    
Modular dependency problem:

 Problem: conflicting requests
  - nothing provides module(platform:f32) needed by module gimp:2.10:3220191106095052:43bbeeef-0.x86_64
Dependencies resolved.
Nothing to do.
Complete!

We currently don't have any reasonable solution to the request.
Closing CANTFIX.

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days