I can reproduce this from f29 with:

dnf --disablerepo=updates-modular --disablerepo=updates-testing-modular --disablerepo=fedora-modular --disablerepo=fedora --disablerepo=updates-testing --enablerepo=rawhide --releasever=rawhide --enablerepo=rawhide-modular distro-sync -x liborcus

but I can't make it happen only using f29 repos, ATM. Might be something specific to Rawhide repos. Have we actually started allowing modules to be auto-enabled, or something?

What....what *is* this:

https://github.com/rpm-software-management/dnf/commit/1b13ee79df086f13ea330184b4ea386d9be51085

"[module] Auto-enable module streams based on installed RPMs."

wat? why?

Hmm, if I comment that code out of base.py, then I get this:

...

Downgrading:
 ant                                                  noarch 1.10.4-1.module_1886+ece9a977      rawhide-modular 188 k
 ant-lib                                              noarch 1.10.4-1.module_1886+ece9a977      rawhide-modular 2.0 M
 apache-commons-cli                                   noarch 1.4-4.module_1885+a6f9b3e6         rawhide-modular  73 k
 apache-commons-codec                                 noarch 1.11-3.module_1885+a6f9b3e6        rawhide-modular 287 k
 apache-commons-logging                               noarch 1.2-13.module_1885+a6f9b3e6        rawhide-modular  84 k
 glassfish-el-api                                     noarch 3.0.1-0.7.b08.module_1885+a6f9b3e6 rawhide-modular 104 k
 hawtjni-runtime                                      noarch 1.16-1.module_1889+fac7270b        rawhide-modular  42 k
 jansi                                                noarch 1.17.1-1.module_1889+fac7270b      rawhide-modular  78 k
 jansi-native                                         x86_64 1.7-5.module_1889+fac7270b         rawhide-modular  37 k
 jline                                                noarch 2.14.6-2.module_1889+fac7270b      rawhide-modular 156 k
 slf4j                                                noarch 1.7.25-4.module_1885+a6f9b3e6      rawhide-modular  76 k
 LibRaw                                               x86_64 0.19.0-3.fc29                      rawhide         308 k
 clang6.0-libs                                        x86_64 6.0.1-5.fc29                       rawhide          12 M
 dnssec-tools-libs                                    x86_64 2.2.1-1.fc29                       rawhide         148 k
 gdb                                                  x86_64 8.1.90.20180727-45.fc30            rawhide         132 k
 gdb-headless                                         x86_64 8.1.90.20180727-45.fc30            rawhide         3.3 M
 gtk-update-icon-cache                                x86_64 3.24.0-4.fc30                      rawhide          32 k
 gtk3                                                 x86_64 3.24.0-4.fc30                      rawhide         4.4 M
 gtk3-devel                                           x86_64 3.24.0-4.fc30                      rawhide         4.4 M
 gtk3-immodule-xim                                    x86_64 3.24.0-4.fc30                      rawhide          20 k
 libknet1                                             x86_64 1.3-2.fc29                         rawhide          67 k

Transaction Summary

note, it still wants to install packages from the modular repo, but it doesn't enable the modules. So, it's not that the 'auto-enable module streams' code is wrongly kicking in and enabling those modules, *then* packages from those modules are getting selected - it's the other way around: packages from the modular repo are somehow getting pulled into the transaction, *then* the 'auto-enable module streams' code causes the modules they come from to be enabled. So that code isn't the core of the problem it seems.

dnf 3.2.0 / libdnf 0.17.0 does the same (I just downgraded and checked). So this isn't something introduced recently in dnf, it seems. Most likely it's some sort of problem in the Rawhide repodata?

Adding sgallagh and lsedlar for any comments...

Module auto-enablement based on installed RPMs is a key feature.
This behavior is expected.
From DNF perspective, this is NOTABUG.

(In reply to Adam Williamson from comment #2)
> What....what *is* this:
> 
> https://github.com/rpm-software-management/dnf/commit/
> 1b13ee79df086f13ea330184b4ea386d9be51085
> 
> "[module] Auto-enable module streams based on installed RPMs."
> 
> wat? why?

To clarify this: what it means is that if the current system has RPMs installed that are part of a module but the module itself is not marked as enabled, it will do so to make sure that future updates are handled properly. This is mostly supposed to be a protective measure around when people install RPMs directly from Koji or similar.

Essentially what is happening here is that there are module streams that are a default stream for Rawhide. Since module streams "win" over bare RPMs, the default streams are being marked enabled during the distro-sync and are installing those packages.

This doesn't happen on F29 right now because only Rawhide has default streams for those modules.

I agree with Daniel that this is NOTABUG, but I think we probably need to figure out how to communicate that to the user better.

"Essentially what is happening here is that there are module streams that are a default stream for Rawhide."

ah, ok. I did not know that was happening and had not caught any announcements of it. So to be clear, that means that when you try to install a package that is in one of these 'default streams', you will get the package *from the module* in preference to the package from the non-modular distro, and that module and stream will be automatically enabled?

Yes. That is correct. And you’re right, we failed to announce that.

Another question occurs: I can't find that PackageKit calls set_modules_enabled_by_pkgset anywhere. So, what happens if we run an update like this with PackageKit (e.g. gnome-software)?

According to my investigation, packages installed from default module by microdnf trigger enablement of module. Unfortunately this is not true for PackageKit. Tested with command "pkcon install dwm". I think that people from PackageKit can investigate the issue more closely and find the issue.

That certainly sounds like a bug, but it's not *this* bug.

After the explanation by sgallagh, Kevin and I understand and are fine with the actual dnf *behaviour* here, but we still think it doesn't explain what's going on very well. If you just run 'dnf upgrade' and see this, it's not at all clear that modules are getting enabled to update packages which are now in default module streams. Thus this bug is now a request for dnf to explain what's going on a little more clearly.

We should certainly file a separate bug if PK installs the package from the module but does not enable the module, in this case.

I filed https://bugzilla.redhat.com/show_bug.cgi?id=1632518 for the bug whereby PackageKit does not enable the module stream when installing a package from a default module stream, as jmracek reported in #c11. I also suspect dnfdragora does not do this; I'll test that.

I believe that DNF works here like expected -> closing

Um. Did you read comment #12? It specifically says "After the explanation by sgallagh, Kevin and I understand and are fine with the actual dnf *behaviour* here, but we still think it doesn't explain what's going on very well. If you just run 'dnf upgrade' and see this, it's not at all clear that modules are getting enabled to update packages which are now in default module streams. Thus this bug is now a request for dnf to explain what's going on a little more clearly."

Please could you provide an example of the output that you would expect to see? Thanks a lot.

I'd like to see dnf say something like: 

package xyz is in default module abc, installing package ayx version xxx from module abc and marking module abc enabled.

+1 kevin.

If it needs to be shorter, perhaps "Enabling default module stream module:stream for packages ayx[, ayz, ...]"

In general, I agree that this would be helpful to the user.

If it needs to be shorter, perhaps "Enabling default module stream module:stream for packages ayx[, ayz, ...]"

In general, I agree that this would be helpful to the user.

I would recommend to log it using debug level. The list could be ling and it is according other workflows in dnf. Do you agree?

No. What we want is for regular users to be alerted to this happening so they understand what is going on. Debug logging does not achieve that.

I'm not sure what you mean by "it is according other workflows in dnf" - can you clarify?

DNF move many information about transaction details into debug logging level. We prefer to provide a single transaction table that includes essential information for most of users. Also according to my experience, information printed outside of transaction table is often ignored. I believe that we should keep that strategy to not overload users.

This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Fedora 31 changed to end-of-life (EOL) status on 2020-11-24. Fedora 31 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.