Bug 1686966

Summary: DNF upgrade modular content to non-modular
Product: [Fedora] Fedora Reporter: Igor Raits <igor.raits>
Component: distributionAssignee: Aoife Moloney <amoloney>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: high    
Version: 34CC: hvtaifwkbgefbaei, jmracek, jrohel, kevin, mblaha, mhatina, packaging-team-maint, pkratoch, ppisar, psabata, rpm-software-management, sct, vmukhame
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-06-07 20:08:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1666770    

Description Igor Raits 2019-03-08 18:40:26 UTC
Description of problem:
⋊> ~ sudo dnf update --assumeno                                         19:37:27
[sudo] password for brain: 
Last metadata expiration check: 0:27:58 ago on Fri 08 Mar 2019 07:09:37 PM CET.
Modular dependency problems:

 Problem 1: conflicting requests
  - nothing provides module(platform:f30) needed by module stratis:1:20181215204600:a5b0195c-0.x86_64
 Problem 2: conflicting requests
  - nothing provides module(platform:f30) needed by module standard-test-roles:3.0:3020190214144451:a5b0195c-0.x86_64
 Problem 3: conflicting requests
  - nothing provides module(platform:f30) needed by module ripgrep:latest:3020190214090003:a5b0195c-0.x86_64
 Problem 4: conflicting requests
  - nothing provides module(platform:f30) needed by module ninja:latest:3020190131012415:a5b0195c-0.x86_64
 Problem 5: conflicting requests
  - nothing provides module(platform:f30) needed by module meson:latest:3020190123223713:36245242-0.x86_64
 Problem 6: conflicting requests
  - nothing provides module(platform:f30) needed by module libgit2:0.27:3020190128145600:a5b0195c-0.x86_64
 Problem 7: conflicting requests
  - nothing provides module(platform:f30) needed by module fish:3:3020190216163513:602da195-0.x86_64
 Problem 8: conflicting requests
  - nothing provides module(platform:f30) needed by module exa:latest:3020190214120734:e50d0d19-0.x86_64
 Problem 9: conflicting requests
  - nothing provides module(platform:f30) needed by module dwm:6.1:3020190213215420:a5b0195c-0.x86_64
 Problem 10: conflicting requests
  - nothing provides module(platform:f30) needed by module bat:latest:3020190214090936:e50d0d19-0.x86_64
 Problem 11: conflicting requests
  - nothing provides module(platform:f30) needed by module avocado:stable:3020190213205848:a5b0195c-0.x86_64
Dependencies resolved.
================================================================================
 Package    Arch   Version                 Repository                      Size
================================================================================
Installing:
 kernel     x86_64 5.1.0-0.rc0.git2.2.fc31 fedora-rawhide-kernel-nodebug   24 k
 kernel-core
            x86_64 5.1.0-0.rc0.git2.2.fc31 fedora-rawhide-kernel-nodebug   26 M
 kernel-modules
            x86_64 5.1.0-0.rc0.git2.2.fc31 fedora-rawhide-kernel-nodebug   28 M
 kernel-modules-extra
            x86_64 5.1.0-0.rc0.git2.2.fc31 fedora-rawhide-kernel-nodebug  2.1 M
Upgrading:
 libgit2    x86_64 0.27.8-2.fc30           rawhide                        414 k
 libgit2-devel
            x86_64 0.27.8-2.fc30           rawhide                        207 k
 meson      noarch 0.49.2-1.fc30           rawhide                        743 k
 ninja-build
            x86_64 1.9.0-2.fc30            rawhide                        127 k
Removing:
 kernel     x86_64 5.0.0-0.rc4.git2.2.fc30 @fedora-rawhide-kernel-nodebug   0  
 kernel-core
            x86_64 5.0.0-0.rc4.git2.2.fc30 @fedora-rawhide-kernel-nodebug  61 M
 kernel-modules
            x86_64 5.0.0-0.rc4.git2.2.fc30 @fedora-rawhide-kernel-nodebug  28 M
 kernel-modules-extra
            x86_64 5.0.0-0.rc4.git2.2.fc30 @fedora-rawhide-kernel-nodebug 2.1 M

Transaction Summary
================================================================================
Install  4 Packages
Upgrade  4 Packages
Remove   4 Packages

Total download size: 57 M
Operation aborted.

Version-Release number of selected component (if applicable):
dnf-4.1.0-1.fc30.noarch
libdnf-0.26.0-2.fc30.x86_64

How reproducible:
Always.

Steps to Reproduce:
1. Run `dnf update`

Actual results:
DNF upgrades modular RPMs to non-modular.

Expected results:
DNF does not upgrade modular RPMs to non-modular.

Additional info:
Name             : meson
Stream           : latest [d][e]
Version          : 3020190123223713
Context          : 36245242
Profiles         : default [d]
Default profiles : default
Repo             : rawhide-modular
Summary          : The Meson Build system
Description      : Meson is an open source build system meant to be both extremely fast, and, even more importantly, as user friendly as possible.
                 : The main design point of Meson is that every moment a developer spends writing or debugging build definitions is a second wasted. So is every second spent waiting for the build system to actually start compiling code.
Artifacts        : meson-0:0.49.1-1.module_f30+2754+087fe931.noarch

Comment 1 Jaroslav Mracek 2019-04-04 12:19:57 UTC
The issue could be easily solved by setting a Platform ID to platform:f30.

Comment 2 Jaroslav Mracek 2019-07-26 07:38:34 UTC
The important question is what should be a correct behaviour when dnf is unable to resolve correct context due to modular conflict or missing dependency.

Enabled or default module stream with conflict:
Current behaviour:
dnf excludes all modular packages from conflicting module and nonmodular packages are visible and installable.

What do you think is the best behaviour in this case?
I suggest that dnf must exclude all package from module. The important part is nonmodular packages that were excluded before a conflict appears.

Also important part is whatever we should make a difference between enabled modules and default modules?

Comment 3 Stephen Tweedie 2019-07-26 11:49:35 UTC
(In reply to Jaroslav Mracek from comment #2)
> The important question is what should be a correct behaviour when dnf is
> unable to resolve correct context due to modular conflict or missing
> dependency.
> 
> Enabled or default module stream with conflict:
> Current behaviour:
> dnf excludes all modular packages from conflicting module and nonmodular
> packages are visible and installable.
> 
> What do you think is the best behaviour in this case?
> I suggest that dnf must exclude all package from module. The important part
> is nonmodular packages that were excluded before a conflict appears.
> 
> Also important part is whatever we should make a difference between enabled
> modules and default modules?

There still seems to be a difficulty with this --- If there are multiple contexts available and we can't select one, then how do we chose which module[s] to use for the purpose of name-based filtering of non-modular RPMs?

We need to resolve that; but generally, yes, I agree that even if the individual RPMs from the module here cannot be included because of context or dependency conflicts, we still need to use them for filtering and excluding non-modular RPMs.

[This also relates to another request, that we only use the *latest* module version for that name-based filtering; this would allow us to deliberately turn modular RPMs into non-modular ones by issuing an update that drops the RPM from the module but includes it as a non-modular package.  So in the case of dependency or context conflicts, maybe we use the latest module version from each possible context in the enabled stream?]

Comment 4 Ben Cotton 2019-08-13 17:09:28 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 5 Ben Cotton 2019-08-13 18:54:06 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Comment 6 Jaroslav Mracek 2019-09-07 14:55:39 UTC
We need a design decision and discussion - see  Comment 3

Comment 7 Stephen Tweedie 2019-09-12 15:42:07 UTC
I think there are two separate problems apparent here:

1. How to handle streams and contexts when we are updating to a new release
2. How to avoid updating to a non-modular RPM when the current stream+context is no longer valid (eg. because we are updating to a new release)

The first is about how to handle such rolling updates cleanly; that seems hard to do in general, and needs deeper discussion.

But the DNF modular fail-safe mechanisms can still help with #2.  We just need to make sure that even when the current repo modulemd cannot resolve a valid module stream+context given the current system configuration, we still keep locally-installed modular RPMs intact and avoid updating them to a non-modular RPM.

One of the DNF fail-safe mechanisms is to use a recent cached modulemd for any enabled module stream that does not have a valid modulemd in the repos themselves.  That seems like it would solve the immediate problem here.  We would need to make sure that the saved modulemd is not used for any decision-making purpose other than name-based filtering — we cannot use it for more than that as it is likely to have the same dependency and context problems as the modulemd in the repos.  

But as long as we have that copy for use in name-based filtering, then we will filter out any non-modular RPMs that might otherwise be incorrectly applied as an update.


That doesn't help us move to the right latest context when we do a major update; but it does at least stop us from updating to a clearly-incorrect non-modular RPM.

Comment 8 Ben Cotton 2020-11-03 17:18:59 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 9 Ben Cotton 2021-02-09 16:22:30 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Comment 10 Jaroslav Mracek 2021-10-06 13:53:11 UTC
The situation with modules was improved. We introduced static_context and new modular solver which make broken dependencies less problematic. The new solver also allows modular dependency changes. Also there is a plan to remove Platform Id from metadata == one source of problems will disappear. The platform is primary detected from metadata and then from OS release file therefore the issue will appear less frequently. We also have module obsoletes to help with upgrades.

As you can see so many things were delivered.

Because the problem reported here is related to Platform Id and DNF does not create it in metadata, and doesn't need it for anything I can close it or move it to distribution. Personally I feel that Platform ID should be removed from module dependencies therefore I am moving it to distribution.

Comment 11 Petr Pisar 2021-10-07 12:03:49 UTC
Removing a run-time dependency on platform breaks building in MBS <https://pagure.io/fm-orchestrator/issue/1714>. MBS did it, and reverted back.
There is another possible approach: DNF will ignore the dependencies on platform.

Comment 12 Ben Cotton 2022-05-12 14:58:59 UTC
This message is a reminder that Fedora Linux 34 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '34'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 34 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 13 Ben Cotton 2022-06-07 20:08:02 UTC
Fedora Linux 34 entered end-of-life (EOL) status on 2022-06-07.

Fedora Linux 34 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 14 Red Hat Bugzilla 2023-09-18 00:15:37 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days