Bug 1298474

Summary: dnf group upgrade produces strange and inconsistent results
Product: [Fedora] Fedora Reporter: David Jansen <jansen>
Component: dnfAssignee: rpm-software-management
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: christian.lupien, iand, jansen, jmracek, jsilhan, mluscon, packaging-team-maint, pnemade, vmukhame
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-12-06 14:27:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Debugdata from: dnf group upgrade 'Text-based Internet' none

Description David Jansen 2016-01-14 08:38:35 UTC
Description of problem:

I was experimenting with package groups for my own local repository, and wanted to use the documented use of 'dnf group upgrade "groupname"' to get the current content of that group installed. 
That didn't work.
So, to make certain I understood the intended behaviour of dnf, I tried the same with some package groups from the fedora repositories, with similar unexpected results

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. dnf group install 'Text-based Internet'
2. (review package selection and answer yes to install).
3. dnf group upgrade 'Text-based Internet'

Actual results:

[root@eendracht ~]# dnf group upgrade 'Text-based Internet' --best
Last metadata expiration check performed 1:18:32 ago on Thu Jan 14 08:00:42 2016.
Error: package duplicity-0.7.06-1.fc23.x86_64 requires ncftp >= 3.1.9, but none of the providers can be installed
(try to add '--allowerasing' to command line to replace conflicting packages)

Expected results:

No errors, just confirmation that everything was installed already

Additional info:
Packages mentioned in the error are present in correct versions:
[root@eendracht ~]# rpm -q duplicity ncftp
duplicity-0.7.06-1.fc23.x86_64
ncftp-3.2.5-10.fc23.x86_64

System is fully up-to-date.

Adding '--best' to the dnf command makes no difference here.

There are more package groups that display this behaviour.

PS: before running these tests, I had disabled my own and all other non-fedora repositories, just to be sure it was no external conflict. 
Problem is reproducible on multiple computers.

Comment 1 Honza Silhan 2016-01-19 14:32:51 UTC
Can you please attach debugdata [1]?

[1] https://github.com/rpm-software-management/dnf/wiki/Bug-Reporting#dependency-resolution-problem

Comment 2 David Jansen 2016-01-22 10:14:11 UTC
Created attachment 1117140 [details]
Debugdata from: dnf group upgrade 'Text-based Internet'

Comment 3 Ian Donaldson 2016-05-20 06:38:57 UTC
Not sure if its releted but...

Prior to fc22 I used to use 'yum group update ..' after a distro-sync OS 
releasever upgrade in order to grab extra packages that are part of the new release's main package groups.
Usually there were a few packages pulled in in each OS upgrade.


With fc23 I did the same thing with dnf ... but it just says
the group is already installed and suspiciously does nothing.

eg: dnf groupupdate 'Minimal Install'
    dnf groupupdate 'Fedora Workstation'

Is this expected behaviour?

Comment 4 Ian Donaldson 2016-05-20 06:39:55 UTC
BTW I did these groupupdate's in the fc21->fc22 OS update also.

Comment 5 Fedora Admin XMLRPC Client 2016-07-08 09:35:57 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 6 Christian Lupien 2016-11-07 09:46:13 UTC
On Fedora 24, with dnf-1.1.10-1.fc24.noarch, I see the problem.

When installing a group where optional packages are already installed (but without the with-optional option), then the /var/lib/dnf/groups.json full_list entry for that package will list all the mandatory, default and installed optional packages.

When trying to upgrade the group next, things go wrong because it tries to erase packages it should not.

I tracked the problem to the dnf/comps.py python source. The method group_upgrade of the Solver class does things wrong. new_set is only the mandatory and default pacakges from the comps file. old_set is the full_list from the groups.json file (persistor). Therefore upon upgrade, "remove = old_set - new_set" tries to remove all previously installed optional packages which is not correct.

Even worst is that because of bug 1318852 (when group install/upgrade ends up in failure then full_list contains every mandatory, default and option, even those that are not installed) even more packages are tried to be deleted.

All those deletes will probably break some dependencies and that produces the errors that are seen.

I think the fix is to only fill full_list of groups.json with mandatory and default packages, not the optional one (unless they are asked for).

Comment 7 Christian Lupien 2016-11-07 17:22:55 UTC
It should also be noted that group upgrade does not behave as remove when removing packaging: it does not use the _removable_pkg method to check the yumdb reason for the package to see if its reason is group.

Comment 8 Fedora End Of Life 2016-11-24 14:55:55 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 9 Jaroslav Mracek 2016-12-06 14:27:45 UTC
We made a lot of changes in handling of groups in dnf-2.0. It is now available in rawhide or from our testing repository ('dnf copr enable rpmsoftwaremanagement/dnf-nightly'). Please can you test it and if your problem still appears, reopen the bug report.