Bug 1336879

Summary: dnf fails to respect conditional level in comps.xml
Product: [Fedora] Fedora Reporter: Luya Tshimbalanga <luya>
Component: dnfAssignee: rpm-software-management
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: 24CC: 3ndymion.1, awilliam, bruno, ghborrmann, gmarr, jmracek, luya, marek78uk, mcatanzaro+wrong-account-do-not-cc, mluscon, packaging-team-maint, pnemade, red, robatino, vmukhame
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: RejectedBlocker RejectedFreezeException
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-02-10 15:54:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1369129    
Bug Blocks:    

Description Luya Tshimbalanga 2016-05-17 16:45:30 UTC
Description of problem:
dnf failed to respect the conditional level of packages from comps.xml during nightly compose for either Fedora Spins and Labs (in this case Design Suite[0]). As an example from comps.f241[1]

<packagereq type="conditional" requires="gimp">gimp-data-extras</packagereq>

should automatically be installed because Gimp is set as default

<packagereq type="default">gimp</packagereq>

yet it is missing in the built[2]

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Livemedia task [3]
2.
3.

Actual results:
gimp-data-extras is not installed during the built

Expected results:
gimp-data-extras should be added 

Additional info:
[0] http://koji.fedoraproject.org/koji/taskinfo?taskID=14119221
[1] https://pagure.io/fedora-comps/blob/master/f/comps-f24.xml.in
[2] https://kojipkgs.fedoraproject.org//work/tasks/9221/14119221/packaging.log
[3] http://koji.fedoraproject.org/koji/tasks?state=all&view=tree&method=livemedia&order=-id

Comment 1 Luya Tshimbalanga 2016-05-19 04:22:05 UTC
According to Fedora Comps (https://pagure.io/fedora-comps), "conditional" level of packages should be brought in with required packages installed. That process worked on previous release of Fedora 23, it now failed in Fedora 24 and more as observed on livemedia built.
The comps.xml in question is listed on the comment #0 which include the resulting packaging.log. 
It would be nice to fix as soon as possible.

Comment 2 Fedora Blocker Bugs Application 2016-05-19 06:34:59 UTC
Proposed as a Blocker for 24-final by Fedora user luya using the blocker tracking app because:

 This bug was found with the livemedia compose using conditional level of packages inside comps.xm from Fedora 24 and up. The result is those conditional packages will not be fulfilled when installing the required packages. Affected groups are:
- Assamese Support
- Bengali Support 
- Bodo Support
- Design Suite
- Dogro Support
- Finnish Support
- Graphics
- Gujarati Support
- Haskell
- Hindi Support
- Input Methods
- Japanese Support
- Kannada Support
- Kashmiri Support
- KDE
- Konkani Support
- Korean Support
- Lepcha Support
- Maithili Support
- Malayalam Support
- Manipuri Support
- Marathi Support
- MinGW cross-compiler
- Multimedia
- Nepali Support
- Online Help and Documentation
- Odia Support
- Punjabi Support
- Sanskrit Support
- Santali Support
- Simplified Chinese Support
- Sindhi Support
- Sinhala Support
- Standard
- Tamil Support
- Telugu Support
- Thai Support
- Tibetan Support
- Urdu Support
- Vietnamese Support

Comment 3 Parag Nemade 2016-05-19 07:16:09 UTC
I can verify this bug. Just check the yum vs dnf output for say a haskell group.

[parag@f24 ~]$ sudo dnf groupinstall haskell --assumeno |grep emacs
Operation aborted.

[parag@f24 ~]$ sudo yum-deprecated groupinstall haskell --assumeno |grep emacs
Yum command has been deprecated, use dnf instead.
See 'man dnf' and 'man yum2dnf' for more information.

---> Package emacs-haskell-mode.noarch 0:13.18-1.fc24 will be installed
 emacs-haskell-mode            noarch 13.18-1.fc24        fedora          290 k


So the conditional packages specified in comps are not processed by dnf but by yum.

Comment 4 Geoffrey Marr 2016-05-23 18:08:42 UTC
Discussed during the 2016-05-23 blocker review meeting: [1]

This bug was classified as a RejectedBlocker and a RejectedFreezeException as the impact of this bug does not violate the blocker criteria and a fix as this point in the release cycle could prove to be too dangerous for the benefit it would provide.

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2016-05-23/f24-blocker-review.2016-05-23-16.00.txt

Comment 5 Michal Luscon 2016-06-20 11:19:55 UTC
*** Bug 1346251 has been marked as a duplicate of this bug. ***

Comment 6 Parag Nemade 2016-06-24 08:50:30 UTC
Well I don't know when DNF developers will get time to fix this but anyway I would like to change priority to High to get their attention.

Comment 7 David Shea 2016-07-01 16:47:36 UTC
*** Bug 1352122 has been marked as a duplicate of this bug. ***

Comment 8 3ndymion 2016-07-01 17:21:27 UTC
I confirm that this problem happens with XFCE as well.  Trying to do a fresh, clean install of XFCE from the net-installer, & it installs Cinnamon instead.  Happens every time.

I don't mean to sound negative, but how was this not considered an urgent issue???  I've seen unimportant bugs that don't get fixed for years, but this is the most absurd I've seen yet.  I ask to install XFCE, & I get Cinnamon instead.  As a somewhat similar example, it's like if somebody tries to download the new Windows OS, & they get Mac OS instead.  That's definitely an urgent issue, & so is this.

I apologize that I don't have the knowledge to help with this, but I think that this really needs more attention.  I can only really use XFCE on my little netbook, & now I'm stuck with no Fedora at all.  I'm pretty sure other people are going through the same thing as well.

Please help.  Thanks.

Comment 9 Luya Tshimbalanga 2016-07-01 18:17:26 UTC
Raising the (In reply to 3ndymion from comment #8)
> I confirm that this problem happens with XFCE as well.  Trying to do a
> fresh, clean install of XFCE from the net-installer, & it installs Cinnamon
> instead.  Happens every time.

It turned out the issues also affected KDE and MATE according to the duplicated reports with comments provided by David Shea. DNF developers are working on it for Fedora 25


> I apologize that I don't have the knowledge to help with this, but I think 
> that this really needs more attention.  I can only really use XFCE on my 
> little netbook, & now I'm stuck with no Fedora at all. 

Current workaround is to download XFCE livemedia spin for the time being.

Comment 10 3ndymion 2016-07-02 18:22:38 UTC
(In reply to Luya Tshimbalanga from comment #9)
...
> Current workaround is to download XFCE livemedia spin for the time being.

OK, thank you.  I really don't like the live media install & I've always avoided it, but since I have no choice this time, I tried it out.  It seems to be working.

I really hope the developers can get this fixed really soon.  This is definitely NOT the way to release a new product.

Comment 11 markm 2016-07-04 19:10:37 UTC
(In reply to 3ndymion from comment #10)
> (In reply to Luya Tshimbalanga from comment #9)
> ...
> > Current workaround is to download XFCE livemedia spin for the time being.
> 
> OK, thank you.  I really don't like the live media install & I've always
> avoided it, but since I have no choice this time, I tried it out.  It seems
> to be working.
> 
> I really hope the developers can get this fixed really soon.  This is
> definitely NOT the way to release a new product.

Developers have limited capacity and this is an open source project.

Also cinnamon is relatively easy to remove once MATE is installed and configured. Worked for me!

Comment 12 3ndymion 2016-07-05 10:14:18 UTC
(In reply to markm from comment #11)
...
> Developers have limited capacity and this is an open source project.
> 
> Also cinnamon is relatively easy to remove once MATE is installed and
> configured. Worked for me!

Cinnamon is definitely nice.  I like it so much that I'm getting ready to put it on my main laptop instead of KDE.  : )

I apologize to all if my previous words were too harsh.  It's just very frustrating as a user, especially having to find out about all of this the hard way.  I do know how hard this is for developers, especially when they have to work jobs.

Comment 13 3ndymion 2016-07-05 10:20:37 UTC
TO ALL DEVELOPERS EVERYWHERE, PLEASE READ THIS

In light of this situation, I'd like to state some important things.  This is not meant to be offensive in any way, so please don't take it as such.  Instead, please keep these things in mind, & feel free to spread this message everywhere you possibly can.

The main point I'd like to make is this:
If something is broken (specifically something that's very important)...

EITHER
   ---Do not release it.
OR
   ---Release it with a warning that something is wrong, & what can be done about it.

As developers, ask yourselves this: What are you working so hard for???  Are you working hard because you want people to turn away from your work & badmouth your efforts in the worst ways possible???  Of course not.  BUT, when something like what happened here is done, the answer to that question is YES.  Yes, you are working hard only to have people throw your work in the garbage.  Yes, you want people to speak bad about your work in the worst ways possible, & spread the word to everyone they know.  You are only shooting yourself in the foot.

Why is that???  It is because of this:  It is not only Fedora experts & advanced Linux users that are trying to use your product.  No, it is also normal people who are not so technically inclined that want to try out what you have to offer.  People who are coming from Windows & want to try something better.  People who come from many different areas of life.  This includes high reaching public facing figures.

What happens when somebody tries to use this nice, new OS, & finds out the hard way that it's broken & doesn't work right???  They'll curse it & turn away from it.  They'll remember it, & tell everyone to stay away from it.  Now what happens when that person happens to be a journalist, or blogger???  Rest assured, they will let the entire world know about your broken product & how much it sucks.  They'll spread the bad news all over the internet.  "Stay away from Fedora.  It's broken, it sucks, etc..."

Certainly, nobody who puts lots of hard work & effort into something wants those types of results.  As an upcoming developer myself, I know how time consuming all of this stuff is, & how hard it is to do this while working some job that takes all of your time away.  I also know that there will always be problems.  I'm not asking for things to be perfect from the start, but please, PLEASE, reconsider your priorities.

In this particular case, the net-installer is broken.  This is of the highest priority.  What good is an OS that can't even be installed???  It's almost like buying a brand new car, & then finding out on your own that the key is broken & does not start the car.  The dealer knew of the problem, but decided that it's not important for you to know, & never told you about it.  Clearly, the dealer should have EITHER fixed the problem before selling the new car, OR sell it with the warning that something is wrong, & what can be done to work around it.

In this case, IF net-installer is broken, EITHER do not release it, OR make it available for download with a warning right next to it, stating that there is an issue, what it is, & what can be done to work around it.  PLEASE do not give people something that is broken, hoping they won't notice.  Trust me, they will definitely notice, & it also takes the risk of the media spreading bad words about your hard work.

I speak all of this as both a user, & a developer.  I now know what it's like to be on both sides; how frustrating it can be for users, & how difficult things can be for developers.  Again, I do not intend to offend anyone by any of this, but only to raise awareness.  I do not like to see something great, & people's hard work, tarnished by something like this.  Please consider these things.

Please feel free to copy any of this everywhere possible.  I think there are many Linux developers everywhere that need to see this.  I do appreciate the effort you all have done for this Fedora that I like to use, & for that, I thank you all.  Thank you for taking the time to look through all of this.

Best Regards,
3ndymion

Comment 14 Fedora Admin XMLRPC Client 2016-07-08 09:32:56 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 15 David Shea 2016-07-11 12:52:59 UTC
*** Bug 1354072 has been marked as a duplicate of this bug. ***

Comment 16 Michal Luscon 2016-08-17 11:53:36 UTC
As there is no official documentation for comps I want to make sure we are on the same page here. 

I have already implemented behavior from https://pagure.io/fedora-comps in https://github.com/rpm-software-management/dnf/pull/559 . Effectively it means that a conditional package is pulled into transaction if and only if it's required package has been already installed on the system. 

Please take note that if you are installing a group with conditional package and a requirement of that conditional package together in one transaction, the conditional package won't be installed.

Please let me know whether the behavior described above is expected.

Comment 17 Adam Williamson 2016-08-17 14:47:27 UTC
No, that would not match my expectations. I would expect the opposite behaviour in the second-described case. Remember one of comps' main use case is to define the package groups available during *initial install* of the system; obviously no packages at all are already installed during initial system installation.

Comment 18 Adam Williamson 2016-08-17 14:48:19 UTC
...however I take the point about there being no documentation, so I'd suggest we double-check the behaviour of the old code in this case. Just the fact of the bug report suggests it previously behaved differently, but we can run an install from an old DVD to check.

Comment 19 Davide Repetto 2016-08-19 17:31:16 UTC
I agree with Adam, as you can see from bugs 1352122 and 1346251 this unpredictable behaviour creates strange results when you're trying to install a new system (or just to add a new group).

Comment 20 Davide Repetto 2016-08-19 17:35:43 UTC
ie. it installs things that you don't intend to, when you do things such as trying to install the mate-desktop environment group. In which case you end up with installed pieces of KDE and Cinnamon that shouldn't be there.

Comment 21 Jaroslav Mracek 2017-02-10 15:54:37 UTC
Bugs fixed in version of dnf-2.0.1-1.