Bug 1554010 - No env. group installed after Fedora installation [NEEDINFO]
Summary: No env. group installed after Fedora installation
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: spin-kickstarts
Version: 33
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Václav Pavlín
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: RejectedBlocker AcceptedFreezeException
: 1780280 1841241 (view as bug list)
Depends On:
Blocks: F32BetaFreezeException F33BetaBlocker 1891500
TreeView+ depends on / blocked
 
Reported: 2018-03-10 18:26 UTC by Jaroslav Mracek
Modified: 2021-11-30 16:12 UTC (History)
26 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-30 16:12:16 UTC
Type: Bug
jmracek: needinfo? (vpavlin)


Attachments (Terms of Use)

Description Jaroslav Mracek 2018-03-10 18:26:56 UTC
Description of problem:
After standard installation of Fedora 26/27 non of env. group is marked as installed. The mark of env. group as installed is critical for DNF to keep integrity of system. Like after installation of Fedora workstation user can install "KDE Plasma Workspaces" later on will remove the KDE Plasma Workspaces and the operation will try to remove everything including "Core" group.

Version-Release number of selected component (if applicable):
Fedora-Workstation-Live-x86_64-27-1.6.iso
Fedora-Cinnamon-Live-x86_64-26-1.5.iso

How reproducible:
Just install Fedora

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:
One of env. group installed according to installed environment.

Additional info:

Comment 1 Martin Kolman 2018-03-14 11:28:14 UTC
I'm currently working on module installation support in Anaconda and I think the new PoC API mhatina added for this can fix this issue with environments as well.

For some background:
- when the Anaconda DNF payload was first written DNF apparently did not support group exclusion in it's API
- Anaconda needs group exclusion to work, even just for the (--nobase and --nocore options for the %packages section)
- this was implemented by resolving all groups requested for installation to a list and then removing all excluded groups from this list (all this in DNF payload on the Anaconda side)
- this included resolving groups belonging to the currently selected environment (if any)
- we never told DNF to install an environment, as that would pull all its groups in unconditionally, nullifying our group exclusion logic

For the new solution:
In the proposed DNF API addition mhatina developed there is now a function called install_specs(), which takes two iterables of package/group/module/environment specifications, one including items to include and the other to exclude from the installation transaction. This is how a call to it could look like:

def install_specs(install=["pkg", "@group", "@group/optional", "@module", "@module:stream", "@module/profile", "@module:stream/profile", "@^environment"],
            exclude=["pkg", "@group"])

The main idea is that we drop our own group exclusion code and just tell DNF what to include and exclude in the transaction. This way we can also tell DNF to install the environment, as the group exclusion no longer happens in Anaconda code, so it should not break anything anymore.

Anaconda side PR:
https://github.com/rhinstaller/anaconda/pull/1391

mhatinas PoC API:
https://github.com/mhatina/dnf/tree/install_specs

Feedback welcome!

Comment 2 Fedora End Of Life 2018-05-03 07:55:31 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 3 Martin Kolman 2018-05-03 10:13:29 UTC
This is being worked on a part of module installation support together with DNF.

Comment 4 Jan Kurik 2018-08-14 10:11:21 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle.
Changing version to '29'.

Comment 5 Ben Cotton 2019-10-31 20:40:37 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 6 Jaroslav Mracek 2019-11-04 19:33:13 UTC
Unfortunately still valid for Fedora 31 (tested with official workstation iso dvd). Please could you change the component to component that sets kickstarts for disks?

Comment 7 Fedora Blocker Bugs Application 2019-11-04 19:41:01 UTC
Proposed as a Blocker for 32-beta by Fedora user jmracek using the blocker tracking app because:

 The presence of environmental group is critical part to keep integrity of the system. When user on fresh systems (workstation distribution) installs additional env. group (KDE) and then removes it it also removes most of the part of Workstation (also it tries to remove dnf). Official distributions must always mark one env. group as installed.

Comment 8 Jiri Konecny 2019-11-05 11:23:24 UTC
Based on my tests this is happening only on Live media where we don't install packages using DNF but rsync the Live image.

Switching to fedora kickstarts because when the image will have environment set we will rsync that to the resulting system. Please set the environment in the kickstart used for live image creation.

Comment 9 Kevin Fenzi 2019-11-07 01:41:01 UTC
Can you expand on what exactly should be set in the kickstart here?

Comment 10 Jaroslav Mracek 2019-11-07 07:12:25 UTC
Let me use an example with 'Fedora Workstation'. It consist from following components.
Environment Group: Fedora Workstation
 Description: Fedora Workstation is a user friendly desktop system for laptops and PCs.
 Mandatory Groups:
   Common NetworkManager Submodules
   Container Management
   Core
   Fedora Workstation product core
   Firefox Web Browser
   Fonts
   GNOME
   Guest Desktop Agents
   Hardware Support
   LibreOffice
   Multimedia
   Printing Support
   base-x
 Optional Groups:
   ARM Tools

I suggest that spin-kickstarts to setup workstation do not ask to install 'Fedora Workstation' but it sends a list of groups that consist env. group 'Fedora Workstation' because not all groups want to install. I am proposing oposite approach that is supported by Anaconda and DNF => send request to anaconda to install env. group 'Fedora Workstation', and also add excluded groups that you want to skip.

Is it clear?

Comment 11 Adam Williamson 2019-11-08 20:17:42 UTC
I'm inclined to -1 blocker. We've shipped this way for several releases, after all, and it doesn't violate any criteria really. It should be fixed, but I don't see that it blocks F32.

Comment 12 Kevin Fenzi 2019-11-08 22:15:34 UTC
Sure, that makes sense.

I'm not sure I have time/energy to fix all of them, but we can urge maintainers to do so.

Comment 13 Stephen Gallagher 2019-11-11 13:58:51 UTC
-1 blocker from me as well.

Comment 14 Adam Williamson 2019-11-13 00:26:00 UTC
So, IIRC, *historically*, the kickstarts for live images didn't use environment groups because livecd-creator actually couldn't - it didn't understand them, it just didn't work. That's why we always wound up effectively duplicating the environment groups in the kickstarts, by listing all the package groups from them individually.

It may be the case that livemedia-creator *does* accept environment group syntax, in which case we can certainly look at changing at least the more prominent kickstarts to use that. We'd also have to check that this really does solve the problem (there's still the question of whether the information on which groups are installed actually makes it into the generated live image, and then from the generated live image to the installed system).

Comment 15 Jiri Konecny 2019-11-18 14:44:36 UTC
I think the LMC should accept the groups as expected but lets ask the correct person here.

Brian could you please confirm if livemedia-creator does or does not accept environment groups? See comment 14.

Comment 16 Brian Lane 2019-11-18 20:27:04 UTC
(In reply to Jiri Konecny from comment #15)
> I think the LMC should accept the groups as expected but lets ask the
> correct person here.
> 
> Brian could you please confirm if livemedia-creator does or does not accept
> environment groups? See comment 14.

Yes, lmc is using Anaconda, and is just passing the kickstart over to it so environment groups should work just fine.

Comment 17 Adam Williamson 2019-11-19 17:31:07 UTC
I guess the other question is whether we want the kickstarts to remain compatible with livecd-creator as we know some folks are still using it, including Ben Williams for the live respins, I think. CCing him.

Comment 18 Ben Williams 2019-11-19 18:20:36 UTC
to me i dont care as long as something works to let folks like me create updates isos.
i only use LCC now when i run into issues with LMC (they both use the same ks)

Comment 19 Neal Gompa 2019-11-22 19:50:25 UTC
I'm confused why this is broken in livecd-tools? We mostly just hand over to DNF to do group handling: https://github.com/livecd-tools/livecd-tools/blob/master/imgcreate/dnfinst.py#L120-L153

Comment 20 Adam Williamson 2019-11-22 20:01:40 UTC
It may not be any more. I didn't test. I just know it didn't work *at one time* and that's why the live kickstarts didn't use it.

Comment 21 Jaroslav Mracek 2019-12-17 13:44:27 UTC
*** Bug 1780280 has been marked as a duplicate of this bug. ***

Comment 22 Adam Williamson 2020-02-06 21:46:59 UTC
Discussed at 2020-02-03 blocker review meeting: https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2020-02-03/f32-blocker-review.2020-02-03-17.13.html . We agreed that improving this would be a great idea and definitely worth doing, but it doesn't really violate any criteria, and the thing it causes problems with (removing an entire desktop environment) is a thing we've previously kinda indicated you shouldn't really try to do. We've also been shipping this way for years. So it was rejected as a blocker.

We did approve it as a freeze exception issue on the grounds it can't really be fixed with an update (you can change comps in the update repo, but the comps in the stable repo will not change, and the interactions there are not super predictable).

Comment 23 Ben Cotton 2020-02-11 15:39:45 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 24 Chris Murphy 2020-03-09 16:42:09 UTC
I'm trying to figure out why new packages added to e.g. "Fedora Workstation product core" are not brought in on 'dnf systemd-upgrade', so I found myself reading this bug, but I don't understand it.

The title says "No env. group installed after Fedora installation"

But on a clean install of 'Fedora-Workstation-Live-x86_64-31-1.9.iso' I see this:

# dnf group list --installed 
Last metadata expiration check: 0:09:01 ago on Mon 09 Mar 2020 10:00:01 AM MDT.
Installed Environment Groups:
   Fedora Workstation
Installed Groups:
   Container Management
   LibreOffice

And the subset of 'dnf group list --hidden' includes

Installed Groups:
   base-x
   Container Management
   Core
   Firefox Web Browser
   Fonts
   GNOME
   Guest Desktop Agents
   Hardware Support
   LibreOffice
   Multimedia
   Common NetworkManager Submodules
   Printing Support
   Fedora Workstation product core


If the idea is to drop 'installed environment groups: Fedora Workstation' and just have 'installed groups' (set explicitly) then should this bug be retitled?

Since 'dnf system-upgrade' does distro-sync, I'd expect that dnf should install any newly added packagereq type="mandatory" or packagereq type="default" from comps for the new release version; true/false?

Comment 25 Chris Murphy 2020-03-09 22:20:41 UTC
A nevermind is in order because the man page for distro-sync says it only applies to installed packages. I guess we don't have a facility for bringing in new packages other than through adding a dependency to something already installed, or a clean install.

Comment 26 Jaroslav Mracek 2020-06-15 14:07:13 UTC
*** Bug 1841241 has been marked as a duplicate of this bug. ***

Comment 27 Jaroslav Mracek 2020-06-15 14:13:19 UTC
Please is there any progress with the issue? Today we got another report, but this is not the only one. I probably did a mistake that I didn't close all of the reports as a duplicate of this one to highlight the issue.

Comment 28 Jaroslav Mracek 2020-06-25 12:13:09 UTC
The issue is still valid for Fedora 32. Hope that it will be fixed in Fedora 33.

Comment 29 Fedora Blocker Bugs Application 2020-06-25 12:24:57 UTC
Proposed as a Blocker for 33-beta by Fedora user jmracek using the blocker tracking app because:

 The problem is that Fedora is shipped permanently with broken content. The fix is quite easy - mark environmental group as installed and all unwanted group will be marked as excluded. DNF and Anaconda team invested a lot of time in deliver perfect product and all is wasted by incorrect recipe to create Fedora images.
The issue is there for several years, and users of Fedora that plays with env. groups periodicity reports the problem. Therefore it has negative impact on Fedora project and it consume limited resources of DNF team.

Comment 30 Adam Williamson 2020-06-25 15:45:37 UTC
We already discussed this as a blocker and rejected it for the last release cycle. Nothing about it has changed. There's very little point in discussing it again. I am still -1.

Comment 31 Adam Williamson 2020-06-25 16:07:53 UTC
the desired changes here are straightforward, though, so why not just go ahead and send a PR for https://pagure.io/fedora-kickstarts ?

Comment 32 Chris Murphy 2020-06-25 16:20:24 UTC
I mention in comment 24 that 'dnf group list --installed' does list 'Fedora Workstation' on a clean installed system. So I can't tell what this bug is still complaining about. Upgrades? Or some other indication of environment other than what I'm using? I don't think it can be fixed during an upgrade.

Comment 33 Aki Ketolainen 2020-06-27 21:53:12 UTC
Chris Murphy: dnf group list --installed is empty after installation of Fedora 31 KDE Spin (F31-KDE-x86_64-LIVE-20200113.iso).

[akik@localhost ~]$ sudo -i dnf group list --installed

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for akik: 
Fedora Modular 31 - x86_64                   562 kB/s | 5.2 MB     00:09    
Fedora Modular 31 - x86_64 - Updates         260 kB/s | 4.1 MB     00:16    
Fedora 31 - x86_64 - Updates                 949 kB/s |  26 MB     00:27    
Fedora 31 - x86_64                           3.4 MB/s |  71 MB     00:20    
Last metadata expiration check: 0:00:01 ago on Sun 28 Jun 2020 12:45:29 AM EEST.
[akik@localhost ~]$

Comment 34 Chris Murphy 2020-06-27 22:04:18 UTC
I guess it should be 'KDE Plasma Workspaces'?

Comment 35 Aki Ketolainen 2020-06-27 22:17:39 UTC
Chris Murhpy: Yes that's listed in sudo -i dnf group list.

Comment 36 Ben Cotton 2020-08-11 13:04:40 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 33 development cycle.
Changing version to 33.

Comment 37 Pavel Holica 2020-09-12 17:07:40 UTC
Please note this bug can be seen when installing some other spins too. I've just installed XFCE spin of Fedora-33-20200911.n.0 and observed similar behaviour:

# dnf groups list --installed
Last metadata expiration check: 0:10:50 ago on Sat 12 Sep 2020 06:54:06 PM CEST.
Installed Groups:
   Administration Tools

Comment 38 Jaroslav Mracek 2020-11-16 15:44:21 UTC
I create PR - that should fix the issue - https://pagure.io/fedora-kickstarts/pull-request/729

Comment 39 Ben Cotton 2021-11-04 14:46:44 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 40 Ben Cotton 2021-11-04 15:45:03 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 41 Adam Williamson 2021-11-05 23:07:22 UTC
well, the PR that intended to fix this got merged. So I think we can let it die, unless someone found a case where this is still true in F34/F35.

Comment 42 Ben Cotton 2021-11-30 16:12:16 UTC
Fedora 33 changed to end-of-life (EOL) status on 2021-11-30. Fedora 33 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.