Bug 1131969 - [rfe] group_install: accepts package specs as excludes
Summary: [rfe] group_install: accepts package specs as excludes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Honza Silhan
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-08-20 12:25 UTC by Bohuslav "Slavek" Kabrda
Modified: 2014-09-23 04:40 UTC (History)
6 users (show)

Fixed In Version: dnf-0.6.1-1.fc21
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-09-23 04:40:15 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Bohuslav "Slavek" Kabrda 2014-08-20 12:25:21 UTC
What I'm trying to achieve is resolving livecd dependencies from kickstart in the way that livecd-creator does.
livecd-creator parses kickstart, creates a Yum transaction, adds required groups and packages to the transaction and then _removes packages_ that are listed in the kickstart to be removed. DNF can almost match that behaviour, but kickstarts also allow specifying dependencies for removal in format "name*", for example "imsettings*" (e.g. matching imsettings-foo etc).
While yum can remove packages from an existing transaction (that's how I read the python-imgcreate code [1]), DNF doesn't allow that, nor it allows excluding packages from added group based on this type of "globbed" name.
This makes DNF unusable not just for my simple script, but also for livecd-creator in the future, so it should be fixed - otherwise DNF won't be able to properly resolve livecd dependencies from kickstarts.

There are basically two ways to fix this, I think:
- either recognizing "name*" in "exclude" parameter of base.group_install [2]
- or allowing to remove members from transaction (which could turn out to be very problematic I think)

Thanks for looking into this!


[1] There is no web-browsable code repository for livecd-tools AFAICS, one has to "git clone git://git.fedorahosted.org/livecd" and then see method "deselectPackage" in livecd/imgcreate/yuminst.py - in revision e0833c8, it's on line 114 of that file.
[2] http://akozumpl.github.io/dnf/api_base.html#dnf.Base.group_install

Comment 1 Honza Silhan 2014-08-20 13:19:38 UTC
Hi, thanks for the report, kickstarter plugin should support removing packages. Excluding packages from transaction is IMO bad idea, this should be done before depsolving otherwise you could remove dependency. We can do that by passing exclude arg of concatenated Subject(pattern).get_best_query(sack) into Base.group_install.

Comment 2 Ales Kozumplik 2014-08-21 07:52:25 UTC
Jan, I think Slavek here simply wants an extended semantics on the parameters accepted by Base.group_install() which he can call from his own code, not use the kickstart plugin.

Comment 3 Bohuslav "Slavek" Kabrda 2014-08-21 08:48:12 UTC
(In reply to Ales Kozumplik from comment #2)
> Jan, I think Slavek here simply wants an extended semantics on the
> parameters accepted by Base.group_install() which he can call from his own
> code, not use the kickstart plugin.

Yes, precisely. What I want is to be able to achieve exclusion of "name*" from selected group using DNF Base API in my code. Thanks for looking into this.

Comment 4 Honza Silhan 2014-08-22 09:02:44 UTC
Ok, I misunderstand that. exclude argument will accept package specs then.

Comment 5 Honza Silhan 2014-08-22 09:55:18 UTC
* misunderstood

Comment 6 Honza Silhan 2014-08-27 19:43:14 UTC
Feature added in the upstream. Excludes are package name glob patterns (http://dnf.readthedocs.org/en/latest/api_base.html#dnf.Base.group_install).

Comment 7 Bohuslav "Slavek" Kabrda 2014-08-28 07:30:04 UTC
(In reply to Jan Silhan from comment #6)
> Feature added in the upstream. Excludes are package name glob patterns
> (http://dnf.readthedocs.org/en/latest/api_base.html#dnf.Base.group_install).

Thank you very much!

Comment 8 Bohuslav "Slavek" Kabrda 2014-08-29 08:24:13 UTC
Ok, so I've just tried to use dnf 0.6.1 for the task. It mostly works, but there is still one problem. As documentation notes: "but e.g. packages tagged mandatory will be marked for installation no matter the value of exclude". I can understand why you would want to do this, but it makes DNF unusable for the general use-case of resolving dependencies from kickstart. An example is fedora-workstation-packages.ks [1], which adds group "@gnome-desktop" and excludes "deja-dup", which is a mandatory package for that group.
Could you please also consider excluding mandatory packages? I'm afraid that even in current state DNF would still be unusable by livecd-creator (or my script, for that matter :)).
Thanks


[1] https://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-workstation-packages.ks?id=d2b1c2a323f82676062e25a17fe33acea1ff0655

Comment 9 Honza Silhan 2014-08-29 10:55:36 UTC
By common sense when packages are mandatory they should stay mandatory. They are marked so for some reason. How many cases are when mandatory packages are removed from group in kickstarts? Maybe file a bug to exclude them from mandatories or install the rest of packages selectively.

Comment 10 Bohuslav "Slavek" Kabrda 2014-08-29 11:52:36 UTC
(In reply to Jan Silhan from comment #9)
> By common sense when packages are mandatory they should stay mandatory. They
> are marked so for some reason. How many cases are when mandatory packages
> are removed from group in kickstarts? Maybe file a bug to exclude them from
> mandatories or install the rest of packages selectively.

The way I look at it, there are two ways of understanding groups. The first (which I think you're seeing right now) is that groups are coherent set of packages that need to be installed together to achieve something. The other perspective (a pragmatic one, applied by kickstarts IIUC) is that groups are collections of packages that are useful, because you can use a group name instead of providing insanely long list of packages - but sometimes you just don't want that whole group, because you need something bit different.

What I'm trying to say here is that "mandatory" is a point of view - what's mandatory for one use case can be "optional" for the other one. (Yes, we could argue that people who create kickstarts should then create different groups that would have precisely the mandatory packages that they need, but I don't think that it'd be very practical).

Expanding your thought, why does DNF allow adding only "default" packages from a group to transaction, e.g. without "mandatory" (tested on @libreoffice, seems to work)? By your logic, "mandatory" should either be added by default or exception should be raised if it is not provided.

For the record, I don't have idea how rare this case is and finding out wouldn't be that trivial, I think.

Thanks

Comment 11 Ales Kozumplik 2014-09-01 07:18:05 UTC
Hello Jan and Slavek,

you both have valuable insights into the issue. On one hand, if we want to take packaging further we need to start considering groups as indivisible objects in their own right: the DNF's CLI has made big strides in that direction recently. On the other hand, currently there are use cases where people (wrongly) consider groups just roughly gathered lists of packages. My vision of how groups should work leans towards the former and projects like livecd-creator can only operate by explicitly stating they do not use groups this way or by accepting that mandatory packages can not be excluded (and potentially taking it up with comps authors).

Having said that, it should be possible for API clients to introspect groups and run arbitrary operations on their contents. And something like the following already works in the current API:

group = base.comps.group_by_id('some-group')
for pkg in group.packages_iter():
    if pkg.name in desired_set:
        base.install(pkg.name)

Hope that helps.

Comment 12 Bohuslav "Slavek" Kabrda 2014-09-02 13:48:56 UTC
Hi Ales,
thanks, that makes sense. I guess I can do what you've demonstrated in the example and use fnmatch to exclude the unwanted "globbed" package names.

Thanks a lot to both of you!

Comment 13 Fedora Update System 2014-09-04 15:49:31 UTC
dnf-0.6.1-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/dnf-0.6.1-1.fc21

Comment 14 Fedora Update System 2014-09-06 01:00:01 UTC
Package dnf-0.6.1-1.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dnf-0.6.1-1.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-10199/dnf-0.6.1-1.fc21
then log in and leave karma (feedback).

Comment 15 Fedora Update System 2014-09-23 04:40:15 UTC
dnf-0.6.1-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.