Bug 1555134 - anaconda does not use comps arch filtering
Summary: anaconda does not use comps arch filtering
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: 32
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Jiri Konecny
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-03-14 02:13 UTC by Adam Williamson
Modified: 2021-04-15 11:18 UTC (History)
15 users (show)

Fixed In Version: anaconda-32.20-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-04-15 11:18:49 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Adam Williamson 2018-03-14 02:13:26 UTC
comps allows syntax like this:

      <packagereq type="mandatory" arch="i386,x86_64">grub2-pc</packagereq>

to signify that a package should only be included on particular arches. However, you have to do something specific to actually make this take effect: when calling the dnf Base instance's `read_comps` method, you have to set `arch_filter=True`. The patch would be this:

--- a/pyanaconda/payload/dnfpayload.py
+++ b/pyanaconda/payload/dnfpayload.py
@@ -550,7 +550,7 @@ class DNFPayload(payload.PackagePayload):
         # and group properties. Unset reposdir to ensure dnf has nothing it can
         # check automatically
         conf.reposdir = []
-        self._base.read_comps()
+        self._base.read_comps(arch_filter=True)
 
         conf.reposdir = REPO_DIRS
 
@@ -893,7 +893,7 @@ class DNFPayload(payload.PackagePayload):
             for repo in self._base.repos.iter_enabled():
                 self._sync_metadata(repo)
         self._base.fill_sack(load_system_repo=False)
-        self._base.read_comps()
+        self._base.read_comps(arch_filter=True)
         self._refreshEnvironmentAddons()
 
     def install(self):

     
I'm filing this as an issue rather than a PR as we should probably kick around the consequences of doing it a bit. The *reason* to do it would be to let us make comps fully correct: without this mechanism we cannot list precisely the packages required in each comps group, without any *non-existent* packages, because some packages are essential on one arch but do not exist on another (bootloader packages are a good example). If we ever want to make composes fail on a group package not existing again (which seems like a good idea at least in theory), we need this.

However, there *could* be unexpected consequences. The arch actually used to filter comps with would be the dnf basearch; we'd have to consider whether there are any cases where we might actually wind up filtering by the *wrong* basearch (e.g. if we're producing an i686 image on an x86_64 system, or something), and if so, can this be addressed.

The same situation exists in lorax, and I filed https://github.com/rhinstaller/lorax/issues/336 for that.

Comment 1 Martin Kolman 2018-03-14 11:36:52 UTC
I'm not sure we can do much from the Anaconda side - we don't really have much non-x86 testing resources and I'm not sure how to even test stuff like this. Also the API calls go directly to DNF & depend on correctness of the comps metadata. 

So I think this basically requires a discussion/testing/ACKing of the solution by all the stakeholders:
DNF
release engineering
non-x86 arch people

So I'm adding some relevant people to CC, please add more if I missed someone.

Comment 2 Jaroslav Mracek 2018-03-14 11:55:06 UTC
I created a patch that should allow comps filtering according to arch (https://github.com/rhinstaller/anaconda/pull/1392)

Comment 3 Adam Williamson 2018-03-14 18:07:23 UTC
I'm working on trying to make the comps at least for Fedora "arch-correct", but it's of course a pointless effort if anaconda doesn't apply the filter :)

My *instinct* is that there shouldn't actually be a problem, the 'active' basearch for any given compose operation should be the right one so far as the arch filtering is concerned. After all, it's not like we install x86_64 packages into i686 images even if we compose the i686 image on an x86_64 system, and presumably that means the basearch for the transaction is correct. But from experience this felt like the kind of thing where there might be some weird, non-obvious case that would be problematic, so I thought I'd best flag it up for discussion before just sending a PR.

Comment 4 Dennis Gilmore 2018-03-14 19:07:22 UTC
the arch lines in comps have only ever been used and supported by pungi for filtering packages from variants

Comment 5 Adam Williamson 2018-03-14 19:10:56 UTC
DNF has been applying the arch filter in CLI operation for several years. See:

https://github.com/rpm-software-management/dnf/commit/5350a4145e799e5d84ffe1b1c31981bea2339eba

It is also clearly the case that we *ought* to do this, if ever want comps to be fully correct. There is no other plausible way to do things for stuff like the bootloader packages, which *must* be included on certain arches but cannot sensibly exist on other arches. If we ever want to get to the point where we can assert that comps only lists packages that actually exist, which seems like a reasonable goal, we need the arch filter to achieve this.

Comment 6 Jan Kurik 2018-08-14 10:57:03 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle.
Changing version to '29'.

Comment 7 Ben Cotton 2019-10-31 20:45:50 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 8 Adam Williamson 2019-11-04 19:21:21 UTC
PR is still outstanding. Let's tag this as a feature request as that's more or less what it is.

Comment 9 Jiri Konecny 2020-01-08 13:54:01 UTC
I'll handle this bug from now on. Waiting for the PR rebase to fix conflicts. Then I'll test it and merge it.

Comment 10 Jiri Konecny 2020-01-15 12:37:38 UTC
I've done sanity checks and nothing seems to be broken from the original scenario.

Adam if you know how to test this functionality feel free to test this on a new Anaconda version.

Comment 11 Adam Williamson 2020-01-15 13:26:13 UTC
Well, it should actually change the behaviour of existing installs slightly, when using groups where we've already put arch filters into comps, e.g. anaconda-tools (if there's actually a scenario where a run of anaconda itself pulls that group in, anyway) and base-x. Easiest check would probably be if we can tell from the logs whether an install with the base-x group used to try and pull in xorg-x11-drv-armada on arches other than ARM, but now does not, as we have this line in the base-x group:

      <packagereq arch="armhfp">xorg-x11-drv-armada</packagereq>

Comment 12 Jiri Konecny 2020-01-15 14:45:31 UTC
Maybe I'm looking wrong but I don't see that package anywhere on x86_64 Everything comps.

Comment 14 Jiri Konecny 2020-01-21 12:38:21 UTC
Anaconda has this feature enabled from now on. So we can start mixing packages in the repositories if there will be requirement.

Comment 15 Ben Cotton 2020-02-11 15:40:49 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.


Note You need to log in before you can comment on or make changes to this bug.