Bug 1337731 - Live image compose fails with dnf 1.1.9
Live image compose fails with dnf 1.1.9
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: anaconda (Show other bugs)
24
x86_64 Linux
urgent Severity urgent
: ---
: ---
Assigned To: Brian Lane
Fedora Extras Quality Assurance
RejectedBlocker
:
: 1339869 (view as bug list)
Depends On:
Blocks: 1339742
  Show dependency treegraph
 
Reported: 2016-05-19 20:18 EDT by Adam Williamson
Modified: 2017-02-10 10:53 EST (History)
19 users (show)

See Also:
Fixed In Version: anaconda-24.13.5-1 anaconda-24.13.6-1.fc24
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-06-09 10:09:10 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Adam Williamson 2016-05-19 20:18:10 EDT
F24 live images all failed to compose on 2016-05-19. Looking at the logs, I see a lot of errors like this:

  File "/usr/lib/python3.5/site-packages/dnf/base.py", line 1204, in _add_comps_trans
    raise dnf.exceptions.MarkingError(it)

dnf.exceptions.MarkingError: s390utils

  File "/usr/lib/python3.5/site-packages/dnf/base.py", line 1204, in _add_comps_trans
    raise dnf.exceptions.MarkingError(it)

dnf.exceptions.MarkingError: extlinux-bootloader

These seem to be a result of this change in DNF:

https://github.com/rpm-software-management/dnf/commit/91f9ce98dbb630800017f44180d636af395435cd

which was made to address http://bugzilla.redhat.com/show_bug.cgi?id=1292892 . But it's breaking this. The relevant packages mostly seem to be in anaconda-tools. s390utils will of course not be available on non-s390 arches; extlinux-bootloader is available only on arm.

I'm not sure what the appropriate resolution here would be, whether it's a change in dnf or in comps (or even in anaconda?)

You can find all the relevant logs from here:

https://kojipkgs.fedoraproject.org/compose/branched/Fedora-24-20160519.n.0/logs/i386-x86_64/

each of those logs links to a Koji task, and you can see the actual compose logs there. e.g. the Workstation task is http://koji.fedoraproject.org/koji/taskinfo?taskID=14167034 , the KDE task is http://koji.fedoraproject.org/koji/taskinfo?taskID=14166955 , and you can see the errors in anaconda.log in each task.

This didn't affect the Rawhide compose, oddly enough; perhaps the dnf build didn't quite make it into the Rawhide compose but it did make F24...

This would be an automatic Final blocker, except that the dnf in question is not in stable yet, only updates-testing. I believe it's affecting the live compose because it's been given a buildroot override?
Comment 1 Adam Williamson 2016-05-19 20:22:16 EDT
Proposing as a Final blocker, just to keep it on the radar in case the update gets pushed out or anything.
Comment 2 Igor Gnatenko 2016-05-20 02:26:16 EDT
Hi Adam, new DNF built for f23+ including rawhide and was added to build root override for building dnf-plugins-core.

Do we have different comps for each arch? If yes, it's probably good idea to remove non-existing packages from there (s390xutils from x86_64).
Comment 3 Adam Williamson 2016-05-20 03:07:56 EDT
There is a 'basearchonly' thing that some comps entries use:

<packagereq basearchonly="true" type="default">compat-gcc-34</packagereq>

I'm not at all sure what that does. But I don't see any entries that specify something like 'include this package only for these specific arches'.
Comment 4 Michal Luscon 2016-05-20 03:24:05 EDT
Hi Adam,

the group operation install takes optional parameter strict that was previously ignored and I fixed this by the referenced commit. I think the most straightforward solution would be to use strict=False in live image composer and prepare a joint upgrade of dnf and composer. Is this suitable for you?
Comment 5 Adam Williamson 2016-05-20 03:31:40 EDT
on the face of it that sounds reasonable, sure.
Comment 6 Adam Williamson 2016-05-20 12:36:12 EDT
well, no, actually that may not be possible, as it's anaconda that's doing this. bcl, any thoughts?
Comment 7 Dennis Gilmore 2016-05-20 12:38:29 EDT
anaconda would have to set strict=False always as this will likely effect some installs, as pungi gains the ability to use dnf to make installer trees and resolve deps we will hit it there, also, I think this is a case where dnf needs to follow yums behaviour. If not we will likely hit corner cases all the time. and need to find workarounds
Comment 8 Brian Lane 2016-05-20 20:31:31 EDT
I think we're better off catching missing packages than we are silently ignoring it. comps for the various arches should be updated with the correct packages.

Anaconda also needs to start catching this:

https://github.com/rhinstaller/anaconda/pull/640
Comment 9 Adam Williamson 2016-05-20 20:33:48 EDT
The problem is that we don't have 'comps for various arches', so we can't "fix" comps.
Comment 10 Brian Lane 2016-05-20 20:55:21 EDT
Ok, well then maybe we need that? There's certainly a different file for each arch's repo. Still seems wrong to silently ignore these.
Comment 11 Adam Williamson 2016-05-20 21:02:15 EDT
well maybe, but we don't have unlimited time for this: it's breaking live composes in Rawhide already, and F24 for now; it'll stop breaking F24 when the override expires, I think, but the update shouldn't go stable until it's fixed. so...I don't know, but we don't really have weeks to sit around and kibbitz about it.
Comment 12 Honza Silhan 2016-05-23 09:14:41 EDT
If we don't have the enough time, then setting the `strict=False` in Anaconda is way to go until we find better solution. DNF does should not skip the mandatory packages by default. Otherwise we would have to deal again with the issues like why some package from group is not installed in the buildroot from koji. Reassigning...

We're glad to help with designing proper solution that would differentiate comps for different architectures. Either by
1) adding the new tag into comps + support in libcomps
2) replacing comps by metapackages with requirements and weak deps + specifying different packages for different archs by if statements in the spec file.
3) adding the new tag into comps + support into createrepo that would modify the comps according to target architecture
Comment 13 Adam Williamson 2016-05-23 12:10:13 EDT
I'm frankly not sure any of those changes is appropriate for F24. I'm not arguing against your general line of reasoning, but this does not seem to be being handled particularly well in practical terms: when a change has a significant and unexpected negative consequence on something else, surely the appropriate thing to do is loop in all concerned parties and come up with a co-ordinated plan to address that problem, rather than just pushing your own change regardless and leaving the bits all over the floor for someone else to clean up?
Comment 14 Geoffrey Marr 2016-05-23 14:01:53 EDT
Discussed during the 2016-05-23 blocker review meeting: [1]

Decision to classify this as a conditional AcceptedBlocker has been made, the condition being that if the update is pushed stable without a fix, the bug will be filed as AcceptedBlocker.

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2016-05-23/f24-blocker-review.2016-05-23-16.00.txt
Comment 15 Brian Lane 2016-05-24 14:15:38 EDT
I've proposed adding string=False for F24 even though I really think dnf should revert this.

For rawhide a better solution needs to be found for arch-specific packages in groups.
Comment 16 David Shea 2016-05-26 09:20:51 EDT
*** Bug 1339869 has been marked as a duplicate of this bug. ***
Comment 17 dsavov 2016-05-27 00:49:34 EDT
Similar problem has been detected:

I tried to install Fedora 25 rawhide from Fedora-Everything-netinst-x86_64-Rawhide-20160523.n.0.iso.
When I tried to select 'Fedora Workstation', I got the following error.

addons:         com_redhat_kdump, com_redhat_docker
cmdline:        /usr/bin/python3  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-E-dvd-x86_64-rawh quiet
dnf.rpm.log:    May 27 04:37:35 INFO --- logging initialized ---
hashmarkername: anaconda
kernel:         4.7.0-0.rc0.git5.1.fc25.x86_64
package:        anaconda-25.14-1
product:        Fedora
reason:         dnf.exceptions.MarkingError: xorg-x11-drv-armsoc
release:        Cannot get release name.
reproducible:   Not sure how to reproduce the problem
version:        rawhide
Comment 18 Geoffrey Marr 2016-05-30 14:08:20 EDT
Discussed during the 2016-05-30 blocker review meeting: [1]

Decision to classify this as a non-blocker was made, as the update has not been pushed and freeze is tomorrow so there is no longer a need to track this as a blocker.

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2016-05-30/f24-blocker-review.2016-05-30-16.01.txt
Comment 19 Fedora Update System 2016-06-06 15:41:40 EDT
anaconda-24.13.6-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-b0494cdc04
Comment 20 Fedora Update System 2016-06-07 14:54:16 EDT
anaconda-24.13.6-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-b0494cdc04
Comment 21 Fedora Update System 2016-06-09 10:08:53 EDT
anaconda-24.13.6-1.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.
Comment 22 Jaroslav Mracek 2016-08-16 08:56:53 EDT
I have created PR https://github.com/rpm-software-management/dnf/pull/558 that can solve release problems. Now basearchonly will be honored by dnf. And for packages, that are only available for certain archs, arch flag can be used and only on listed arch package will be installed like in example:

<packagereq arch arch="x86_64 s390">pkg3</packagereq>

Then the package 'pkg3' will be installed or listed with group only on machine with arch 'x86_64' or 's390'.

Please can you react to proposed changes if they are according to your needs?
Comment 23 Jaroslav Mracek 2017-02-10 10:53:47 EST
Bugs fixed in version of dnf-2.0.1-1 in rawhide.

Note You need to log in before you can comment on or make changes to this bug.