Bug 1504602

Summary: Image build fails because of dracut warning "/dev/root does not exist"
Product: [Fedora] Fedora Reporter: Tomas Mlcoch <tmlcoch>
Component: loraxAssignee: Brian Lane <bcl>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 27CC: anaconda-maint-list, bcl, dcantrell, dmach, gmarr, herrold, mgracik, sgallagh, tmlcoch, wwoods
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: RejectedBlocker
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-01-29 16:42:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tomas Mlcoch 2017-10-20 10:06:26 UTC 
Compose: Fedora-Modular-Bikeshed-20171019.n.1

fedora-modular-container-base-minimal generation failed for ppc64le and aarch64:
https://koji.fedoraproject.org/koji/taskinfo?taskID=22555870

Because of dracut warning "/dev/root does not exist" and entering interactive mode, see screenshot in the failed task:
https://koji.fedoraproject.org/koji/taskinfo?taskID=22555875


Snippet from the compose log:

[tmlcoch@branched-composer ~][PROD]$ tail /mnt/koji/compose/Fedora-Modular-Bikeshed-20171019.n.1/logs/global/pungi.global.log
2017-10-19 19:52:07 [INFO    ] [DONE ] ---------- PHASE: LIVE_IMAGES ----------
2017-10-19 19:52:07 [DEBUG   ] Notification: '/usr/bin/pungi-fedmsg-notification' 'phase-stop', {'phase_name': 'live_images', 'location': 'http://kojipkgs.fedoraproject.org/compose/Fedora-Modular-Bikeshed-20171019.n.1/compose', 'compose_id': u'Fedora-Modular-Bikeshed-20171019.n.1'}
2017-10-19 19:52:09 [INFO    ] [DONE ] ---------- PHASE: CREATEISO ----------
2017-10-19 19:52:09 [DEBUG   ] Notification: '/usr/bin/pungi-fedmsg-notification' 'phase-stop', {'phase_name': 'createiso', 'location': 'http://kojipkgs.fedoraproject.org/compose/Fedora-Modular-Bikeshed-20171019.n.1/compose', 'compose_id': u'Fedora-Modular-Bikeshed-20171019.n.1'}
2017-10-19 19:52:10 [ERROR   ] Compose run failed: [Errno 2] No such file or directory: '/mnt/koji/packages/Fedora-Modular-Container-Minimal/Bikeshed/20171019.n.1/images/Fedora-Modular-Container-Minimal-Bikeshed-20171019.n.1.x86_64.tar.xz'
2017-10-19 19:52:10 [ERROR   ] Extended traceback in: /mnt/koji/compose/Fedora-Modular-Bikeshed-20171019.n.1/logs/global/traceback.global.log
2017-10-19 19:52:10 [CRITICAL] Compose failed: /mnt/koji/compose/Fedora-Modular-Bikeshed-20171019.n.1
2017-10-19 19:52:10 [INFO    ] Failed live on variant <Server>, arch <armhfp>, subvariant <Server>.
2017-10-19 19:52:10 [INFO    ] Failed buildinstall on variant <Server>, arch <s390x>, subvariant <None>.
2017-10-19 19:52:10 [DEBUG   ] Notification: '/usr/bin/pungi-fedmsg-notification' 'status-change', {'status': 'DOOMED', 'location': 'http://kojipkgs.fedoraproject.org/compose/Fedora-Modular-Bikeshed-20171019.n.1/compose', 'compose_id': u'Fedora-Modular-Bikeshed-20171019.n.1'}
Comment 1 Stephen Gallagher 2017-10-20 10:53:11 UTC 
Can you paste the extended traceback from /mnt/koji/compose/Fedora-Modular-Bikeshed-20171019.n.1/logs/global/traceback.global.log please?
Comment 2 Tomas Mlcoch 2017-10-20 13:22:51 UTC 
Extended logs from the buildinstall task could be found here:

https://kojipkgs.fedoraproject.org/compose/Fedora-Modular-Bikeshed-20171019.n.1/logs/ppc64le/buildinstall-Server-logs/
Comment 3 Brian Lane 2017-10-20 16:51:08 UTC 
I don't really understand how this is being used, but it looks like oz/imagefactory is involved in the booting?

The error in the screenshot is typically caused by not telling dracut where to find the root device when booting.

From https://kojipkgs.fedoraproject.org//work/tasks/5875/22555875/oz-ppc64le.log:

method=https://kojipkgs.fedoraproject.org/compose/Fedora-Modular-Bikeshed-20171019.n.1/compose/Server/ppc64le/os ks=file:/ks.cfg

1. method is a REALLY old way to specify this and needs to be updated to whatever the current version of anaconda prefers.

2. Is this working for x86 or any other arch? Has it worked in the past? Compare logs and see what's different.

I don't think this is a lorax issue, it's either oz/imagefactory or whatever you are using to launch it (koji?)
Comment 4 Geoffrey Marr 2017-10-23 21:34:17 UTC 
Discussed during the 2017-10-23 blocker review meeting: [1]

The decision to classify this bug as an AcceptedBlocker was made, even though there is no current criteria to warrant blocking on this. We plan to make a near-future change to the criteria so that a bug like this from this point on will be blocker-qualifying.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2017-10-23/f27-blocker-review.2017-10-23-16.00.txt
Comment 5 Geoffrey Marr 2017-10-23 21:42:26 UTC 
Discussed during the 2017-10-23 blocker review meeting: [1]

The decision to classify this bug as an AcceptedBlocker was made as this failure causes composition of images to fail altogether.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2017-10-23/f27-blocker-review.2017-10-23-16.00.txt
Comment 6 Geoffrey Marr 2017-10-30 20:37:52 UTC 
Discussed during the 2017-10-30 blocker review meeting: [1]

The decision to reclassify this bug as a RejectedBlocker was made as we now understand this bug to be 'semi-random' and have a workaround that does not completely prevent us from composing.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2017-10-30/f27-blocker-review.2017-10-30-16.00.txt