Bug 987657 - Beaker does not provision rhel7 due to dracut problem
Beaker does not provision rhel7 due to dracut problem
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: anaconda (Show other bugs)
x86_64 Linux
unspecified Severity urgent
: rc
: 7.0
Assigned To: Anaconda Maintenance Team
Release Test Team
Depends On:
Blocks: 920743
  Show dependency treegraph
Reported: 2013-07-23 15:56 EDT by Russ Anderson
Modified: 2013-09-18 08:29 EDT (History)
14 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-09-18 08:29:48 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Additional failure details. (109.44 KB, text/plain)
2013-07-23 16:00 EDT, Russ Anderson
no flags Details
kickstart contents (7.72 KB, text/plain)
2013-07-24 15:39 EDT, Will Woods
no flags Details
rdsosreport.txt with rd.debug (871.59 KB, text/plain)
2013-07-25 10:53 EDT, Russ Anderson
no flags Details

  None (edit)
Description Russ Anderson 2013-07-23 15:56:03 EDT
Description of problem:  When using Beaker to provision rhel7 on UV2, dracut complains "dracut Warning: /dev/root does not exist" and "Could not boot." It then stops at the dracut prompt.

The last lines were:
[   46.921628] dracut: Scanning for all btrfs devices
[   47.443514] dracut: Scanning for all btrfs devices
[   47.456978] dracut Warning: Could not boot.

dracut Warning: [   47.463714] dracut Warning: /dev/root does not exist
Could not boot.

Couldn't open /dev/ttyS0

dracut Warning: Could not boot.
dracut Warning: /dev/root does not exist

Generating "/run/initramfs/rdsosreport.txt"

Version-Release number of selected component (if applicable):
"RHEL-7.020130722.n.0 Server x86_64"

How reproducible: 100%

Steps to Reproduce:
1. In Beaker select system sgi-uv2-01.rhts.eng.bos.redhat.com
2. Then select  "RHEL-7.020130722.n.0 Server x86_64"
3. Look at the console to see the failure.

Actual results: Provisioning fails.

Expected results: Provisioning to work.

Additional info:  Fedora-19-Beta and Fedora19-Development fail the same way.
Specifying "root=/dev/sda1" on the linux boot line gets rid of "Warning: /dev/root does not exist" warning message, but it still complains "Could not boot." and stops in dracut.
Comment 1 Russ Anderson 2013-07-23 15:58:59 EDT
Update CC.
Comment 2 Russ Anderson 2013-07-23 16:00:40 EDT
Created attachment 777450 [details]
Additional failure details.
Comment 4 Harald Hoyer 2013-07-24 04:03:51 EDT
Well seems like a problem with the anaconda part. Don't know, maybe an kernel command line option for the network is missing...

Current kernel command line:

Comment 5 Will Woods 2013-07-24 15:39:12 EDT
Created attachment 777965 [details]
kickstart contents

Anaconda's dracut module will try to get a root device from the contents of the kickstart if the root device isn't otherwise specified.

I'm attaching the linked kickstart, just in case it goes away at some point.
Comment 6 Will Woods 2013-07-24 15:59:21 EDT
Anaconda's `parse-kickstart` command would generate the following boot arguments for the given kickstart: 

inst.text inst.repo=nfs:vtap-bos-eng01.bos.redhat.com:/vol/engineering/devarchive/redhat/nightly/RHEL-7.0-20130722.n.0/compose/Server/x86_64/os/

Normally that would be written to /etc/cmdline.d/80-kickstart.conf. That file doesn't seem to exist, which implies that the kickstart was never parsed.

So: dracut is bailing out before we have a chance to fetch/parse the kickstart.

I'm guessing that the "neednet" interface may have changed yet again, and anaconda isn't doing the right thing to get dracut to wait for the network.

Right now, anaconda does:

  set_neednet() {
      # if there's no netroot, make sure /tmp/net.ifaces exists
      [ -z "$netroot" ] && >> /tmp/net.ifaces

to tell dracut to wait until the network comes up. Harald, is this the correct way to make sure dracut will wait for the network?
Comment 7 Harald Hoyer 2013-07-25 05:43:40 EDT
getargbool 0 rd.neednet && NEEDNET=1

# Don't continue if we don't need network
if [ -z "$netroot" ] && [ ! -e "/tmp/net.ifaces" ] && [ "$NEEDNET" != "1" ]; then
Comment 8 Harald Hoyer 2013-07-25 05:49:02 EDT
Can you rerun the installation with "rd.debug" added on the kernel command line?
Comment 9 Russ Anderson 2013-07-25 10:27:56 EDT
OK, trying it right now...
Comment 10 Russ Anderson 2013-07-25 10:53:24 EDT
Created attachment 778289 [details]
rdsosreport.txt with rd.debug

Failure output with rd.debug on the installer kernel boot line.
Comment 11 Harald Hoyer 2013-07-25 13:43:32 EDT
[   22.094830] dracut: ///lib/dracut/hooks/cmdline/26-parse-anaconda-kickstart.sh@8(source): kickstart=http://beaker.engineering.redhat.com/kickstart/386387
[   22.094920] dracut: ///lib/dracut/hooks/cmdline/26-parse-anaconda-kickstart.sh@9(source): '[' -z http://beaker.engineering.redhat.com/kickstart/386387 ']'
[   22.095010] dracut: ///lib/dracut/hooks/cmdline/26-parse-anaconda-kickstart.sh@13(source): '[' http://beaker.engineering.redhat.com/kickstart/386387 ']'
[   22.095086] dracut: ///lib/dracut/hooks/cmdline/26-parse-anaconda-kickstart.sh@13(source): '[' -z '' ']'
[   22.095161] dracut: ///lib/dracut/hooks/cmdline/26-parse-anaconda-kickstart.sh@13(source): root=anaconda-kickstart
[   22.095241] dracut: ///lib/dracut/hooks/cmdline/26-parse-anaconda-kickstart.sh@15(source): case "${kickstart%%:*}" in
[   22.095313] dracut: ///lib/dracut/hooks/cmdline/26-parse-anaconda-kickstart.sh@17(source): set_neednet
[   22.095376] dracut: //lib/anaconda-lib.sh@165(set_neednet): '[' -z '' ']'
[   22.095444] dracut: ///lib/dracut/hooks/cmdline/26-parse-anaconda-kickstart.sh@33(source): export kickstart


[   23.711280] dracut: ///lib/dracut/hooks/cmdline/28-parse-anaconda-net.sh@38(source): bootif=01-08-00-69-15-C5-50
[   23.724198] dracut: ///lib/dracut/hooks/cmdline/28-parse-anaconda-net.sh@39(source): warn ''''ksdevice=<MAC>''' is deprecated. Using BOOTIF=01-08-00-69-15-C5-50 instead.'
[   23.742707] dracut: //lib/dracut-lib.sh@411(warn): check_quiet
[   23.750746] dracut: //lib/dracut-lib.sh@397(check_quiet): '[' -z yes ']'
[   23.759798] dracut: //lib/dracut-lib.sh@412(warn): echo '<28>dracut Warning: '''ksdevice=<MAC>''' is deprecated. Using BOOTIF=01-08-00-69-15-C5-50 instead.'
[   23.776968] dracut: //lib/dracut-lib.sh@413(warn): echo 'dracut Warning: '''ksdevice=<MAC>''' is deprecated. Using BOOTIF=01-08-00-69-15-C5-50 instead.'
[   23.793737] dracut: dracut Warning: 'ksdevice=<MAC>' is deprecated. Using BOOTIF=01-08-00-69-15-C5-50 instead.
[   23.806448] dracut: ///lib/dracut/hooks/cmdline/28-parse-anaconda-net.sh@40(source): echo BOOTIF=01-08-00-69-15-C5-50

[   23.947078] dracut: ///lib/dracut/hooks/cmdline/28-parse-anaconda-net.sh@64(source): '[' -n 08:00:69:15:C5:50 ']'
[   23.960072] dracut: ///lib/dracut/hooks/cmdline/28-parse-anaconda-net.sh@64(source): set_neednet
[   23.971417] dracut: //lib/anaconda-lib.sh@165(set_neednet): '[' -z '' ']'
[   23.980549] dracut: ////lib/dracut/hooks/cmdline/28-parse-anaconda-net.sh@67(source): getarg inst.dhcpclass

[   29.118663] dracut: ///lib/dracut/hooks/cmdline/98-parse-ip-opts.sh@115(source): BOOTIF=01-08-00-69-15-C5-50
[   29.118742] dracut: ////lib/dracut/hooks/cmdline/98-parse-ip-opts.sh@116(source): fix_bootif 01-08-00-69-15-C5-50

[   33.606169] dracut: ////lib/dracut/hooks/pre-udev/60-net-genrules.sh@51(source): cat /tmp/net.bootdev
[   33.618006] dracut: ///lib/dracut/hooks/pre-udev/60-net-genrules.sh@51(source): bootdev=08:00:69:15:c5:50
[   34.049954] dracut: ///lib/dracut/hooks/pre-udev/60-net-genrules.sh@73(source): echo 'ATTR{address}=="08:00:69:15:c5:50", RUN+="/sbin/initqueue --onetime /sbin/ifup $env{INTERFACE} -m", GOTO="net_end"'

So, dracut installs ifup as an initqueue job for a network interface with MAC=="08:00:69:15:c5:50"

... Looks good to me... are you sure the MAC is correct?
Comment 12 Russ Anderson 2013-07-25 14:31:24 EDT
> ... Looks good to me... are you sure the MAC is correct?

Ah, the MAC was not correct.  The correct MAC is 08:00:69:16:8C:FE .

With the correct MAC the install gets further, but fails.  It created
BZ 988508 with details.
Comment 13 Russ Anderson 2013-07-25 17:09:54 EDT
Which Kickstart device MAC is correct?

"ksdevice=08:00:69:15:c5:50" hits this problem.
"ksdevice=08:00:69:16:8C:FE" hits BZ 988508 (probably because it cannot find the right kickstart device because that is the MAC of the UV2 system.
Comment 14 Russ Anderson 2013-08-06 18:33:51 EDT
What is the correct  Kickstart device MAC?

Still seeing the dracut "/dev/root does not exist" problem.
Comment 15 Siddhesh Poyarekar 2013-08-12 05:57:19 EDT
Sorry, I only meant to add myself to cc and did not notice that the option to clear needinfo is checked by default.  Setting needinfo again.
Comment 16 Will Woods 2013-08-13 14:38:39 EDT
(In reply to Russ Anderson from comment #13)
> Which Kickstart device MAC is correct?
> "ksdevice=08:00:69:16:8C:FE" hits BZ 988508 (probably because it cannot find
> the right kickstart device because that is the MAC of the UV2 system.

No, I'm pretty sure bug 988508 is unrelated to networking. The logs there show that the install completed successfully, but there was a problem setting up the bootloader afterward.

As far as I can tell, there's no bug here - you just had the wrong MAC.
Comment 17 Russ Anderson 2013-09-18 08:29:48 EDT
Closing this as pilot error.  With the correct MAC address it is working.

Note You need to log in before you can comment on or make changes to this bug.