Bug 1226105

Summary:

dracut reports /dev/root does not exist with virtio, and multipathd failed to get path uid

Product:

Red Hat Enterprise Linux 7

Reporter:

FuXiangChun <xfu>

Component:

device-mapper-multipath

Assignee:

Ben Marzinski <bmarzins>

Status:

CLOSED INSUFFICIENT_DATA

QA Contact:

Lin Li <lilin>

Severity:

high

Docs Contact:

Priority:

high

Version:

7.2

CC:

agk, bmarzins, bugproxy, chayang, cjdick, dan, dracut-maint-list, dwysocha, extras-qa, fdinitto, gduarte, hannsj_uhl, harald, heinzm, jbrassow, jjarvis, jonathan, juzhang, lilin, loberman, lvm-team, mcsontos, michen, msnitzer, normand, prajnoha, prockai, qzhang, xfu, yanwang, zbyszek

Target Milestone:

Keywords:

Triaged

Target Release:

7.2

Hardware:

x86_64

OS:

Linux

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

1153080

Environment:

Last Closed:

2016-03-16 13:43:22 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

1153080

Bug Blocks:

1118366, 1295577, 1313485

Attachments:

Description	Flags
rhel7 os installation log	none

Description FuXiangChun 2015-05-29 02:32:57 UTC

+++ This bug was initially created as a clone of Bug #1153080 +++

Description of problem:
Fail to install RHEL7.1 and RHEL7.2 os on Intel Skylake host. show error log as below.  but success to install RHEL6 os on host.  I attached full installation log.  Tested two RHEL7 tree like this.
1. http://download.eng.bos.redhat.com/nightly/RHEL-7.2-20150527.n.0/compose/Server/x86_64/os/

2. http://download.eng.bos.redhat.com/released/RHEL-7/7.1/Server/x86_64/os/

..............
May 28 21:36:55 localhost multipathd[381]: sda: add path (uevent)
May 28 21:36:55 localhost multipathd[381]: sda: spurious uevent, path already in
May 28 21:36:55 localhost kernel: scsi 0:0:0:0: Direct-Access     KVM      vmDis
May 28 21:36:55 localhost kernel: sd 0:0:0:0: [sdb] Attached SCSI removable disk
May 28 21:36:55 localhost multipathd[381]: sdb: add path (uevent)
May 28 21:36:55 localhost multipathd[381]: sdb: failed to get path uid
May 28 21:36:55 localhost multipathd[381]: uevent trigger error
May 28 21:39:59 localhost dracut-initqueue[656]: Warning: Could not boot.
May 28 21:39:59 localhost dracut-initqueue[656]: Warning: /dev/root does not exi
May 28 21:39:59 localhost systemd[1]: Starting Dracut Emergency Shell...
..............

Comment 3 FuXiangChun 2015-05-29 02:40:40 UTC

Created attachment 1031550 [details]
rhel7 os installation log

Comment 4 FuXiangChun 2015-05-29 09:27:58 UTC

As this bug block installation os on host. so set priority and severity as High.

Comment 5 Ben Marzinski 2015-05-29 16:50:54 UTC

Are sda and sdb actual scsi devices?  They aren't coming up as vda and vdb, like I'd expect virtio devices to.

When dracut starts the emergency shell are you able to add the device there? Try running:

# multipathd add path sdb

If that fails, could you try running:

# udevadm info /dev/sdb | grep ID_SERIAL

If that doesn't show anything, try running

# echo "change" > /sys/block/sdb/uevent

and then retry the other two commands to see if they work now.

Comment 6 FuXiangChun 2015-06-01 03:27:00 UTC

(In reply to Ben Marzinski from comment #5)
> Are sda and sdb actual scsi devices?  They aren't coming up as vda and vdb,
> like I'd expect virtio devices to.

yep, sda and sdb is used as scsi devices.
> 
> When dracut starts the emergency shell are you able to add the device there?
> Try running:
> 
> # multipathd add path sdb
> 
fail

> If that fails, could you try running:
> 
> # udevadm info /dev/sdb | grep ID_SERIAL
> 

E: ID_SERIAL=KVM_vmDisk_10343009E8B-0:0
E: ID_SERIAL_SHORT=10343009E8B

> If that doesn't show anything, try running
> 
> # echo "change" > /sys/block/sdb/uevent
> 
> and then retry the other two commands to see if they work now.


In additional, This is content of sdb uevent file. e.g
#cat /sys/block/sdb/uevent
MAJOR=8
MINOR=16
DEVNAME=sdb
DEVTYPE=disk

Comment 7 Ben Marzinski 2015-06-01 17:25:38 UTC

O.k. So multipathd is still failing to add the path, but ID_SERIAL is there, so it should be able to get the uid.  instead of running

# multipathd add path sdb

could you run

# multipath -v /dev/sdb

Hopefully that will fail too, but with more information about what went wrong.

Comment 8 FuXiangChun 2015-06-02 01:56:31 UTC

(In reply to Ben Marzinski from comment #7)
> O.k. So multipathd is still failing to add the path, but ID_SERIAL is there,
> so it should be able to get the uid.  instead of running
> 
> # multipathd add path sdb
> 
> could you run
> 
> # multipath -v /dev/sdb
> 
> Hopefully that will fail too, but with more information about what went
> wrong.

I guess you want to get this info.

# multipath -v 3 /dev/sdb  
Jun 01 21:49:17 | loading /lib64/multipath/libcheckdirectio.so checker
Jun 01 21:49:17 | loading /lib64/multipath/libprioconst.so prioritizer
Jun 01 21:49:17 | sdb: not found in pathvec
Jun 01 21:49:17 | sdb: mask = 0x31
Jun 01 21:49:17 | sdb: dev_t = 8:16
Jun 01 21:49:17 | sdb: size = 0
Jun 01 21:49:17 | sdb: vendor = KVM     
Jun 01 21:49:17 | sdb: product = vmDisk          
Jun 01 21:49:17 | sdb: rev = 0.01
Jun 01 21:49:17 | sdb: h:b:t:l = 0:0:0:0
Jun 01 21:49:17 | sdb: path state = running

Jun 01 21:49:17 | scope is nul
Jun 01 21:49:17 | unloading const prioritizer
Jun 01 21:49:17 | unloading directio checker

additional info:
1.#multipath -v /dev/sdb missing option 'lvl', so it cann't get correct result.

2. -v option help info as below

-v lvl  verbosity level
          . 0 no output
          . 1 print created devmap names only
          . 2 default verbosity
          . 3 print debug information

3. This 3 commands haven't any output message.
dracut:/# multipath -v 1 /dev/sdb 
dracut:/# multipath -v 2 /dev/sdb 
dracut:/# multipath -v 0 /dev/sdb

Comment 9 Ben Marzinski 2015-06-02 18:42:44 UTC

(In reply to FuXiangChun from comment #8)
> (In reply to Ben Marzinski from comment #7)
> > O.k. So multipathd is still failing to add the path, but ID_SERIAL is there,
> > so it should be able to get the uid.  instead of running
> > 
> > # multipathd add path sdb
> > 
> > could you run
> > 
> > # multipath -v /dev/sdb
> > 
> > Hopefully that will fail too, but with more information about what went
> > wrong.
> 
> I guess you want to get this info.

Wasn't there more printed?  Can I see the entire output?

> # multipath -v 3 /dev/sdb  
> Jun 01 21:49:17 | loading /lib64/multipath/libcheckdirectio.so checker
> Jun 01 21:49:17 | loading /lib64/multipath/libprioconst.so prioritizer
> Jun 01 21:49:17 | sdb: not found in pathvec
> Jun 01 21:49:17 | sdb: mask = 0x31
> Jun 01 21:49:17 | sdb: dev_t = 8:16
> Jun 01 21:49:17 | sdb: size = 0
> Jun 01 21:49:17 | sdb: vendor = KVM     
> Jun 01 21:49:17 | sdb: product = vmDisk          
> Jun 01 21:49:17 | sdb: rev = 0.01
> Jun 01 21:49:17 | sdb: h:b:t:l = 0:0:0:0
> Jun 01 21:49:17 | sdb: path state = running
> 
> Jun 01 21:49:17 | scope is nul
> Jun 01 21:49:17 | unloading const prioritizer
> Jun 01 21:49:17 | unloading directio checker
>

Comment 10 FuXiangChun 2015-06-04 05:12:32 UTC

(In reply to Ben Marzinski from comment #9)
> (In reply to FuXiangChun from comment #8)
> > (In reply to Ben Marzinski from comment #7)
> > > O.k. So multipathd is still failing to add the path, but ID_SERIAL is there,
> > > so it should be able to get the uid.  instead of running
> > > 
> > > # multipathd add path sdb
> > > 
> > > could you run
> > > 
> > > # multipath -v /dev/sdb
> > > 
> > > Hopefully that will fail too, but with more information about what went
> > > wrong.
> > 
> > I guess you want to get this info.
> 
> Wasn't there more printed?  Can I see the entire output?

sorry, Host has been returned to beaker. and os is re-installed.  I tried to re-run multipath command in dracut. As didn't find /dev/sdb.  so I executed this multipath command for /dev/sda.  not sure if it is useful. 

dracut:/# multipath -v 3 /dev/sda 
Jun 04 01:07:52 | loading /lib64/multipath/libcheckdirectio.so checker
Jun 04 01:07:52 | loading /lib64/multipath/libprioconst.so prioritizer
Jun 04 01:07:52 | sda: not found in pathvec
Jun 04 01:07:52 | sda: mask = 0x31
Jun 04 01:07:52 | sda: dev_t = 8:0
Jun 04 01:07:52 | sda: size = 937703088
Jun 04 01:07:52 | sda: vendor = ATA     
Jun 04 01:07:52 | sda: product = INTEL SSDSC2BW48
Jun 04 01:07:52 | sda: rev = DC32
Jun 04 01:07:52 | sda: h:b:t:l = 0:0:0:0
Jun 04 01:07:52 | sda: tgt_node_name = ata-1.00
Jun 04 01:07:52 | sda: path state = running

Jun 04 01:07:52 | sda: uid_attribute = ID_SERIAL (internal default)
Jun 04 01:07:52 | sda: uid = INTEL_SSDSC2BW480A4_PHDA438500EA4805GN (udev)
Jun 04 01:07:52 | scope limited to INTEL_SSDSC2BW480A4_PHDA438500EA4805GN
Jun 04 01:07:52 | sda: mask = 0x1f
Jun 04 01:07:52 | sda: dev_t = 8:0
Jun 04 01:07:52 | sda: size = 937703088
Jun 04 01:07:52 | sda: vendor = ATA     
Jun 04 01:07:52 | sda: product = INTEL SSDSC2BW48
Jun 04 01:07:52 | sda: rev = DC32
Jun 04 01:07:52 | sda: h:b:t:l = 0:0:0:0
Jun 04 01:07:52 | sda: tgt_node_name = ata-1.00
Jun 04 01:07:52 | sda: path state = running

Jun 04 01:07:52 | sda: 58369 cyl, 255 heads, 63 sectors/track, start at 0
Jun 04 01:07:52 | sda: serial = PHDA438500EA4805GN  
Jun 04 01:07:52 | sda: get_state
Jun 04 01:07:52 | sda: path checker = directio (internal default)
Jun 04 01:07:52 | sda: checker timeout = 30000 ms (sysfs setting)
Jun 04 01:07:52 | directio: starting new request
Jun 04 01:07:52 | directio: io finished 4096/0
Jun 04 01:07:52 | sda: state = up
Jun 04 01:07:52 | sda: detect_prio = 1 (config file default)
Jun 04 01:07:52 | sda: prio = const (internal default)
Jun 04 01:07:52 | sda: prio =  (internal default)
Jun 04 01:07:52 | sda: const prio = 1
Jun 04 01:07:52 | dm-0: device node name blacklisted
Jun 04 01:07:52 | dm-1: device node name blacklisted
Jun 04 01:07:52 | dm-2: device node name blacklisted
===== paths list =====
uuid                                   hcil    dev dev_t pri dm_st chk_st vend
INTEL_SSDSC2BW480A4_PHDA438500EA4805GN 0:0:0:0 sda 8:0   1   undef ready  ATA 
Jun 04 01:07:52 | params = 0 0 1 1 service-time 0 1 2 8:0 1 1 
Jun 04 01:07:52 | status = 2 0 0 0 1 1 A 0 1 2 8:0 A 0 0 1 
Jun 04 01:07:52 | mpatha: disassemble map [0 0 1 1 service-time 0 1 2 8:0 1 1 ]
Jun 04 01:07:52 | mpatha: disassemble status [2 0 0 0 1 1 A 0 1 2 8:0 A 0 0 1 ]
Jun 04 01:07:52 | directio checker refcount 1
Jun 04 01:07:52 | const prioritizer refcount 1
Jun 04 01:07:52 | unloading const prioritizer
Jun 04 01:07:52 | unloading directio checker

> 
> > # multipath -v 3 /dev/sdb  
> > Jun 01 21:49:17 | loading /lib64/multipath/libcheckdirectio.so checker
> > Jun 01 21:49:17 | loading /lib64/multipath/libprioconst.so prioritizer
> > Jun 01 21:49:17 | sdb: not found in pathvec
> > Jun 01 21:49:17 | sdb: mask = 0x31
> > Jun 01 21:49:17 | sdb: dev_t = 8:16
> > Jun 01 21:49:17 | sdb: size = 0
> > Jun 01 21:49:17 | sdb: vendor = KVM     
> > Jun 01 21:49:17 | sdb: product = vmDisk          
> > Jun 01 21:49:17 | sdb: rev = 0.01
> > Jun 01 21:49:17 | sdb: h:b:t:l = 0:0:0:0
> > Jun 01 21:49:17 | sdb: path state = running
> > 
> > Jun 01 21:49:17 | scope is nul
> > Jun 01 21:49:17 | unloading const prioritizer
> > Jun 01 21:49:17 | unloading directio checker
> >

Comment 11 Ben Marzinski 2015-06-08 19:02:49 UTC

Unfortunately, sda looks like it is working fine in this output, so it doesn't really help.

Comment 12 Ben Marzinski 2015-08-05 21:35:04 UTC

Would it be possible to get reproducer instructions for this issue, so I can try it out?

Comment 16 IBM Bug Proxy 2016-01-23 02:51:17 UTC

------- Comment From chavez.com 2016-01-22 21:43 EDT-------
Can this bug be closed? There haven't been any updates in 5 months.

Comment 17 Ben Marzinski 2016-02-08 20:07:46 UTC

I am fine with closing this INSUFFICIENT_DATA, unless there are any objections.

Comment 18 John Jarvis 2016-03-16 13:43:22 UTC

Closing to match IBM status.