Bug 1224233
| Summary: | When a FC storage attached to a scsi (HBA or vHBA) with target_number!=0, the FC storage pool cannot display it with "virsh vol-list <poolName>" | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | yisun |
| Component: | libvirt | Assignee: | John Ferlan <jferlan> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.2 | CC: | dyuan, jferlan, mprivozn, rbalakri, shyu, xuzhang, yanyang, yisun |
| Target Milestone: | rc | Keywords: | Upstream |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-1.2.17-1.el7 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-11-19 06:36:29 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Any chance you can get a system into this state and then contact me so I can debug on a live system? The debug you show is for "25:*" while the lsscsi output shows "19:*", while that's perhaps just cut-n-paste related - it is "confusing". I'm hoping it's not timing related, but could well be a timing related problem. Since libvirt doesn't "control" when the devices appear and are "linked" to the /dev/sd* block device, it may be that 19:0:2:0 didn't exist when libvirt looked. Does refreshing the pool when this happens cause the volumes to appear? If so, then this is just timing related. If not, then digging deeper via a debug libvirtd will probably be necessary. FWIW: On a host I have access to, I see: # lsscsi ... [10:0:0:0] enclosu HP MSA2012fc J200 - [10:0:1:0] enclosu HP MSA2012fc J200 - [10:0:2:0] enclosu HP MSA2012fc J200 - [10:0:3:0] enclosu HP MSA2012fc J200 - [10:0:4:0] disk DGC LUNZ 0429 /dev/sdd [10:0:5:0] disk DGC LUNZ 0429 /dev/sde # virsh vol-list vhbapool_host3 Name Path ------------------------------------------------------------------------------ unit:0:4:0 /dev/disk/by-path/pci-0000:10:00.0-fc-0x5006016844602198-lun-0 unit:0:5:0 /dev/disk/by-path/pci-0000:10:00.0-fc-0x5006016044602198-lun-0 # So just from a quick look, I'm not completely convinced the problem statement is correct. A patch has been posted upstream to resolve this issue: http://www.redhat.com/archives/libvir-list/2015-June/msg00567.html I've just pushed the patch upstream:
commit 5c9fc1c11fb54236576c2bff40e6e3df1e710983
Author: John Ferlan <jferlan>
AuthorDate: Fri Jun 12 07:22:34 2015 -0400
Commit: Michal Privoznik <mprivozn>
CommitDate: Thu Jun 18 17:14:00 2015 +0200
scsi: Adjust return status from getBlockDevice
https://bugzilla.redhat.com/show_bug.cgi?id=1224233
Currently it's not possible to determine the difference between a
fatal memory allocation or failure to open/read the directory error
with a perhaps less fatal, I didn't find the "block" device in the
directory (which may be a disk entry without a block device).
In the case of the latter, we shouldn't cause failure to continue
searching in the caller (virStorageBackendSCSIFindLUs), rather we
should allow trying reading the next directory entry.
Signed-off-by: John Ferlan <jferlan>
Signed-off-by: Michal Privoznik <mprivozn>
v1.2.16-235-g5c9fc1c
verified and passed on: libvirt-1.2.17-2.el7.x86_64 qemu-kvm-rhev-2.3.0-9.el7.x86_64 kernel-3.10.0-263.el7.x86_64 (I didn't use the latest kernel because kernel has problem to create vHBA as bug 1240912 mentioned, which will block the test. I'll keep an eye on that issue) Steps: 1. choose a HBA as parent to create vHBA # ll /sys/class/fc_host/ total 0 lrwxrwxrwx. 1 root root 0 Jul 15 20:58 host4 -> ../../devices/pci0000:00/0000:00:0d.0/0000:04:00.0/host4/fc_host/host4 lrwxrwxrwx. 1 root root 0 Jul 15 20:58 host5 -> ../../devices/pci0000:00/0000:00:0d.0/0000:04:00.1/host5/fc_host/host5 <==== we'll use scsi_host5 as parent HBA. 2. create a fc pool # cat fc-pool.xml <pool type='scsi'> <name>fc-pool</name> <source> <adapter type='fc_host' parent='scsi_host5' wwnn='2101001b32a90002' wwpn='2101001b32a90002'/> </source> <target> <path>/dev/disk/by-path</path> <permissions> <mode>0700</mode> <owner>0</owner> <group>0</group> </permissions> </target> </pool> # virsh pool-create fc-pool.xml Pool fc-pool created from fc-pool.xml 3. check the newly create vHBA connected to san storage # lsscsi [0:0:0:0] disk ATA WDC WD2502ABYS-1 3B04 /dev/sda [2:0:0:0] cd/dvd PLDS DVD-ROM DH-16D3S SD11 /dev/sr0 [4:0:0:0] disk IBM 1726-4xx FAStT 0617 - [4:0:1:0] disk IBM 1726-4xx FAStT 0617 - [4:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdb [4:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sdc [5:0:0:0] disk IBM 1726-4xx FAStT 0617 - [5:0:1:0] disk IBM 1726-4xx FAStT 0617 - [5:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdd [5:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sde [13:0:0:0] disk IBM 1726-4xx FAStT 0617 - [13:0:1:0] disk IBM 1726-4xx FAStT 0617 - [13:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdf <=== new and not attached to scsi lun 13:0:0:0 [13:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sdg 4. check the vol-list can display the newly connected storage # virsh vol-list fc-pool Name Path ------------------------------------------------------------------------------ unit:0:2:0 /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0 #ll /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0 lrwxrwxrwx. 1 root root 9 Jul 16 04:51 /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0 -> ../../sdf Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2202.html |
description: When a FC storage attached to a scsi (HBA or vHBA) with target_number!=0, the FC storage pool cannot display it with "virsh vol-list <poolName>" versions: kernel-3.10.0-253.el7.x86_64 libvirt-1.2.15-2.el7.x86_64 qemu-kvm-rhev-2.3.0-1.el7.x86_64 How reproducible: 50% (when creating a npiv pool, sometimes the fc storage will always be attached to scsi target_number=0 and it's cannot be reproduced in this situation.) repro steps: 1. create a fc pool with valid fc_host&wwpn&wwnn 1.1 find an available HBA # ll /sys/class/fc_host/ total 0 lrwxrwxrwx. 1 root root 0 May 18 03:18 host4 -> ../../devices/pci0000:00/0000:00:0d.0/0000:04:00.0/host4/fc_host/host4 lrwxrwxrwx. 1 root root 0 May 18 03:18 host5 -> ../../devices/pci0000:00/0000:00:0d.0/0000:04:00.1/host5/fc_host/host5 <==== use this as parent HBA 1.2 create a pool xml #cat pool.xml <pool type='scsi'> <name>via_wwn</name> <source> <adapter type='fc_host' parent='scsi_host5' wwnn='2101001b32a90002' wwpn='2101001b32a90003'/> </source> <target> <path>/dev/disk/by-path/</path> <permissions> <mode>0755</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool> 1.3 create pool # virsh pool-create pool.xm Pool via_wwn created from pool.xm 2. Check the newly created fc pool 2.1 use lsscsi to check the new fc sotrage ( in my test, vhba scsi_host17 created) # lsscsi [0:0:0:0] disk ATA WDC WD2502ABYS-1 02.0 /dev/sda [2:0:0:0] cd/dvd PLDS DVD-ROM DH-16D3S SD11 /dev/sr0 [4:0:0:0] disk IBM 1726-4xx FAStT 0617 - [4:0:1:0] disk IBM 1726-4xx FAStT 0617 - [4:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdb [4:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sdc [5:0:0:0] disk IBM 1726-4xx FAStT 0617 - [5:0:1:0] disk IBM 1726-4xx FAStT 0617 - [5:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdd [5:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sde [17:0:0:0] disk IBM 1726-4xx FAStT 0617 /dev/sdf <========== fc disk attached to scsi x.x.0.0 [17:0:1:0] disk IBM 1726-4xx FAStT 0617 /dev/sdg [17:0:2:0] disk IBM 1726-4xx FAStT 0617 - [17:0:3:0] disk IBM 1726-4xx FAStT 0617 - 2.2 check the volume in this pool # virsh vol-list via_wwn Name Path ------------------------------------------------------------------------------ unit:0:0:0 /dev/disk/by-path/pci-0000:04:00.1-fc-0x203600a0b85b5dd4-lun-0 3.destroy and create the pool again ( again and again until the fc storage attached to a scsi host with target_number!=0) 3.1 destroy the pool # virsh pool-destroy via_wwn Pool via_wwn destroyed 3.2 start/create the pool again # virsh pool-start via_wwn Pool via_wwn started 3.3 check the fc storage # lsscsi <=== now vhba on scsi_host 19 created [0:0:0:0] disk ATA WDC WD2502ABYS-1 02.0 /dev/sda [2:0:0:0] cd/dvd PLDS DVD-ROM DH-16D3S SD11 /dev/sr0 [4:0:0:0] disk IBM 1726-4xx FAStT 0617 - [4:0:1:0] disk IBM 1726-4xx FAStT 0617 - [4:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdb [4:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sdc [5:0:0:0] disk IBM 1726-4xx FAStT 0617 - [5:0:1:0] disk IBM 1726-4xx FAStT 0617 - [5:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdd [5:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sde [19:0:0:0] disk IBM 1726-4xx FAStT 0617 - [19:0:1:0] disk IBM 1726-4xx FAStT 0617 - [19:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdf <========== fc storage attached to scsi x.x.2.0 [19:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sdg 3.4 refresh the pool # virsh pool-refresh via_wwn Pool via_wwn refreshed 3.5 list the vol in the pool , nothing displayed. # virsh vol-list via_wwn Name Path ------------------------------------------------------------------------------ <====================== no volume shows up here. Actual result: When FC storage attached on scsi (HBA/vHBA) doesn't start from target=0, it disappears in pool's volume list. Expected result: The volumes in the pool should always be listed no matter if it starts from target=0. Additional info: ====libvirt debug info======= 2015-05-22 09:23:12.245+0000: 6371: debug : virStorageBackendSCSIFindLUs:476 : Found possible LU '25:0:0:0' 2015-05-22 09:23:12.245+0000: 6371: debug : processLU:401 : Processing LU 25:0:0:0 2015-05-22 09:23:12.245+0000: 6371: debug : getDeviceType:105 : Device type is 0 2015-05-22 09:23:12.245+0000: 6371: debug : processLU:418 : 25:0:0:0 is a Direct-Access LUN 2015-05-22 09:23:12.245+0000: 6371: debug : processLU:421 : Failed to find block device for this LUN ==============================