Bug 1260464 - The spapr vscsi disks for lun id '9-31' and channel id '4-7' could not be recognized inside a power pc guest
Summary: The spapr vscsi disks for lun id '9-31' and channel id '4-7' could not be rec...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.2
Hardware: ppc64le
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: David Gibson
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-09-07 01:49 UTC by Gu Nini
Modified: 2015-12-04 16:56 UTC (History)
11 users (show)

Fixed In Version: qemu-kvm-rhev-2.3.0-23.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-04 16:56:26 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2546 normal SHIPPED_LIVE qemu-kvm-rhev bug fix and enhancement update 2015-12-04 21:11:56 UTC

Description Gu Nini 2015-09-07 01:49:37 UTC
Description of problem:
The spapr vscsi disks for lun id '9-31' and channel id '4-7' could not be recognized inside a power pc guest

Version-Release number of selected component (if applicable):
Host kernel: 3.10.0-306.0.1.el7.ppc64le
Guest kernel: 3.10.0-306.0.1.el7.ppc64le
Qemu-kvm-rhev: qemu-kvm-rhev-2.3.0-22.el7.ppc64le

How reproducible:
100%

Steps to Reproduce:
1. Open a host terminal, change the open files limit to 3072:
# ulimit - n 3072
# ulimit -a

2. Boot up a guest with following qemu cmd in the same host terminal:

/usr/libexec/qemu-kvm -name virtioblk-0828-le -machine pseries,accel=kvm,usb=off -m 256000M -realtime mlock=off -smp 152,sockets=1,cores=20,threads=8 -uuid 95346a10-1828-403a-a610-ac5a52a29414 -no-user-config -nodefaults -monitor stdio -rtc base=utc,clock=host -no-shutdown -boot strict=on -device nec-usb-xhci,id=xhci,addr=0x1 -device spapr-vscsi,id=scsi0,reg=0x1000 -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=/home/virtioblk-0828-le -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,addr=04,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device spapr-vlan,netdev=hostnet0,id=net0,mac=52:54:00:c4:e7:14,reg=0x2000 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,reg=0x30000000 -device usb-kbd,id=input0 -device usb-mouse,id=input1 -device usb-tablet,id=input2 -vnc 0:04 -device VGA,id=video0,bus=pci.0,addr=0x8 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -object rng-random,id=rng0,filename=/dev/random -device virtio-rng-pci,rng=rng0,max-bytes=1234,period=2000,bus=pci.0,addr=0x5 -msg timestamp=on -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x9 -monitor unix:/tmp/monitor2,server,nowait

3. Use following shell script to hot plug the lun id '0-8' and channel id '0-3' spapr vscsi disks and make sure they can be found with hmp cmd 'info block':

#!/bin/bash
j=0
k=0
l=0
for j in {0..63}
do
for k in {0..8}
do
for l in {0..3}
do
qemu-img create -f qcow2 /home/disk-spapr/s$j-l$k-c$l 10M
echo "__com.redhat_drive_add file=/home/disk-spapr/s$j-l$k-c$l,format=qcow2,media=disk,id=s$j-l$k-c$l" | nc -U /tmp/monitor2
echo "device_add scsi-hd,bus=scsi0.0,drive=s$j-l$k-c$l,scsi-id=$j,lun=$k,channel=$l,id=hds$j-l$k-c$l" | nc -U /tmp/monitor2
l=$(($l+1))
done
k=$(($k+1))
done
j=$(($j+1))
done

4. After the hotpluging operation finished in step3, reboot the guest and check the spapr-vscsi disks number inside the guest with following cmd:
# ll /dev/sd* | wc -l

5. Hot unplug above disks with following script:

#!/bin/bash
j=0
k=0
l=0
for j in {0..63}
do
for k in {0..8}
do
for l in {0..3}
do
echo "device_del hds$j-l$k-c$l" | nc -U /tmp/monitor2
l=$(($l+1))
done
k=$(($k+1))
done
j=$(($j+1))
done

6. Reboot guest after the operation in step5 finished

7. Repeat steps 3-6 for the lun id '0-8' and channel id '4-7' spapr vscsi disks

8. Repeat steps 3-6 for the lun id '0-8' and channel id '3-7' spapr vscsi disks

9. Repeat steps 3-6 for the lun id '9-17' and channel id '0-3' spapr vscsi disks

10. Repeat steps 3-6 for the lun id '8-17' and channel id '0-3' spapr vscsi disks

11. Repeat steps 3-6 for the lun id '18-26' and channel id '0-3' spapr vscsi disks

12. Repeat steps 3-6 for the lun id '27-31' and channel id '0-3' spapr vscsi disks


Actual results:
In step4, 2304(64*9*4) disks were found inside the guest
In step7, 0 disk was found inside the guest
In step8, 576(64*8*1) disks were found
In step9, 0 disk was found
In step10, 256(64*1*4) disks were found
In steps 11/12, 0 disk was found
As a conclusion, the lun id '9-31' and channel id '4-7' spapr vscsi disks could not be found inside the guest

Expected results:
The lun id '9-31' and channel id '4-7' spapr vscsi disks could not be found inside the guest as the lun id '0-8' and channel id '0-3' ones

Additional info:

Comment 2 Laurent Vivier 2015-09-08 21:03:05 UTC
This result is normal as the default values for ibmvscsi module are:

# cat /sys/module/ibmvscsi/parameters/max_channel
3

And the maximum number of LUN is hardcoded in the driver:

linux/drivers/scsi/ibmvscsi/ibmvscsi.c
...
2298 
2299         host->max_lun = 8;
2300         host->max_id = max_id;
2301         host->max_channel = max_channel;
2302         host->max_cmd_len = 16;
...

Comment 3 David Gibson 2015-09-09 01:10:57 UTC
The spapr-vscsi paravirtual interface isn't really designed to handle multiple SCSI targets, although it can to a limited extent.  Under PowerVM the interface only ever has a single target per controller.

For the LUNs we should probably reduce the advertised number of LUNs in the qemu side to 8 (currently 31).

Max channels sounds like jut a guest configuration problem, not a bug.

Comment 4 Qunfang Zhang 2015-09-09 02:52:27 UTC
Thanks Laurent and David. 

Hello, David

Based on your comment, do you want to move the bug to POST status? 

Regards,
Qunfang

Comment 5 David Gibson 2015-09-09 03:14:34 UTC
I already did.

Comment 6 David Gibson 2015-09-15 02:31:36 UTC
So we do want to fix this so it's handled more gracefully, but there are some questions about whether it's best address in qemu or the guest.

In any case this is essentially a cosmetic problem (poor error reporting) so I'm deferring to RHEL 7.3.

Comment 7 Miroslav Rezanina 2015-09-15 10:51:00 UTC
Fix included in qemu-kvm-rhev-2.3.0-23.el7

Comment 8 Gu Nini 2015-09-18 11:10:30 UTC
Verify the bug on following sw versions:
Host kernel: 3.10.0-313.el7.ppc64le
Qemu-kvm-rhev: qemu-kvm-rhev-2.3.0-23.el7.ppc64le
Guest kernel: 3.10.0-306.0.1.el7.ppc64le

Now, the spapr-vscsi lun with id 8 or larger could not be initialized with following error prompt:
(qemu) device_add scsi-hd,bus=scsi0.0,drive=s63-l8-c1,scsi-id=63,lun=8,channel=1,id=hds63-l8-c1
2015-09-18T10:45:58.288290Z bad scsi device lun: 8
2015-09-18T10:45:58.288399Z Device 'scsi-hd' could not be initialized

Comment 11 errata-xmlrpc 2015-12-04 16:56:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html


Note You need to log in before you can comment on or make changes to this bug.