Bug 1260464 - The spapr vscsi disks for lun id '9-31' and channel id '4-7' could not be recognized inside a power pc guest
The spapr vscsi disks for lun id '9-31' and channel id '4-7' could not be rec...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev (Show other bugs)
7.2
ppc64le Unspecified
medium Severity medium
: rc
: ---
Assigned To: David Gibson
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-06 21:49 EDT by Gu Nini
Modified: 2015-12-04 11:56 EST (History)
11 users (show)

See Also:
Fixed In Version: qemu-kvm-rhev-2.3.0-23.el7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-04 11:56:26 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Gu Nini 2015-09-06 21:49:37 EDT
Description of problem:
The spapr vscsi disks for lun id '9-31' and channel id '4-7' could not be recognized inside a power pc guest

Version-Release number of selected component (if applicable):
Host kernel: 3.10.0-306.0.1.el7.ppc64le
Guest kernel: 3.10.0-306.0.1.el7.ppc64le
Qemu-kvm-rhev: qemu-kvm-rhev-2.3.0-22.el7.ppc64le

How reproducible:
100%

Steps to Reproduce:
1. Open a host terminal, change the open files limit to 3072:
# ulimit - n 3072
# ulimit -a

2. Boot up a guest with following qemu cmd in the same host terminal:

/usr/libexec/qemu-kvm -name virtioblk-0828-le -machine pseries,accel=kvm,usb=off -m 256000M -realtime mlock=off -smp 152,sockets=1,cores=20,threads=8 -uuid 95346a10-1828-403a-a610-ac5a52a29414 -no-user-config -nodefaults -monitor stdio -rtc base=utc,clock=host -no-shutdown -boot strict=on -device nec-usb-xhci,id=xhci,addr=0x1 -device spapr-vscsi,id=scsi0,reg=0x1000 -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=/home/virtioblk-0828-le -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,addr=04,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device spapr-vlan,netdev=hostnet0,id=net0,mac=52:54:00:c4:e7:14,reg=0x2000 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,reg=0x30000000 -device usb-kbd,id=input0 -device usb-mouse,id=input1 -device usb-tablet,id=input2 -vnc 0:04 -device VGA,id=video0,bus=pci.0,addr=0x8 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -object rng-random,id=rng0,filename=/dev/random -device virtio-rng-pci,rng=rng0,max-bytes=1234,period=2000,bus=pci.0,addr=0x5 -msg timestamp=on -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x9 -monitor unix:/tmp/monitor2,server,nowait

3. Use following shell script to hot plug the lun id '0-8' and channel id '0-3' spapr vscsi disks and make sure they can be found with hmp cmd 'info block':

#!/bin/bash
j=0
k=0
l=0
for j in {0..63}
do
for k in {0..8}
do
for l in {0..3}
do
qemu-img create -f qcow2 /home/disk-spapr/s$j-l$k-c$l 10M
echo "__com.redhat_drive_add file=/home/disk-spapr/s$j-l$k-c$l,format=qcow2,media=disk,id=s$j-l$k-c$l" | nc -U /tmp/monitor2
echo "device_add scsi-hd,bus=scsi0.0,drive=s$j-l$k-c$l,scsi-id=$j,lun=$k,channel=$l,id=hds$j-l$k-c$l" | nc -U /tmp/monitor2
l=$(($l+1))
done
k=$(($k+1))
done
j=$(($j+1))
done

4. After the hotpluging operation finished in step3, reboot the guest and check the spapr-vscsi disks number inside the guest with following cmd:
# ll /dev/sd* | wc -l

5. Hot unplug above disks with following script:

#!/bin/bash
j=0
k=0
l=0
for j in {0..63}
do
for k in {0..8}
do
for l in {0..3}
do
echo "device_del hds$j-l$k-c$l" | nc -U /tmp/monitor2
l=$(($l+1))
done
k=$(($k+1))
done
j=$(($j+1))
done

6. Reboot guest after the operation in step5 finished

7. Repeat steps 3-6 for the lun id '0-8' and channel id '4-7' spapr vscsi disks

8. Repeat steps 3-6 for the lun id '0-8' and channel id '3-7' spapr vscsi disks

9. Repeat steps 3-6 for the lun id '9-17' and channel id '0-3' spapr vscsi disks

10. Repeat steps 3-6 for the lun id '8-17' and channel id '0-3' spapr vscsi disks

11. Repeat steps 3-6 for the lun id '18-26' and channel id '0-3' spapr vscsi disks

12. Repeat steps 3-6 for the lun id '27-31' and channel id '0-3' spapr vscsi disks


Actual results:
In step4, 2304(64*9*4) disks were found inside the guest
In step7, 0 disk was found inside the guest
In step8, 576(64*8*1) disks were found
In step9, 0 disk was found
In step10, 256(64*1*4) disks were found
In steps 11/12, 0 disk was found
As a conclusion, the lun id '9-31' and channel id '4-7' spapr vscsi disks could not be found inside the guest

Expected results:
The lun id '9-31' and channel id '4-7' spapr vscsi disks could not be found inside the guest as the lun id '0-8' and channel id '0-3' ones

Additional info:
Comment 2 Laurent Vivier 2015-09-08 17:03:05 EDT
This result is normal as the default values for ibmvscsi module are:

# cat /sys/module/ibmvscsi/parameters/max_channel
3

And the maximum number of LUN is hardcoded in the driver:

linux/drivers/scsi/ibmvscsi/ibmvscsi.c
...
2298 
2299         host->max_lun = 8;
2300         host->max_id = max_id;
2301         host->max_channel = max_channel;
2302         host->max_cmd_len = 16;
...
Comment 3 David Gibson 2015-09-08 21:10:57 EDT
The spapr-vscsi paravirtual interface isn't really designed to handle multiple SCSI targets, although it can to a limited extent.  Under PowerVM the interface only ever has a single target per controller.

For the LUNs we should probably reduce the advertised number of LUNs in the qemu side to 8 (currently 31).

Max channels sounds like jut a guest configuration problem, not a bug.
Comment 4 Qunfang Zhang 2015-09-08 22:52:27 EDT
Thanks Laurent and David. 

Hello, David

Based on your comment, do you want to move the bug to POST status? 

Regards,
Qunfang
Comment 5 David Gibson 2015-09-08 23:14:34 EDT
I already did.
Comment 6 David Gibson 2015-09-14 22:31:36 EDT
So we do want to fix this so it's handled more gracefully, but there are some questions about whether it's best address in qemu or the guest.

In any case this is essentially a cosmetic problem (poor error reporting) so I'm deferring to RHEL 7.3.
Comment 7 Miroslav Rezanina 2015-09-15 06:51:00 EDT
Fix included in qemu-kvm-rhev-2.3.0-23.el7
Comment 8 Gu Nini 2015-09-18 07:10:30 EDT
Verify the bug on following sw versions:
Host kernel: 3.10.0-313.el7.ppc64le
Qemu-kvm-rhev: qemu-kvm-rhev-2.3.0-23.el7.ppc64le
Guest kernel: 3.10.0-306.0.1.el7.ppc64le

Now, the spapr-vscsi lun with id 8 or larger could not be initialized with following error prompt:
(qemu) device_add scsi-hd,bus=scsi0.0,drive=s63-l8-c1,scsi-id=63,lun=8,channel=1,id=hds63-l8-c1
2015-09-18T10:45:58.288290Z bad scsi device lun: 8
2015-09-18T10:45:58.288399Z Device 'scsi-hd' could not be initialized
Comment 11 errata-xmlrpc 2015-12-04 11:56:26 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html

Note You need to log in before you can comment on or make changes to this bug.