Hide Forgot
Description of problem: The spapr vscsi disks for lun id '9-31' and channel id '4-7' could not be recognized inside a power pc guest Version-Release number of selected component (if applicable): Host kernel: 3.10.0-306.0.1.el7.ppc64le Guest kernel: 3.10.0-306.0.1.el7.ppc64le Qemu-kvm-rhev: qemu-kvm-rhev-2.3.0-22.el7.ppc64le How reproducible: 100% Steps to Reproduce: 1. Open a host terminal, change the open files limit to 3072: # ulimit - n 3072 # ulimit -a 2. Boot up a guest with following qemu cmd in the same host terminal: /usr/libexec/qemu-kvm -name virtioblk-0828-le -machine pseries,accel=kvm,usb=off -m 256000M -realtime mlock=off -smp 152,sockets=1,cores=20,threads=8 -uuid 95346a10-1828-403a-a610-ac5a52a29414 -no-user-config -nodefaults -monitor stdio -rtc base=utc,clock=host -no-shutdown -boot strict=on -device nec-usb-xhci,id=xhci,addr=0x1 -device spapr-vscsi,id=scsi0,reg=0x1000 -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=/home/virtioblk-0828-le -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,addr=04,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device spapr-vlan,netdev=hostnet0,id=net0,mac=52:54:00:c4:e7:14,reg=0x2000 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,reg=0x30000000 -device usb-kbd,id=input0 -device usb-mouse,id=input1 -device usb-tablet,id=input2 -vnc 0:04 -device VGA,id=video0,bus=pci.0,addr=0x8 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -object rng-random,id=rng0,filename=/dev/random -device virtio-rng-pci,rng=rng0,max-bytes=1234,period=2000,bus=pci.0,addr=0x5 -msg timestamp=on -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x9 -monitor unix:/tmp/monitor2,server,nowait 3. Use following shell script to hot plug the lun id '0-8' and channel id '0-3' spapr vscsi disks and make sure they can be found with hmp cmd 'info block': #!/bin/bash j=0 k=0 l=0 for j in {0..63} do for k in {0..8} do for l in {0..3} do qemu-img create -f qcow2 /home/disk-spapr/s$j-l$k-c$l 10M echo "__com.redhat_drive_add file=/home/disk-spapr/s$j-l$k-c$l,format=qcow2,media=disk,id=s$j-l$k-c$l" | nc -U /tmp/monitor2 echo "device_add scsi-hd,bus=scsi0.0,drive=s$j-l$k-c$l,scsi-id=$j,lun=$k,channel=$l,id=hds$j-l$k-c$l" | nc -U /tmp/monitor2 l=$(($l+1)) done k=$(($k+1)) done j=$(($j+1)) done 4. After the hotpluging operation finished in step3, reboot the guest and check the spapr-vscsi disks number inside the guest with following cmd: # ll /dev/sd* | wc -l 5. Hot unplug above disks with following script: #!/bin/bash j=0 k=0 l=0 for j in {0..63} do for k in {0..8} do for l in {0..3} do echo "device_del hds$j-l$k-c$l" | nc -U /tmp/monitor2 l=$(($l+1)) done k=$(($k+1)) done j=$(($j+1)) done 6. Reboot guest after the operation in step5 finished 7. Repeat steps 3-6 for the lun id '0-8' and channel id '4-7' spapr vscsi disks 8. Repeat steps 3-6 for the lun id '0-8' and channel id '3-7' spapr vscsi disks 9. Repeat steps 3-6 for the lun id '9-17' and channel id '0-3' spapr vscsi disks 10. Repeat steps 3-6 for the lun id '8-17' and channel id '0-3' spapr vscsi disks 11. Repeat steps 3-6 for the lun id '18-26' and channel id '0-3' spapr vscsi disks 12. Repeat steps 3-6 for the lun id '27-31' and channel id '0-3' spapr vscsi disks Actual results: In step4, 2304(64*9*4) disks were found inside the guest In step7, 0 disk was found inside the guest In step8, 576(64*8*1) disks were found In step9, 0 disk was found In step10, 256(64*1*4) disks were found In steps 11/12, 0 disk was found As a conclusion, the lun id '9-31' and channel id '4-7' spapr vscsi disks could not be found inside the guest Expected results: The lun id '9-31' and channel id '4-7' spapr vscsi disks could not be found inside the guest as the lun id '0-8' and channel id '0-3' ones Additional info:
This result is normal as the default values for ibmvscsi module are: # cat /sys/module/ibmvscsi/parameters/max_channel 3 And the maximum number of LUN is hardcoded in the driver: linux/drivers/scsi/ibmvscsi/ibmvscsi.c ... 2298 2299 host->max_lun = 8; 2300 host->max_id = max_id; 2301 host->max_channel = max_channel; 2302 host->max_cmd_len = 16; ...
The spapr-vscsi paravirtual interface isn't really designed to handle multiple SCSI targets, although it can to a limited extent. Under PowerVM the interface only ever has a single target per controller. For the LUNs we should probably reduce the advertised number of LUNs in the qemu side to 8 (currently 31). Max channels sounds like jut a guest configuration problem, not a bug.
Thanks Laurent and David. Hello, David Based on your comment, do you want to move the bug to POST status? Regards, Qunfang
I already did.
So we do want to fix this so it's handled more gracefully, but there are some questions about whether it's best address in qemu or the guest. In any case this is essentially a cosmetic problem (poor error reporting) so I'm deferring to RHEL 7.3.
Fix included in qemu-kvm-rhev-2.3.0-23.el7
Verify the bug on following sw versions: Host kernel: 3.10.0-313.el7.ppc64le Qemu-kvm-rhev: qemu-kvm-rhev-2.3.0-23.el7.ppc64le Guest kernel: 3.10.0-306.0.1.el7.ppc64le Now, the spapr-vscsi lun with id 8 or larger could not be initialized with following error prompt: (qemu) device_add scsi-hd,bus=scsi0.0,drive=s63-l8-c1,scsi-id=63,lun=8,channel=1,id=hds63-l8-c1 2015-09-18T10:45:58.288290Z bad scsi device lun: 8 2015-09-18T10:45:58.288399Z Device 'scsi-hd' could not be initialized
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2546.html