Bug 1431584

AIUI, the LUN field is only 3-bits, so only 0..7 are valid LUN numbers.

qemu could do with a more informative error message here, but this is essentially a cosmetic problem only.

AFAICT this is also not ppc specific, so reassigning to the general virt pool.

256 is a valid lun number for virtio-scsi, so I have no idea what/how error reporting can be improved here. Not able to boot from it seems a separate bug for SLOF, which I have no clue at the moment. Does this scenario work with seabios on x86?

Hi Fam,

It is also exist on x86,but there is a bit of different, guest could boot up with lun id is only 0! while set the lun id to other value ,guest couldn't boot up,then it would try to boot with pxe.

Test version:
host : 3.10.0-690.el7.x86_64
guest : 3.10.0-656.el7.x86_64
qemu : qemu-kvm-rhev-2.9.0-14.el7

What is the seabios version? Please try the fixed version of bug 1020622.

Hi Fam,

Update the seabios version to seabios-1.10.2-3.el7, then it works, guest could boot up with other lun id besides 0. So It might that the SLOF cause this problem on ppc64le.

David, what did you have in mind about what can be done in QEMU?

Ugh, it's been so long that I've forgotten.

I'm not sure where I got the idea that the LUN was only 3 bits.  Looking again at the virtio scsi code, that doesn't appear to be the case.

So it sounds like this probably is a SLOF bug after all.

Thomas, can you confirm if that seems plausible?

David,
you likely mix up spapr-vscsi with virtio-scsi here. For spapr-vscsi, the LUNs are limited to 3 bits only, see BZ 1260464.

Concerning virtio-scsi, as far as I can see, it works fine with SLOF for LUNs < 256, the problem only occurs if the LUN is bigger than 255.

Yongxue Hong, does it work on x86 if you use a LUN bigger than 256 there?

I think the problem is likely in the virtio_scsi_device_find() function in QEMU:

static inline SCSIDevice *virtio_scsi_device_find(VirtIOSCSI *s, uint8_t *lun)
{
    if (lun[0] != 1) {
        return NULL;
    }
    if (lun[2] != 0 && !(lun[2] >= 0x40 && lun[2] < 0x80)) {
        return NULL;
    }
    return scsi_device_find(&s->bus, 0, lun[1], virtio_scsi_get_lun(lun));
}

For LUNs >= 256, the second if-statement triggers and the function returns NULL. Not sure what this check is really good for, though, it looks very strange to me.

Hi Thomas,
It works on x86 while use a LUN  bigger than 256.
eg:
-drive file=/home/hyx/image/rhel-7_4-11853-20G.qcow2,format=qcow2,if=none,cache=none,media=disk,werror=stop,rerror=stop,id=drive-0 \
-device scsi-hd,bus=scsi-pci-1.0,id=scsi-hd-0,drive=drive-0,channel=0,scsi-id=0,lun=257,bootindex=0 \

Thanks.

I think I now understood what's going on here. It's a bug in SLOF. The virtio spec says:

"When used to address a target and logical unit, the only
 supported format for lun is: first byte set to 1, second byte
 set to target, third and fourth byte representing a single
 level LUN structure, followed by four zero bytes."

Now, according to the "SCSI Architecture Model" specification, there are two formats for "single level LUNs" that fit into two bytes (i.e. into the third and fourth byte here): The "Single level LUN structure using peripheral device addressing method" which supports up to 256 LUNs, and the "Single level LUN structure using flat space addressing method" which supports up to 16384 LUNs. SLOF is using the first way, so it is currently limited to 256 LUNs.

I'll try to come up with a patch to fix this issue...

Suggested patch upstream:
https://lists.ozlabs.org/pipermail/slof/2017-July/001685.html

Patch has been accepted upstream:
https://github.com/aik/SLOF/commit/f72a37713fea47da625238f032c245f40b987a28

The step of verification is same as comment 0.

Version of verification:
host: 3.10.0-693.2.1.el7.ppc64le
qemu-kvm: qemu-kvm-rhev-2.9.0-16.el7_4.8.ppc64le
SLOF: SLOF-20170724-2.git89f519f.el7.noarch

Actual results:
The guest could boot up successfully with luns greater than 256.So this bug is fixed and change the status to verified.

This bug exists in the latest version：
Host (RHEL7.5):
Compose: RHEL7.4 released (RHEL-7.4-20170711.0)
kernel-3.10.0-771.el7
qemu-kvm-rhev-2.10.0-4.el7
SLOF-20170724-2.git89f519f.el7.noarch

Guest:
Compose: RHEL7.4 released (RHEL-7.4-20170711.0)
kernel-3.10.0-771.el7

How reproducible:
100%

Steps to Reproduce:
1:Boot a guest with scsi disk which lun=0
eg:/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine pseries-rhel7.5.0 \
    -nodefaults  \
    -vga std \
    -chardev socket,id=serial_id_serial0,path=/tmp/console0,server,nowait \
    -device spapr-vty,reg=0x30000000,chardev=serial_id_serial0 \
    -device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=04 \
    -chardev socket,path=/tmp/serial0,nowait,id=idQdLRHP,server \
    -device virtserialport,id=idBu8FQH,name=vs,bus=virtio_serial_pci0.0,chardev=idQdLRHP \
    -object rng-random,filename=/dev/random,id=passthrough-rOXjKxaC \
    -device virtio-rng-pci,id=virtio-rng-pci-GVn8yzUA,rng=passthrough-rOXjKxaC,bus=pci.0,addr=05 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=06 \
    -device pci-ohci,id=usb3,bus=pci.0,addr=07 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=08 \
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,media=disk,format=qcow2,file=/root/micai/RHEL.7.4.qcow2  \
    -device scsi-hd,drive=drive_image1,id=mage1,bus=virtio_scsi_pci0.0,bootindex=0,channel=0,scsi-id=0,lun=0  \
    -chardev socket,id=chardev0,path=/tmp/slofpath,server,nowait \
    -device spapr-vty,chardev=chardev0 \
    -object iothread,id=thread0 \
    -device virtio-blk-pci,id=disk,drive=drive-disk0,iothread=thread0 \
    -drive file=/root/micai/r3.qcow2,format=qcow2,if=none,id=drive-disk0,cache=none\
    -object iothread,id=iothread0 \
    -drive file=/root/micai/r1.qcow2,format=qcow2,if=none,cache=none,id=drive_plane,werror=stop,rerror=stop \
    -device virtio-blk-pci,drive=d\rive_plane,id=plane,bus=pci.0,addr=09,iothread=iothread0 \
    -drive file=/root/micai/r2.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop \
    -device spapr-vscsi,reg=0x1000,id=scsi0 \
    -device scsi-hd,drive=drive-system-disk,id=system-disk,bus=scsi0.0 \
    -device virtio-net-pci,mac=9a:4f:50:51:52:53,id=id9HRc5V,vectors=4,netdev=idjlQN53,bus=pci.0,addr=11 \
    -netdev tap,id=idjlQN53,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
    -device spapr-vlan,mac=9a:4f:50:51:52:54,netdev=hostnet0,id=net0 \
    -netdev tap,id=hostnet0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
    -m 8G,slots=4,maxmem=32G \
    -smp 8,cores=4,threads=1,sockets=2 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -device usb-mouse,id=input1,bus=usb1.0,port=2 \
    -device usb-kbd,id=input2,bus=usb1.0,port=3 \
    -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/root/micai/RHEL-7.4-20170711.0-Server-ppc64-dvd1.iso \
    -device scsi-cd,id=cd1,drive=drive_cd1,bus=virtio_scsi_pci0.0,bootindex=1 \
    -vnc :1  \
    -qmp tcp:0:8881,server,nowait \
    -monitor stdio \
    -rtc base=utc,clock=host  \
    -boot order=cdn,once=c,menu=on,strict=on  \
    -enable-kvm  \
    -watchdog i6300esb \
    -watchdog-action reset \
    -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=15
2.shutdown guest and change the  lun id of scsi disk to 256 and boot up guest
eg:
/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine pseries-rhel7.5.0 \
    -nodefaults  \
    -vga std \
    -chardev socket,id=serial_id_serial0,path=/tmp/console0,server,nowait \
    -device spapr-vty,reg=0x30000000,chardev=serial_id_serial0 \
    -device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=04 \
    -chardev socket,path=/tmp/serial0,nowait,id=idQdLRHP,server \
    -device virtserialport,id=idBu8FQH,name=vs,bus=virtio_serial_pci0.0,chardev=idQdLRHP \
    -object rng-random,filename=/dev/random,id=passthrough-rOXjKxaC \
    -device virtio-rng-pci,id=virtio-rng-pci-GVn8yzUA,rng=passthrough-rOXjKxaC,bus=pci.0,addr=05 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=06 \
    -device pci-ohci,id=usb3,bus=pci.0,addr=07 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=08 \
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,media=disk,format=qcow2,file=/root/micai/RHEL.7.4.qcow2  \
    -device scsi-hd,drive=drive_image1,id=mage1,bus=virtio_scsi_pci0.0,bootindex=0,channel=0,scsi-id=0,lun=256 \
    -chardev socket,id=chardev0,path=/tmp/slofpath,server,nowait \
    -device spapr-vty,chardev=chardev0 \
    -object iothread,id=thread0 \
    -device virtio-blk-pci,id=disk,drive=drive-disk0,iothread=thread0 \
    -drive file=/root/micai/r3.qcow2,format=qcow2,if=none,id=drive-disk0,cache=none\
    -object iothread,id=iothread0 \
    -drive file=/root/micai/r1.qcow2,format=qcow2,if=none,cache=none,id=drive_plane,werror=stop,rerror=stop \
    -device virtio-blk-pci,drive=d\rive_plane,id=plane,bus=pci.0,addr=09,iothread=iothread0 \
    -drive file=/root/micai/r2.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop \
    -device spapr-vscsi,reg=0x1000,id=scsi0 \
    -device scsi-hd,drive=drive-system-disk,id=system-disk,bus=scsi0.0 \
    -device virtio-net-pci,mac=9a:4f:50:51:52:53,id=id9HRc5V,vectors=4,netdev=idjlQN53,bus=pci.0,addr=11 \
    -netdev tap,id=idjlQN53,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
    -device spapr-vlan,mac=9a:4f:50:51:52:54,netdev=hostnet0,id=net0 \
    -netdev tap,id=hostnet0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
    -m 8G,slots=4,maxmem=32G \
    -smp 8,cores=4,threads=1,sockets=2 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -device usb-mouse,id=input1,bus=usb1.0,port=2 \
    -device usb-kbd,id=input2,bus=usb1.0,port=3 \
    -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/root/micai/RHEL-7.4-20170711.0-Server-ppc64-dvd1.iso \
    -device scsi-cd,id=cd1,drive=drive_cd1,bus=virtio_scsi_pci0.0,bootindex=1 \
    -vnc :1  \
    -qmp tcp:0:8881,server,nowait \
    -monitor stdio \
    -rtc base=utc,clock=host  \
    -boot order=cdn,once=c,menu=on,strict=on  \
    -enable-kvm  \
    -watchdog i6300esb \
    -watchdog-action reset \
    -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=15

Actual results:
The guest can't boot up and enter the installation.
The console prompt is same as comment 0

Strange, it seems to work for me... The only oddity that I saw was that SLOF rather preferred to boot from the CD-ROM drive instead of the hard disk, but that's likely another problem. Could you please try again and omit the CD-ROM drive in your test? Also, please do not use "-boot order=cdn,once=c" when using "bootindex" for other devices, since that likely gives unexpected results. If it still fails for you without CD-ROM, could you please paste the serial console output here again? Thanks!

FWIW, here's a shorter command line to reproduce the problem with the boot ordering (SLOF boots from CD-ROM, though it should boot from HD instead):

sudo /usr/libexec/qemu-kvm -nodefaults -serial mon:stdio -nographic -enable-kvm -device virtio-scsi-pci,id=vsp0,bus=pci.0 -drive id=hdimg1,if=none,format=qcow2,file=/var/lib/libvirt/images/disk.img -device scsi-hd,drive=hdimg1,id=hd1,bus=vsp0.0,bootindex=0,channel=0,scsi-id=0,lun=256 -drive id=cdimg1,if=none,media=cdrom,file=/var/lib/libvirt/images/RHEL-7.iso -device scsi-cd,id=cd1,drive=cdimg1,bus=vsp0.0,bootindex=1

i just used your command line to reproduce the problem.When configuring the cd-rom bootindex= 1, I also saw the problem of SLOF boots from CD-ROM.Then I removed the configuration of the cd-rom with the parameter of bootindex= 1, and the lun parameter remained 256. The guest boots from HD.This indicates that lun=256 does not affect on it。This indicates that the Bug 1431584 has been fixed. Do I need to open a CD and hd boot order bug? I will discuss this with xianwang tomorrow.

(In reply to Minjia Cai from comment #19)
> This indicates that the Bug 1431584 has been fixed.

Yes, I think it should be fixed, so please mark this as VERIFIED if you agree (I'm moving the bug back to ON_QA for this).

> Do I need to open a CD and hd boot order bug?

Yes, I think we should track this with a separate BZ. I've now already created a new BZ here: https://bugzilla.redhat.com/show_bug.cgi?id=1515393

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0820