Created attachment 1523471 [details] Display one multipath disk:mpatha Description of problem: When using cdrom to install the system, there are two blockdev disks.choose installation destination,the system automatic multipath configuration disk. Display one multipath disk:mpatha But when you use the -drive command,can select these two disk normally. Version-Release number of selected component (if applicable): kernel version:4.18.0-62.el8.x86_64 qemu-kvm version:qemu-kvm-3.1.0-6.module+el8+2711+98525d2b How reproducible: 5/5 Steps to Reproduce: /usr/libexec/qemu-kvm \ -name zhenyzha-RHEL-7.6 \ -m 4G \ -nodefaults \ -vga qxl \ -smp 4,cores=4,threads=1,sockets=1 \ -vnc :30 \ -monitor stdio \ -rtc base=utc,clock=host \ -boot order=cd,menu=off,strict=off \ -enable-kvm \ -qmp unix:/var/tmp/qmp-monitor-zhenyzha,server,nowait \ -qmp tcp:0:3001,server,nowait \ -device virtio-scsi-pci,bus=pci.0,addr=0x05,id=scsi-pci-1 \ -blockdev driver=raw,file.driver=file,node-name=drive_cd1,cache.no-flush=on,cache.direct=off,file.filename=/mnt/os2/linux/RHEL7.6-Server-x86_64.iso,read-only=on \ -device scsi-cd,id=cd1,drive=drive_cd1,bootindex=0 \ -blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=guest.qcow2,node-name=my_file \ -blockdev driver=qcow2,node-name=my,file=my_file \ -device scsi-hd,id=target0,drive=my \ -blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/test/zhenyzha-nfs/disk/disk_2,node-name=disk_2 \ -blockdev driver=qcow2,node-name=my0,file=disk_2 \ -device scsi-hd,id=target1,drive=my0 \ Actual results: Display one multipath disk:mpatha Expected results: Can choose two disks normally Additional info:
Hit this issue when install rhel7.6 guest on rhel8 host.
This has the same root cause as bug 1668248, and we want to address this in both QEMU and libvirt, so I will move this one to libvirt and keep the other one for QEMU. If no serial number is specified, QEMU used to use the BlockBackend name (the -drive id) for the Device Identification Page in scsi-disk. With -blockdev, the BlockBackend becomes anonymous, so we pass an empty string to the guest for both disks, and the guest decides that this is a single multipath disk. As danpb noted, using the BlockBackend name also wasn't a good idea because it means we broke the guest ABI when libvirt changed the way it named the drives. QEMU doesn't have a good way to provide a unique ID that is guaranteed to be stable across QEMU runs (and ideally doesn't abuse backend IDs), so it would be good if libvirt could just always pass a serial number to avoid the whole problem.
Will the 'serial' be passed in the same way as it did with drive? Or is a different argument required for it? We can make sure that we pass in the old string for the cases when -blockdev will be used so that we don't break the ABI again.
Hmm, I'm not sure libvirt can unconditionally pass a serial without causing a different regression If we don't pass 'serial' then we have - code page 0x80 - no data reported - code page 0x83 - report data based on blockbackend name The problem we face is that with -blockdev we caused a guest ABI regression for code page 0x83. If we do pass 'serial' then we get - code page 0x80 - serial reported - code page 0x83 - serial reported so we'll fix the QEMU regression for code page 0x83, but cause a new guest ABI regression for code page 0x80 vs previous libvirt
So it looks like we need a new property for passing in the old name for disks to qemu to use if serial is not provided to keep the old behaviour. Good thing is that -blockdev is not enabled yet so we can base the capability on the presence of the new property to avoid regression.
Fixed upstream: commit 6d3c96f0d965fe182040f6442f0cbeb46d10efda Author: Peter Krempa <pkrempa> Date: Mon Jan 28 17:12:01 2019 +0100 qemu: caps: Add lockout for -blockdev if QEMU_CAPS_SCSI_DISK_DEVICE_ID is not present Avoid regressions by disallowing the BLOCKDEV capability. Signed-off-by: Peter Krempa <pkrempa> commit a1dce96236f6d35167924fa7e6a70f58f394b23c Author: Peter Krempa <pkrempa> Date: Mon Jan 28 16:57:35 2019 +0100 qemu: Use the 'device_id' property of SCSI disks to avoid regressing QEMU accidentally exposed the id of -drive (or same value as disk serial, if provided) in one of the identifiers visible from the guest. To avoid regression in case when -blockdev will be used we need to always specify it ourselves.
Basic test method in libvirt after -blockdev is enabled: 1. Start two scsi disks without serial number specified 2. In vm, check the serial number via: # udevadm info --query=all --name=/dev/sda | grep ID_SERIAL Make sure the serial numbers from disks are different.
Hi Peter, Since this bug depends on the enabling of -blockdev, when could the -blockdev be ready for upstream?
I'm hoping I finish all the block-job related changes necessary to enable blockdev in this release (5.6). For testing purposes (if you don't want to do any block jobs/snapshots) you can use the qemu namespace support for capabilities to enable blockdev temporarily: <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> [...] <qemu:capabilities> <qemu:add capability="blockdev"/> </qemu:capabilities> </domain> This is already present upstream since 5.5 release.
Test on libvirt-5.6 uptream release, qemu-kvm-4.0.0-6.module+el8.1.0+3736+a2aefea3.x86_64 1. Start vm with these scsi disks, with -blockdev disabled - virtio-scsi disk with customized serial - virtio-scsi disk without serial - scsi cdrom with customized serial - scsi cdrom without serial Get serials of all scsi disks by `lsblk -o name,serial` in guest. 2. Start vm with these scsi disks, with -blockdev enabled as comment15 - virtio-scsi disk with customized serial - virtio-scsi disk without serial - scsi cdrom with customized serial - scsi cdrom without serial Get serials of all scsi disks by `lsblk -o name,serial` in guest. Expectations: 1. All scsi disks has different serial in step1 or step2. 2. The results of `lsblk -o name,serial` are the same in step1 and step2 Detailed steps: The disk XMLs: <disk type="file" device="disk"> <driver name="qemu" type="qcow2"/> <source file="/var/lib/libvirt/images/copy.s1"/> <backingStore type="file" index="1"> <format type="qcow2"/> <source file="/var/lib/libvirt/images/copy.qcow2"/> <backingStore/> </backingStore> <target dev="sda" bus="scsi"/> <alias name="scsi0-0-0-0"/> <address type="drive" controller="0" bus="0" target="0" unit="0"/> </disk><disk type="file" device="disk"> <driver name="qemu" type="qcow2"/> <source file="/tmp/scsi.qcow2"/> <backingStore/> <target dev="sdb" bus="scsi"/> <serial>second-scsi</serial> <alias name="scsi0-0-0-1"/> <address type="drive" controller="0" bus="0" target="0" unit="1"/> </disk><disk type="file" device="cdrom"> <driver name="qemu"/> <target dev="sdc" bus="scsi"/> <readonly/> <alias name="scsi0-0-0-2"/> <address type="drive" controller="0" bus="0" target="0" unit="2"/> </disk><disk type="file" device="cdrom"> <driver name="qemu"/> <target dev="sdd" bus="scsi"/> <readonly/> <serial>cdrom-xxx</serial> <alias name="scsi0-0-0-3"/> <address type="drive" controller="0" bus="0" target="0" unit="3"/> </disk> `lsblk -o name,serial` in step1: # lsblk -o name,serial NAME SERIAL sda drive-scsi0-0-0-0 ├─sda1 └─sda2 ├─rhel-root └─rhel-swap sdb second-scsi sr0 cdrom-xxx sr1 drive-scsi0-0-0-2 `lsblk -o name,serial` in step2: # lsblk -o name,serial NAME SERIAL sda drive-scsi0-0-0-0 ├─sda1 └─sda2 ├─rhel-root └─rhel-swap sdb second-scsi sr0 cdrom-xxx sr1 drive-scsi0-0-0-2 Worked as expected.
Verified on libvirt-5.6.0-5.module+el8.1.0+4229+2e4e348c.x86_64 qemu-kvm-4.1.0-10.module+el8.1.0+4234+33aa4f57.x86_64: A. Enable -blockdev 1. Start a VM with scsi disk without custom serial or custom alias, scsi disk with custom alias only, scsi disk with custom alias. VM xml: ``` <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/copy.qcow2'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/tmp/scsi.qcow2'/> <backingStore/> <target dev='sdb' bus='scsi'/> <serial>second-scsi</serial> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='sdc' bus='scsi'/> <readonly/> <alias name='ua-cdrom'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> ... <qemu:capabilities> <qemu:add capability='blockdev'/> </qemu:capabilities> </domain> ``` 2. Start VM, check the qemu cmdline # virsh start copy Domain copy started Qemu cmdline: /usr/libexec/qemu-kvm -name guest=copy,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-copy/master-key.aes -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off -m 1024 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1 -object iothread,id=iothread2 -object iothread,id=iothread3 -uuid c3f1cdcb-cc26-462d-9e4a-fdb6f40402ca -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=36,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device qemu-xhci,id=usb1,bus=pci.0,addr=0xe -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0xc -device ahci,id=sata0,bus=pci.0,addr=0x9 -device ahci,id=sata1,bus=pci.0,addr=0xa -device ahci,id=sata2,bus=pci.0,addr=0xb -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -device usb-hub,id=hub0,bus=usb.0,port=3 -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/copy.qcow2","node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-3-format","read-only":false,"driver":"qcow2","file":"libvirt-3-storage","backing":null} -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-3-format,id=scsi0-0-0-0,bootindex=1 -blockdev {"driver":"file","filename":"/tmp/scsi.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2","file":"libvirt-2-storage","backing":null} -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=second-scsi,drive=libvirt-2-format,id=scsi0-0-0-1,serial=second-scsi -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,device_id=drive-ua-cdrom,id=ua-cdrom -netdev tap,fd=38,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:d1:0d:97,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=39,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -spice port=5900,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on 3. Login to vm, check the disks serial (vm) # lsblk -o name,kname,serial NAME KNAME SERIAL sda sda drive-scsi0-0-0-0 ├─sda1 sda1 └─sda2 sda2 ├─rhel-root dm-0 └─rhel-swap dm-1 sdb sdb second-scsi sr0 sr0 drive-ua-cdrom B. Disable -blockdev 1. Start a VM with scsi disk without custom serial or custom alias, scsi disk with custom alias only, scsi disk with custom alias. VM xml: ``` <domain type='kvm'> ... <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/copy.qcow2'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/tmp/scsi.qcow2'/> <backingStore/> <target dev='sdb' bus='scsi'/> <serial>second-scsi</serial> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='sdc' bus='scsi'/> <readonly/> <alias name='ua-cdrom'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> ... </domain> ``` 2. Start VM and check the qemu cmdline: /usr/libexec/qemu-kvm -name guest=copy,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-copy/master-key.aes -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off -m 1024 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1 -object iothread,id=iothread2 -object iothread,id=iothread3 -uuid c3f1cdcb-cc26-462d-9e4a-fdb6f40402ca -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=36,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device qemu-xhci,id=usb1,bus=pci.0,addr=0xe -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0xc -device ahci,id=sata0,bus=pci.0,addr=0x9 -device ahci,id=sata1,bus=pci.0,addr=0xa -device ahci,id=sata2,bus=pci.0,addr=0xb -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -device usb-hub,id=hub0,bus=usb.0,port=3 -drive file=/var/lib/libvirt/images/copy.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/tmp/scsi.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-1 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=second-scsi,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1,serial=second-scsi -drive if=none,id=drive-ua-cdrom,readonly=on -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,device_id=drive-ua-cdrom,drive=drive-ua-cdrom,id=ua-cdrom -netdev tap,fd=38,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:d1:0d:97,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=39,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -spice port=5900,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on 3. Login to VM, and check the serial (vm) # lsblk -o name,kname,serial NAME KNAME SERIAL sda sda drive-scsi0-0-0-0 ├─sda1 sda1 └─sda2 sda2 ├─rhel-root dm-0 └─rhel-swap dm-1 sdb sdb second-scsi sr0 sr0 drive-ua-cdrom Compared with results in A3, there is no change in disk serial.
It is better to chec the difference of `lsblk -O -J` before and after. No changes.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3723