Bug 1669446 - Avoid duplicate default serial numbers configuration when using -blockdev for scsi
Summary: Avoid duplicate default serial numbers configuration when using -blockdev for...
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.0
Hardware: All
OS: Linux
urgent
high
Target Milestone: rc
: 8.0
Assignee: Peter Krempa
QA Contact: Han Han
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-25 10:02 UTC by zhenyzha
Modified: 2019-09-18 07:11 UTC (History)
14 users (show)

Fixed In Version: libvirt-5.3.0-1.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)
Display one multipath disk:mpatha (429.29 KB, image/png)
2019-01-25 10:02 UTC, zhenyzha
no flags Details

Description zhenyzha 2019-01-25 10:02:00 UTC
Created attachment 1523471 [details]
Display one multipath disk:mpatha

Description of problem:
When using cdrom to install the system, there are two blockdev disks.choose installation destination,the system automatic multipath configuration disk.
Display one multipath disk:mpatha
But when you use the -drive command,can select these two disk normally.

Version-Release number of selected component (if applicable):
  kernel version:4.18.0-62.el8.x86_64
  qemu-kvm version:qemu-kvm-3.1.0-6.module+el8+2711+98525d2b

How reproducible:
5/5

Steps to Reproduce:
/usr/libexec/qemu-kvm \
-name zhenyzha-RHEL-7.6 \
-m 4G \
-nodefaults \
-vga qxl \
-smp 4,cores=4,threads=1,sockets=1 \
-vnc :30 \
-monitor stdio \
-rtc base=utc,clock=host \
-boot order=cd,menu=off,strict=off \
-enable-kvm \
-qmp unix:/var/tmp/qmp-monitor-zhenyzha,server,nowait \
-qmp tcp:0:3001,server,nowait \
-device virtio-scsi-pci,bus=pci.0,addr=0x05,id=scsi-pci-1 \
-blockdev driver=raw,file.driver=file,node-name=drive_cd1,cache.no-flush=on,cache.direct=off,file.filename=/mnt/os2/linux/RHEL7.6-Server-x86_64.iso,read-only=on \
-device scsi-cd,id=cd1,drive=drive_cd1,bootindex=0 \
-blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=guest.qcow2,node-name=my_file \
-blockdev driver=qcow2,node-name=my,file=my_file \
-device scsi-hd,id=target0,drive=my \
-blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/test/zhenyzha-nfs/disk/disk_2,node-name=disk_2 \
-blockdev driver=qcow2,node-name=my0,file=disk_2 \
-device scsi-hd,id=target1,drive=my0 \
 

Actual results:
Display one multipath disk:mpatha

Expected results:
Can choose two disks normally

Additional info:

Comment 1 Xueqiang Wei 2019-01-25 11:11:06 UTC
Hit this issue when install rhel7.6 guest on rhel8 host.

Comment 2 Kevin Wolf 2019-01-25 12:12:18 UTC
This has the same root cause as bug 1668248, and we want to address this in both QEMU and libvirt, so I will move this one to libvirt and keep the other one for QEMU.

If no serial number is specified, QEMU used to use the BlockBackend name (the -drive id) for the Device Identification Page in scsi-disk. With -blockdev, the BlockBackend becomes anonymous, so we pass an empty string to the guest for both disks, and the guest decides that this is a single multipath disk. As danpb noted, using the BlockBackend name also wasn't a good idea because it means we broke the guest ABI when libvirt changed the way it named the drives.

QEMU doesn't have a good way to provide a unique ID that is guaranteed to be stable across QEMU runs (and ideally doesn't abuse backend IDs), so it would be good if libvirt could just always pass a serial number to avoid the whole problem.

Comment 3 Peter Krempa 2019-01-25 12:49:07 UTC
Will the 'serial' be passed in the same way as it did with drive? Or is a different argument required for it? We can make sure that we pass in the old string for the cases when -blockdev will be used so that we don't break the ABI again.

Comment 4 Daniel Berrangé 2019-01-25 13:08:28 UTC
Hmm, I'm not sure libvirt can unconditionally pass a serial without causing a different regression

If we don't pass 'serial' then we have

 - code page 0x80  - no data reported
 - code page 0x83  - report data based on blockbackend name

The problem we face is that with -blockdev we caused a guest ABI regression for code page 0x83.

If we do pass 'serial' then we get

 - code page 0x80  - serial reported
 - code page 0x83  - serial reported

so we'll fix the QEMU regression for code page 0x83, but cause a new guest ABI regression for code page 0x80 vs previous libvirt

Comment 5 Peter Krempa 2019-01-25 13:36:34 UTC
So it looks like we need a new property for passing in the old name for disks to qemu to use if serial is not provided to keep the old behaviour. Good thing is that -blockdev is not enabled yet so we can base the capability on the presence of the new property to avoid regression.

Comment 9 Peter Krempa 2019-02-08 11:15:16 UTC
Fixed upstream:

commit 6d3c96f0d965fe182040f6442f0cbeb46d10efda
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Mon Jan 28 17:12:01 2019 +0100

    qemu: caps: Add lockout for -blockdev if QEMU_CAPS_SCSI_DISK_DEVICE_ID is not present
    
    Avoid regressions by disallowing the BLOCKDEV capability.
    
    Signed-off-by: Peter Krempa <pkrempa@redhat.com>

commit a1dce96236f6d35167924fa7e6a70f58f394b23c
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Mon Jan 28 16:57:35 2019 +0100

    qemu: Use the 'device_id' property of SCSI disks to avoid regressing
    
    QEMU accidentally exposed the id of -drive (or same value as disk
    serial, if provided) in one of the identifiers visible from the guest.
    
    To avoid regression in case when -blockdev will be used we need to
    always specify it ourselves.

Comment 10 Han Han 2019-02-11 07:26:21 UTC
Basic test method in libvirt after -blockdev is enabled:
1. Start two scsi disks without serial number specified
2. In vm, check the serial number via:
# udevadm info --query=all --name=/dev/sda | grep ID_SERIAL

Make sure the serial numbers from disks are different.

Comment 14 Han Han 2019-07-17 03:09:55 UTC
Hi Peter,
Since this bug depends on the enabling of -blockdev, when could the -blockdev be ready for upstream?

Comment 15 Peter Krempa 2019-07-17 07:28:44 UTC
I'm hoping I finish all the block-job related changes necessary to enable blockdev in this release (5.6).

For testing purposes (if you don't want to do any block jobs/snapshots) you can use the qemu namespace support for capabilities to enable blockdev temporarily:

<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

[...]

  <qemu:capabilities>
    <qemu:add capability="blockdev"/>
  </qemu:capabilities>
</domain>

This is already present upstream since 5.5 release.

Comment 16 Han Han 2019-08-06 05:57:00 UTC
Test on libvirt-5.6 uptream release, qemu-kvm-4.0.0-6.module+el8.1.0+3736+a2aefea3.x86_64
1. Start vm with these scsi disks, with -blockdev disabled
- virtio-scsi disk with customized serial
- virtio-scsi disk without serial
- scsi cdrom with customized serial
- scsi cdrom without serial

Get serials of all scsi disks by `lsblk -o name,serial` in guest.

2. Start vm with these scsi disks, with -blockdev enabled as comment15
- virtio-scsi disk with customized serial
- virtio-scsi disk without serial
- scsi cdrom with customized serial
- scsi cdrom without serial

Get serials of all scsi disks by `lsblk -o name,serial` in guest.

Expectations:
1. All scsi disks has different serial in step1 or step2.
2. The results of `lsblk -o name,serial` are the same in step1 and step2


Detailed steps:
The disk XMLs:
<disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/var/lib/libvirt/images/copy.s1"/>
      <backingStore type="file" index="1">
        <format type="qcow2"/>
        <source file="/var/lib/libvirt/images/copy.qcow2"/>
        <backingStore/>
      </backingStore>
      <target dev="sda" bus="scsi"/>
      <alias name="scsi0-0-0-0"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk><disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/tmp/scsi.qcow2"/>
      <backingStore/>
      <target dev="sdb" bus="scsi"/>
      <serial>second-scsi</serial>
      <alias name="scsi0-0-0-1"/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk><disk type="file" device="cdrom">
      <driver name="qemu"/>
      <target dev="sdc" bus="scsi"/>
      <readonly/>
      <alias name="scsi0-0-0-2"/>
      <address type="drive" controller="0" bus="0" target="0" unit="2"/>
    </disk><disk type="file" device="cdrom">
      <driver name="qemu"/>
      <target dev="sdd" bus="scsi"/>
      <readonly/>
      <serial>cdrom-xxx</serial>
      <alias name="scsi0-0-0-3"/>
      <address type="drive" controller="0" bus="0" target="0" unit="3"/>
    </disk>

`lsblk -o name,serial` in step1:
# lsblk -o name,serial                                                                                                                                                                          
NAME          SERIAL
sda           drive-scsi0-0-0-0
├─sda1
└─sda2
  ├─rhel-root
  └─rhel-swap
sdb           second-scsi
sr0           cdrom-xxx
sr1           drive-scsi0-0-0-2


`lsblk -o name,serial` in step2:
# lsblk -o name,serial
NAME          SERIAL
sda           drive-scsi0-0-0-0
├─sda1        
└─sda2        
  ├─rhel-root 
  └─rhel-swap 
sdb           second-scsi
sr0           cdrom-xxx
sr1           drive-scsi0-0-0-2


Worked as expected.

Comment 17 Han Han 2019-09-18 07:02:27 UTC
Verified on libvirt-5.6.0-5.module+el8.1.0+4229+2e4e348c.x86_64 qemu-kvm-4.1.0-10.module+el8.1.0+4234+33aa4f57.x86_64:
A. Enable -blockdev
1. Start a VM with scsi disk without custom serial or custom alias, scsi disk with custom alias only, scsi disk with custom alias. 
VM xml:
```
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/copy.qcow2'/>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/tmp/scsi.qcow2'/>
      <backingStore/>
      <target dev='sdb' bus='scsi'/>
      <serial>second-scsi</serial>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='sdc' bus='scsi'/>
      <readonly/>
      <alias name='ua-cdrom'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
...
  <qemu:capabilities>
    <qemu:add capability='blockdev'/>
  </qemu:capabilities>
</domain>
```
2. Start VM, check the qemu cmdline
# virsh start copy                             
Domain copy started

Qemu cmdline:
/usr/libexec/qemu-kvm -name guest=copy,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-copy/master-key.aes -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off -m 1024 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1 -object iothread,id=iothread2 -object iothread,id=iothread3 -uuid c3f1cdcb-cc26-462d-9e4a-fdb6f40402ca -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=36,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device qemu-xhci,id=usb1,bus=pci.0,addr=0xe -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0xc -device ahci,id=sata0,bus=pci.0,addr=0x9 -device ahci,id=sata1,bus=pci.0,addr=0xa -device ahci,id=sata2,bus=pci.0,addr=0xb -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -device usb-hub,id=hub0,bus=usb.0,port=3 -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/copy.qcow2","node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-3-format","read-only":false,"driver":"qcow2","file":"libvirt-3-storage","backing":null} -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-3-format,id=scsi0-0-0-0,bootindex=1 -blockdev {"driver":"file","filename":"/tmp/scsi.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2","file":"libvirt-2-storage","backing":null} -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=second-scsi,drive=libvirt-2-format,id=scsi0-0-0-1,serial=second-scsi -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,device_id=drive-ua-cdrom,id=ua-cdrom -netdev tap,fd=38,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:d1:0d:97,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=39,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -spice port=5900,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on

3. Login to vm, check the disks serial
(vm) # lsblk -o name,kname,serial
NAME          KNAME SERIAL
sda           sda   drive-scsi0-0-0-0
├─sda1        sda1  
└─sda2        sda2  
  ├─rhel-root dm-0  
  └─rhel-swap dm-1  
sdb           sdb   second-scsi
sr0           sr0   drive-ua-cdrom



B. Disable -blockdev
1. Start a VM with scsi disk without custom serial or custom alias, scsi disk with custom alias only, scsi disk with custom alias. 
VM xml:
```
<domain type='kvm'>
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/copy.qcow2'/>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/tmp/scsi.qcow2'/>
      <backingStore/>
      <target dev='sdb' bus='scsi'/>
      <serial>second-scsi</serial>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='sdc' bus='scsi'/>
      <readonly/>
      <alias name='ua-cdrom'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
...
</domain>
```

2. Start VM and check the qemu cmdline:
/usr/libexec/qemu-kvm -name guest=copy,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-copy/master-key.aes -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off -m 1024 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1 -object iothread,id=iothread2 -object iothread,id=iothread3 -uuid c3f1cdcb-cc26-462d-9e4a-fdb6f40402ca -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=36,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device qemu-xhci,id=usb1,bus=pci.0,addr=0xe -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0xc -device ahci,id=sata0,bus=pci.0,addr=0x9 -device ahci,id=sata1,bus=pci.0,addr=0xa -device ahci,id=sata2,bus=pci.0,addr=0xb -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -device usb-hub,id=hub0,bus=usb.0,port=3 -drive file=/var/lib/libvirt/images/copy.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/tmp/scsi.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-1 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=second-scsi,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1,serial=second-scsi -drive if=none,id=drive-ua-cdrom,readonly=on -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,device_id=drive-ua-cdrom,drive=drive-ua-cdrom,id=ua-cdrom -netdev tap,fd=38,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:d1:0d:97,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=39,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -spice port=5900,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on

3. Login to VM, and check the serial
(vm) # lsblk -o name,kname,serial
NAME          KNAME SERIAL
sda           sda   drive-scsi0-0-0-0
├─sda1        sda1  
└─sda2        sda2  
  ├─rhel-root dm-0  
  └─rhel-swap dm-1  
sdb           sdb   second-scsi
sr0           sr0   drive-ua-cdrom

Compared with results in A3, there is no change in disk serial.

Comment 18 Han Han 2019-09-18 07:11:56 UTC
It is better to chec the difference of `lsblk -O -J` before and after. No changes.


Note You need to log in before you can comment on or make changes to this bug.