Bug 578975

Summary: libvirt device_add does not work for scsi hotplug
Product: [Community] Virtualization Tools Reporter: Jamie Strandboge <jamie>
Component: libvirtAssignee: Daniel Veillard <veillard>
Status: CLOSED NOTABUG QA Contact:
Severity: medium Docs Contact:
Priority: low    
Version: unspecifiedCC: crobinso, dustin.kirkland, xen-maint
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-04-02 14:30:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jamie Strandboge 2010-04-01 22:58:50 UTC
Description of problem:
libvirt 0.7.5 would use the following to hotplug a scsi device:
pci_add audo storage file=<path>,if=scsi

libvirt 0.7.7 uses the following:
drive_add dummy file=<path>,if=none,id=drive-scsi0-0-1

Attaching to the qemu monitor and running the above device_add manually results in qemu returning 'OK' but using 'info block' shows the device was not added.

From #virt on OFTC from today:
17:38 < aliguori> anyway, it needs to do pci_add to add a scsi adapter
17:38 < aliguori> remember the adapter bus address
17:39 < aliguori> then do device_add <bus-address> file=....
17:39 < kirkland> aliguori: agreed ... so libvirt needs to special case pci_add 
                  for scsi hotadd?
17:39 < aliguori> yes
17:39 < aliguori> scsi hotadd is different than virtio hotadd
17:39 < aliguori> old qemu used to let you get away with just doing pci_add...
17:39 < aliguori> but that was a bug
17:39 < aliguori> so looks like they tried to update that logic but maybe 
                  didn't do it correctly

The problematic commit seems to be:
264e98d6a85bb467f364006c1dac3993688691ce
Subject: [PATCH] Make hotplug use new device_add where possible

Comment 1 Jamie Strandboge 2010-04-02 13:57:39 UTC
I looked at this a bit more. With the following XML:
<disk type='block'>
  <driver name='phy'/>
  <source dev='/tmp/dev.dd'/>
  <target dev='sdb' bus='scsi'/>
</disk>

If I do:
dd if=/dev/zero of=/tmp/dev.dd bs=1M count=64

Then do:
virsh attach-device guest /tmp/dev.xml

I see the following with LIBVIRT_DEBUG=1:
08:42:17.250: debug : qemuMonitorAddDrive:1365 : mon=0xd2b400, fd=42 drive=file=/tmp/dev.dd,if=none,id=drive-scsi0-0-1
08:42:17.250: debug : qemuMonitorCommandWithHandler:230 : Send command 'drive_add dummy file=/tmp/dev.dd,if=none,id=drive-scsi0-0-1' for write with FD -1
08:42:17.251: info : qemudDispatchSignalEvent:390 : Received unexpected signal 17
08:42:17.255: debug : qemuMonitorAddDevice:1352 : mon=0xd2b400, fd=42 device=scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1
08:42:17.255: debug : qemuMonitorCommandWithHandler:230 : Send command 'device_add scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1' for write with FD -1

If at this point I kill libvirtd and attach to the monitor with:
nc -U /var/lib/.../guest.log

I can see the drive was added:
(qemu) info block
info block
...
drive-scsi0-0-1: type=hd removable=0 file=/tmp/dev.dd ro=0 drv=raw encrypted=0

But it does not show up in the pci list:
(qemu) info pci
info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
      id ""
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
      id ""
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xc000 [0xc00f].
      id ""
  Bus  0, device   1, function 2:
    USB controller: PCI device 8086:7020
      IRQ 11.
      BAR4: I/O at 0xc020 [0xc03f].
      id ""
  Bus  0, device   1, function 3:
    Bridge: PCI device 8086:7113
      IRQ 9.
      id ""
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1013:00b8
      BAR0: 32 bit prefetchable memory at 0xe0000000 [0xe1ffffff].
      BAR1: 32 bit memory at 0xe2000000 [0xe2000fff].
      id ""
  Bus  0, device   3, function 0:
    RAM controller: PCI device 1af4:1002
      IRQ 11.
      BAR0: I/O at 0xc040 [0xc05f].
      id "balloon0"
  Bus  0, device   4, function 0:
    Ethernet controller: PCI device 1af4:1000
      IRQ 11.
      BAR0: I/O at 0xc060 [0xc07f].
      BAR1: 32 bit memory at 0xe2001000 [0xe2001fff].
      id "net0"
  Bus  0, device   5, function 0:
    Ethernet controller: PCI device 10ec:8139
      IRQ 10.
      BAR0: I/O at 0xc100 [0xc1ff].
      BAR1: 32 bit memory at 0xe2002000 [0xe20020ff].
      id "net1"
  Bus  0, device   6, function 0:
    SCSI controller: PCI device 1000:0012
      IRQ 10.
      BAR0: I/O at 0xc200 [0xc2ff].
      BAR1: 32 bit memory at 0xe2002400 [0xe20027ff].
      BAR2: 32 bit memory at 0xe2004000 [0xe2005fff].
      id "scsi0"

It is unclear whether this is a qemu bug or improper use of drive_add and device_add.

Comment 2 Jamie Strandboge 2010-04-02 14:30:04 UTC
Ok, this is neither a libvirt bug nor a qemu bug. From irc:

09:07 < aliguori> jdstrand, you're adding a scsi disk to an existing scsi 
                  adapter
09:07 < aliguori> lspci isn't going to change
09:07 < aliguori> but you won't see anything in /proc/partitions until you 
                  rescan the scsi bus
09:07 < aliguori> which is not something that happens automatically
09:07 < aliguori> it's *not* pci hot add
09:08 < aliguori> it's scsi hot add
09:08 < aliguori> http://kbase.redhat.com/faq/docs/DOC-3942
09:08 < aliguori> this is different than what older libvirts did
09:08 < aliguori> older libvirt added a new scsi controller for every disk add
09:08 < aliguori> which was incorrect
09:14 < jdstrand> aliguori: I'm just used to seeing this show up in the guest, 
                  so the new behavior, while I understand it, needs to be dealt 
                  with in applications that depended on the previously faulty 
                  behavior
09:14 < aliguori> yup


So, after using attach-device, I perform the following:
echo "- - -" > /sys/class/scsi_host/host0/scan

Then the disk shows up in the disk (from kern.log):
[ 2306.626064] scsi 0:0:1:0: Direct-Access     QEMU     QEMU HARDDISK    0.11 PQ: 0 ANSI: 3
[ 2306.626093] scsi target0:0:1: tagged command queuing enabled, command queue depth 16.
[ 2306.626109] scsi target0:0:1: Beginning Domain Validation
[ 2306.627209] scsi target0:0:1: Domain Validation skipping write tests
[ 2306.627212] scsi target0:0:1: Ending Domain Validation
[ 2306.627626] sd 0:0:1:0: Attached scsi generic sg1 type 0
[ 2306.650254] sd 0:0:1:0: [sdb] 131072 512-byte logical blocks: (67.1 MB/64.0 MiB)
[ 2306.650417] sd 0:0:1:0: [sdb] Write Protect is off
[ 2306.650422] sd 0:0:1:0: [sdb] Mode Sense: 1c 00 00 08
[ 2306.650935] sd 0:0:1:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 2306.651920]  sdb: unknown partition table
[ 2306.663843] sd 0:0:1:0: [sdb] Attached SCSI disk

$ cat /proc/partitions 
major minor  #blocks  name
...
   8       16      65536 sdb


While not a bug, it will feel like a regression to users who depend on scsi hotplug to work in the guest. This should perhaps be addressed in documentation, if it isn't already.