Bug 1337100

Summary: redhat_drive_add should report error to qmp if it fails to initialize
Product: Red Hat Enterprise Linux 7 Reporter: Pei Zhang <pzhang>
Component: qemu-kvm-rhevAssignee: Fam Zheng <famz>
Status: CLOSED ERRATA QA Contact: FuXiangChun <xfu>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: chayang, dyuan, famz, hhan, huding, juzhang, knoel, mrezanin, mzhan, pezhang, virt-maint, xuzhang, yanyang, yisun
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.6.0-3.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-07 21:10:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
libvirt QMP log
none
all thread backtrace none

Description Pei Zhang 2016-05-18 09:50:30 UTC
Created attachment 1158696 [details]
libvirt QMP log

Description of problem:
Trying to attach an iscsi hostdev to a running guest, if the iscsi lun does not exist. This attach operation will kill the guest.

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.6.0-1.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. prepare a running guest, and hostdev xml like following
The lun iqn.2015-07.com.example.5.184:t1/2 is not existed.
# cat hostdev-scsi-net.xml
<hostdev mode='subsystem' type='scsi' managed='no'>
	<source protocol='iscsi' name='iqn.2015-07.com.example.5.184:t1/2'>
<host name='$IP' port='3260'/>
</source>
</hostdev>

2. attach-device to guest
# virsh attach-device rhel72 hostdev-scsi-net.xml
error: Failed to attach device from hostdev-scsi-net.xml
error: internal error: unable to execute QEMU command 'device_add': Property 'scsi-generic.drive' can't find value 'drive-hostdev1'

and the guest would be shut down here.


Actual results:
As step2, fail to attach device and guest is shut down.

Expected results:
It should report "Device 'xxx' could not be initialized" and the guest is still running.


Additional info:
log from qemu:
iSCSI: Failed to connect to LUN : SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:LOGICAL_UNIT_NOT_SUPPORTED(0x2500)
Device 'drive-hostdev1' could not be initialized
2016-05-18 09:19:27.874+0000: shutting down

Comment 3 Miroslav Rezanina 2016-05-23 09:20:45 UTC
Fix included in qemu-kvm-rhev-2.6.0-3.el7

Comment 5 Han Han 2016-06-01 07:53:01 UTC
It seems the bug unfixed on qemu-kvm-rhev-2.6.0-4.el7.x86_64. 
I tested it on:
qemu-kvm-rhev-2.6.0-4.el7.x86_64
libvirt-1.3.4-1.el7.x86_64
kernel-3.10.0-418.el7.x86_64

I prepare a VM named 18550, two scsi disk sdb/sdc
# virsh list 
 Id    Name                           State
----------------------------------------------------
 11    18550          
# lsscsi
[1:0:0:0]    disk    ATA      WDC WD5000AAKS-7 1D05  /dev/sda 
[6:0:0:0]    storage IET      Controller       0001  -        
[6:0:0:1]    disk    IET      VIRTUAL-DISK     0001  /dev/sdb 
[7:0:0:0]    disk    Kingston DT Ultimate G2   PMAP  /dev/sdc 
 
Attach two hostdev scsi disk with the same target address:
# cat hostdev1.xml    
<hostdev mode='subsystem' type='scsi' managed='no'>
<source>
<adapter name='scsi_host7'/>
<address bus='0' target='0' unit='0'/>
</source>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</hostdev>
# cat hostdev2.xml
<hostdev mode='subsystem' type='scsi' managed='no'>
<source>
<adapter name='scsi_host6'/>
<address bus='0' target='0' unit='1'/>
</source>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</hostdev>
# virsh attach-device 18550 hostdev1.xml
Device attached successfully
# virsh attach-device 18550 hostdev2.xml
error: Failed to attach device from hostdev2.xml
error: internal error: unable to execute QEMU command 'device_add': lun already used by 'hostdev0'

And I found the SIGSEGV of qemu:
# abrt-cli ls
id a26d4863a43c3df243f6f38c258c2bf0d4f6232c
reason:         qemu-kvm killed by SIGSEGV
time:           Wed 01 Jun 2016 03:51:17 PM CST
cmdline:        /usr/libexec/qemu-kvm -name 18550,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-11-18550/master-key.aes -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off,vmport=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 1d2cd91c-7848-40cf-91f3-a7de96afca78 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-11-18550/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x9 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/var/lib/libvirt/images/18550.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:b7:63:72,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-11-18550/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=0.0.0.0,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on
package:        qemu-kvm-rhev-2.6.0-4.el7
uid:            107 (qemu)
Directory:      /var/spool/abrt/ccpp-2016-06-01-03:51:17-10590
Run 'abrt-cli report /var/spool/abrt/ccpp-2016-06-01-03:51:17-10590' for creating a case in Red Hat Customer Portal

Comment 6 Han Han 2016-06-01 07:56:48 UTC
Created attachment 1163501 [details]
all thread backtrace

Comment 7 Fam Zheng 2016-06-01 08:31:29 UTC
Looks like another bug. Have you tried the xml in comment 0 (with nonexistent iscsi url)?

Comment 8 Han Han 2016-06-01 09:00:12 UTC
(In reply to Fam Zheng from comment #7)
> Looks like another bug. Have you tried the xml in comment 0 (with
> nonexistent iscsi url)?

I did as comment0, no VM crash. So it doesn't belong to this bug. I will open a new one for comment5

Comment 9 FuXiangChun 2016-09-12 11:13:51 UTC
reproduced bug with qemu-kvm-rhev-2.6.0-1.el7.x86_64

# virsh attach-device rhel7.3 hostdev-scsi-net.xml
error: Failed to attach device from hostdev-scsi-net.xml
error: internal error: unable to execute QEMU command 'device_add': Property 'scsi-generic.drive' can't find value 'drive-hostdev0'



<domain type='kvm'>
  <name>rhel7.3</name>
  <uuid>ff8b540c-9ef3-1441-9b0d-da7c97543f11</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>2</vcpu>
 <os>
<type arch='x86_64' machine='pc'>hvm</type>
<boot dev='hd'/>
</os>
  <features>
    <acpi/>
    <apic/>
    <pmu state='off'/>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough'>
    <feature policy='require' name='tsc-deadline'/>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/rhel7.3.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='none'/>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <source mode='bind'/>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='2'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/>
    </memballoon>
  </devices>
</domain>


verified with qemu-kvm-rhev-2.6.0-23.el7.x86_64

# virsh attach-device rhel7.3 hostdev-scsi-net.xml
error: Failed to attach device from hostdev-scsi-net.xml
error: internal error: unable to execute QEMU command '__com.redhat_drive_add': Device 'drive-hostdev0' could not be initialized

Comment 11 errata-xmlrpc 2016-11-07 21:10:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2673.html