Bug 1328020
Summary: | qemu-kvm-rhev gets SEGV when try to attach a disk image in a nfs pool to a vm with insufficient nfs authority | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | yisun | ||||
Component: | qemu-kvm-rhev | Assignee: | Markus Armbruster <armbru> | ||||
Status: | CLOSED DUPLICATE | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
Severity: | high | Docs Contact: | |||||
Priority: | high | ||||||
Version: | 7.3 | CC: | chayang, dyuan, hhan, juzhang, knoel, mrezanin, pzhang, rbalakri, sherold, virt-bugs, virt-maint, xuzhang, yfu, yisun | ||||
Target Milestone: | rc | Keywords: | Regression | ||||
Target Release: | --- | ||||||
Hardware: | x86_64 | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | QEMU 2.6 | Doc Type: | Bug Fix | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2016-08-02 09:09:31 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Description
yisun
2016-04-18 08:56:47 UTC
Version:qemu-kvm-rhev-2.5.0-4.el7.x86_64 Actually the guest is shutoff because of qemu SEGV: # abrt-cli ls id 0b8e32043b9a6b1f4a7fce461a37240a43317da4 reason: qemu-kvm killed by SIGSEGV time: Tue 19 Apr 2016 01:13:05 PM CST cmdline: /usr/libexec/qemu-kvm -name V,debug-threads=on -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off,vmport=off -cpu Opteron_G2 -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 78b740e1-a102-4d11-ba4e-b4010b488087 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-10-V/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/var/lib/libvirt/images/V.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0,vhost=on,vhostfd=27 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:3b:c9:9c,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-10-V/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on package: qemu-kvm-rhev-2.5.0-4.el7 uid: 107 (qemu) count: 1 Directory: /var/spool/abrt/ccpp-2016-04-19-13:13:05-19950 Run 'abrt-cli report /var/spool/abrt/ccpp-2016-04-19-13:13:05-19950' for creating a case in Red Hat Customer Portal It's qemu regression, not reproduced on qemu-kvm-rhev-2.3.0-31.el7_2.12.x86_64 # virsh start V Domain V started # virsh attach-disk V /mnt/q.img vdc error: Failed to attach disk error: internal error: unable to execute QEMU command '__com.redhat_drive_add': Device 'drive-virtio-disk2' could not be initialized # virsh list Id Name State ---------------------------------------------------- 11 V running # virsh attach-device V /tmp/disk.xml error: Failed to attach device from /tmp/disk.xml error: internal error: unable to execute QEMU command '__com.redhat_drive_add': Device 'drive-virtio-disk1' could not be initialized # virsh list Id Name State ---------------------------------------------------- 11 V running Created attachment 1148319 [details]
gdb backtrace info
Unable to reproduce it on qemu-kvm-2.6.0-0.1.rc2.fc25.x86_64 . It seems fixed on upstream version. -----------------------reproduce----------------------- version: qemu:qemu-kvm-rhev-2.5.0-4.el7.x86_64 kernel:kernel-3.10.0-396.el7.x86_64 Test steps: prepare a nfs server on one host: 1).# cat /etc/exports /home/nfs *(rw,sync,root_squash) 2).# service nfs restart 3).create one disk image in the path of nfs: #qemu-img create -f qcow2 /home/nfs/nfs-image.qcow2 5G on vm host: 1).# virsh start rhel7.3 Domain rhel7.3 started 2).# virsh list Id Name State ---------------------------------------------------- 8 rhel7.3 running 3).#mount "ip of nfs server":/home/nfs /mnt 4).create one xml file, for example: #cat disk.xml <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/mnt/nfs-image.qcow2'/> <target dev='vdb' bus='virtio'/> </disk> 5).#virsh attach-device rhel7.3 disk.xml error: Failed to attach device from disk.xml error: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' # 6).# virsh list --all Id Name State ---------------------------------------------------- - rhel7.3 shut off #cat /var/log/libvirt/qemu/rhel7.3.log ... Could not open '/mnt/nfs-image.qcow2': Permission denied Device 'drive-virtio-disk1' could not be initialized 2016-05-18 09:05:42.833+0000: shutting down 7).# abrt-cli ls id d3610947ebd7045036445fd87885d28bbfa12723 reason: qemu-kvm killed by SIGSEGV time: Wed 18 May 2016 05:05:23 AM EDT cmdline: /usr/libexec/qemu-kvm -name rhel7.3 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off,vmport=off -cpu IvyBridge -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid ccb0d6da-26e5-45b0-8674-217a0fe39666 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-rhel7.3/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot menu=on,strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/root/RHEL-Server-7.3-64-virtio-scsi.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:4c:0f:b3,bus=pci.0,addr=0x3,bootindex=2 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-rhel7.3/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on package: qemu-kvm-rhev-2.5.0-4.el7 uid: 107 (qemu) count: 1 Directory: /var/spool/abrt/ccpp-2016-05-18-05:05:23-25163 so reproduce this issue successfully. But if i test with qemu command line: 1).start one vm (the file rhel7.3.sh is get from the qemu command line above): # sh rhel7.3.sh char device redirected to /dev/pts/3 (label charserial0) QEMU 2.5.0 monitor - type 'help' for more information (qemu) __com.redhat_drive_add file=/mnt/nfs-image.qcow2,id=drive-virtio-disk1,format=qcow2 (qemu) device_add virtio-blk-pci,scsi=off,drive=drive-virtio-disk1,id=disk1 Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' (qemu) Attaching disk failed, but guest run well. -----------------------verification----------------------- version: qemu:qemu-kvm-rhev-2.6.0-1.el7.x86_64 kernel:kernel-3.10.0-396.el7.x86_64 Test steps: prepare a nfs server on one host: 1).# cat /etc/exports /home/nfs *(rw,sync,root_squash) 2).# service nfs restart 3).create one disk image in the path of nfs: #qemu-img create -f qcow2 /home/nfs/nfs-image.qcow2 5G on vm host: 1).# virsh start rhel7.3 Domain rhel7.3 started 2).# virsh list Id Name State ---------------------------------------------------- 9 rhel7.3 running 3).#mount "ip of nfs server":/home/nfs /mnt 4).create one xml file, for example: #cat disk.xml <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/mnt/nfs-image.qcow2'/> <target dev='vdb' bus='virtio'/> </disk> 5).#virsh attach-device rhel7.3 disk.xml error: Failed to attach device from disk.xml error: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' # virsh list --all Id Name State ---------------------------------------------------- - rhel7.3 shut off #cat /var/log/libvirt/qemu/rhel7.3.log ... Could not open '/mnt/nfs-image.qcow2': Permission denied Device 'drive-virtio-disk1' could not be initialized 2016-05-18 09:29:38.615+0000: shutting down 6)#abrt-cli ls id 50cf3fe400a84c49590d27d8ff09619432e3947c reason: qemu-kvm killed by SIGSEGV time: Wed 18 May 2016 05:29:14 AM EDT cmdline: /usr/libexec/qemu-kvm -name rhel7.3 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off,vmport=off -cpu IvyBridge -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid ccb0d6da-26e5-45b0-8674-217a0fe39666 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-rhel7.3/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot menu=on,strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/root/RHEL-Server-7.3-64-virtio-scsi.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:4c:0f:b3,bus=pci.0,addr=0x3,bootindex=2 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-rhel7.3/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on package: qemu-kvm-rhev-2.6.0-1.el7 uid: 107 (qemu) count: 1 Directory: /var/spool/abrt/ccpp-2016-05-18-05:29:14-27056 guest is shutoff because of qemu SEGV, so this bug still exist with qemu-kvm-rhev-2.6.0-1.el7.x86_64, still not be fixed. Please provide a stack backtrace and a log of the QMP commands issued by libvirt for the failed attach-device, if possible. (In reply to Markus Armbruster from comment #7) > Please provide a stack backtrace and a log of the QMP commands issued by > libvirt for the failed attach-device, if possible. ping? (In reply to Markus Armbruster from comment #7) > Please provide a stack backtrace and a log of the QMP commands issued by > libvirt for the failed attach-device, if possible. Can't reproduce this issue with the latest version: qemu-kvm-rhev-2.6.0-17.el7.x86_64. ----------------- For old version ----------------- version: qemu-kvm-rhev-2.6.0-1.el7.x86_64. Same steps with comments 6, reproduce this bug. #abrt-cli ls id 34e90d9982b99f4fd9d98518f5810a72a3446412 reason: qemu-kvm killed by SIGSEGV time: Tue 02 Aug 2016 03:50:31 AM EDT cmdline: /usr/libexec/qemu-kvm -name guest=rhel7.3,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-23-rhel7.3/master-key.aes -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off,vmport=off -cpu Opteron_G3 -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 202aff50-eacb-4ea5-84b3-d452542047c6 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-23-rhel7.3/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/home/kvm_autotest_root/images/RHEL-Server-7.3-64-virtio-scsi.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:81:12:22,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-23-rhel7.3/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on package: qemu-kvm-rhev-2.6.0-1.el7 uid: 107 (qemu) Directory: /var/spool/abrt/ccpp-2016-08-02-03:50:31-22147 Run 'abrt-cli report /var/spool/abrt/ccpp-2016-08-02-03:50:31-22147' for creating a case in Red Hat Customer Portal #cd /var/spool/abrt/ccpp-2016-08-02-03:50:31-22147 # gdb /usr/libexec/qemu-kvm coredump .... (gdb) bt #0 qstring_get_str (qstring=0x0) at qobject/qstring.c:128 #1 0x00007fbe6a91c98d in qdict_get_str (qdict=<optimized out>, key=key@entry=0x7fbe6a9b07d2 "id") at qobject/qdict.c:279 #2 0x00007fbe6a756ef5 in hmp_drive_del (mon=<optimized out>, qdict=<optimized out>) at blockdev.c:2843 #3 0x00007fbe6a68d3e5 in handle_qmp_command (parser=<optimized out>, tokens=<optimized out>) at /usr/src/debug/qemu-2.6.0/monitor.c:3922 #4 0x00007fbe6a91e450 in json_message_process_token (lexer=0x7fbe6d7d8448, input=0x7fbe6d7310e0, type=JSON_RCURLY, x=94, y=51) at qobject/json-streamer.c:94 #5 0x00007fbe6a932c1b in json_lexer_feed_char (lexer=lexer@entry=0x7fbe6d7d8448, ch=125 '}', flush=flush@entry=false) at qobject/json-lexer.c:310 #6 0x00007fbe6a932cde in json_lexer_feed (lexer=0x7fbe6d7d8448, buffer=<optimized out>, size=<optimized out>) at qobject/json-lexer.c:360 #7 0x00007fbe6a91e549 in json_message_parser_feed (parser=<optimized out>, buffer=<optimized out>, size=<optimized out>) at qobject/json-streamer.c:114 #8 0x00007fbe6a68ba0b in monitor_qmp_read (opaque=<optimized out>, buf=<optimized out>, size=<optimized out>) at /usr/src/debug/qemu-2.6.0/monitor.c:3938 #9 0x00007fbe6a75e351 in tcp_chr_read (chan=<optimized out>, cond=<optimized out>, opaque=0x7fbe6d7ebc20) at qemu-char.c:2895 #10 0x00007fbe5f7eed7a in g_main_context_dispatch () from /lib64/libglib-2.0.so.0 #11 0x00007fbe6a891440 in glib_pollfds_poll () at main-loop.c:213 #12 os_host_main_loop_wait (timeout=<optimized out>) at main-loop.c:258 #13 main_loop_wait (nonblocking=<optimized out>) at main-loop.c:506 #14 0x00007fbe6a65b55f in main_loop () at vl.c:1934 #15 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4667 ---------------For latest version --------------- version: qemu-kvm-rhev-2.6.0-17.el7.x86_64 Same steps with comment 6: #virsh list Id Name State ---------------------------------------------------- 24 rhel7.3 running # virsh attach-device rhel7.3 disk.xml error: Failed to attach device from disk.xml error: internal error: unable to execute QEMU command '__com.redhat_drive_add': Device 'drive-virtio-disk1' could not be initialized # virsh list Id Name State ---------------------------------------------------- 24 rhel7.3 running Add device failed, but guest run well, no SIGSEGV Your detailed test report shows that this is a duplicate of bug 1341531: hmp_drive_del() gets called as QMP command, and crashes when it accesses its qdict argument. Thanks! *** This bug has been marked as a duplicate of bug 1341531 *** |