Bug 1096576

Summary: QEMU core dumped when boot up two scsi-hd disk on the same virtio-scsi-pci controller in Intel host
Product: Red Hat Enterprise Linux 7 Reporter: Sibiao Luo <sluo>
Component: qemu-kvmAssignee: Fam Zheng <famz>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: medium    
Version: 7.0CC: chayang, famz, hhuang, juli, juzhang, michen, mrezanin, pbonzini, qzhang, rbalakri, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-1.5.3-67.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1096590 (view as bug list) Environment:
Last Closed: 2015-03-05 08:08:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1095609, 1096577, 1096590    

Description Sibiao Luo 2014-05-12 04:54:24 UTC
Description of problem:
QEMU core dumped when boot up two scsi-hd disk on the same virtio-scsi-pci controller in Intel host specified Intel/AMD cpu type.
If the two scsi-hd disks on the different virtio-scsi-pci controller which did not hit such issue.
The AMD host did not hit such issue if using the AMD cpu type, but can hit it if specified the Intel cpu type.

Version-Release number of selected component (if applicable):
host info:
# uname -r && rpm -q qemu-kvm
3.10.0-121.el7.x86_64
qemu-kvm-1.5.3-60.el7_0.1.x86_64
guest info:
# uname -r
3.10.0-121.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.boot up a KVM guest with two scsi-hd disks on the same virtio-scsi-pci controller on the Intel host.
# /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 ...-drive file=/home/RHEL-7.0-20140409.0_Server_x86_64.qcow2bk,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x4,id=scsi0 -device scsi-hd,drive=drive-system-disk,id=system-disk,bus=scsi0.0,bootindex=1 -drive file=/home/my-data-disk.raw,if=none,id=drive-hostdev0 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0
2.
3.

Actual results:
after step 1, qemu core dumped.
Core was generated by `/usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets'.
Program terminated with signal 11, Segmentation fault.
#0  qdev_get_fw_dev_path_helper (dev=0x7f650f977740, p=p@entry=0x7fffbd6a1210 "0", size=128) at hw/core/qdev.c:506
506	        l = qdev_get_fw_dev_path_helper(dev->parent_bus->parent, p, size);

(gdb) bt
#0  qdev_get_fw_dev_path_helper (dev=0x7f650f977740, p=p@entry=0x7fffbd6a1210 "0", size=128) at hw/core/qdev.c:506
#1  0x00007f650e63b823 in qdev_get_fw_dev_path (dev=<optimized out>) at hw/core/qdev.c:525
#2  0x00007f650e70d935 in get_boot_devices_list (size=size@entry=0x7fffbd6a1310) at vl.c:1229
#3  0x00007f650e66b190 in fw_cfg_machine_ready (n=0x7f650f8f24c0, data=<optimized out>) at hw/nvram/fw_cfg.c:503
#4  0x00007f650e81ec17 in notifier_list_notify (list=list@entry=0x7f650f02a2c8 <machine_init_done_notifiers>, 
    data=data@entry=0x0) at util/notify.c:39
#5  0x00007f650e5cbe34 in qemu_run_machine_init_done_notifiers () at vl.c:2692
#6  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4333
(gdb) bt full
#0  qdev_get_fw_dev_path_helper (dev=0x7f650f977740, p=p@entry=0x7fffbd6a1210 "0", size=128) at hw/core/qdev.c:506
        l = 0
#1  0x00007f650e63b823 in qdev_get_fw_dev_path (dev=<optimized out>) at hw/core/qdev.c:525
        path = "0", '\000' <repeats 15 times>, "p\022j\275\377\177\000\000\000\000\000\000\000\000\000\000[\000\000\000n", '\000' <repeats 19 times>, "w\000\000\000|\000\000\000o\022j\275\377\177\000\000\200\240\207\016e\177\000\000`.C\017e\177\000\000\254\004 \te\177\000\000\001\000\000\000\000\000\000\000 ", '\000' <repeats 15 times>, "\351\310p\016e\177\000"
        l = <optimized out>
#2  0x00007f650e70d935 in get_boot_devices_list (size=size@entry=0x7fffbd6a1310) at vl.c:1229
        devpath = 0x0
        bootpath = <optimized out>
        len = <optimized out>
        i = 0x7f650f975850
        total = 0
        list = 0x0
        __PRETTY_FUNCTION__ = "get_boot_devices_list"
#3  0x00007f650e66b190 in fw_cfg_machine_ready (n=0x7f650f8f24c0, data=<optimized out>) at hw/nvram/fw_cfg.c:503
        len = 140072029315072
        s = 0x7f650f8ef070
        bootindex = <optimized out>
#4  0x00007f650e81ec17 in notifier_list_notify (list=list@entry=0x7f650f02a2c8 <machine_init_done_notifiers>, 
    data=data@entry=0x0) at util/notify.c:39
        notifier = <optimized out>
        next = 0x7f650f8fe478
#5  0x00007f650e5cbe34 in qemu_run_machine_init_done_notifiers () at vl.c:2692
No locals.
#6  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4333
        i = <optimized out>
        snapshot = 0
        linux_boot = 0
        icount_option = 0x0
        initrd_filename = 0x0
        kernel_filename = 0x0
        kernel_cmdline = 0x7f650e87a080 ""
        boot_order = 0x7f650e831d46 "cad"
        ds = <optimized out>
        cyls = 0
        heads = 0
        secs = 0
        translation = 0
        hda_opts = <optimized out>
        opts = 0x0
        machine_opts = <optimized out>
        olist = <optimized out>
        optind = 45
        optarg = 0x7fffbd6a27af "scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0"
        loadvm = 0x0
        machine = 0x7f650ec053c0 <pc_machine_rhel700>
        cpu_model = 0x7fffbd6a23b3 "SandyBridge"
        vga_model = 0x7f650e85b84b "none"
        pid_file = 0x0
        incoming = 0x0
        show_vnc_port = 0
        defconfig = <optimized out>
        userconfig = 179
        log_mask = <optimized out>
        log_file = 0x0
        mem_trace = {malloc = 0x7f650e70c8e0 <malloc_and_trace>, realloc = 0x7f650e70c8c0 <realloc_and_trace>, 
          free = 0x7f650e70c8b0 <free_and_trace>, calloc = 0x0, try_malloc = 0x0, try_realloc = 0x0}
        trace_events = 0x0
        trace_file = 0x0
        __PRETTY_FUNCTION__ = "main"
        args = {machine = 0x7f650ec053c0 <pc_machine_rhel700>, ram_size = 4294967296, 
          boot_device = 0x7f650e831d46 "cad", kernel_filename = 0x0, kernel_cmdline = 0x7f650e87a080 "", 
          initrd_filename = 0x0, cpu_model = 0x7fffbd6a23b3 "SandyBridge"}
(gdb)

Expected results:
It should no any qemu core dumped.

Additional info:
# /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo_test -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,path=/tmp/helloworld1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2 -drive file=/home/RHEL-7.0-20140409.0_Server_x86_64.qcow2bk,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x4,id=scsi0 -device scsi-hd,drive=drive-system-disk,id=system-disk,bus=scsi0.0,bootindex=1 -nodefaults -vnc :1 -monitor stdio -drive file=/home/my-data-disk.raw,if=none,id=drive-hostdev0 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0

Comment 1 Sibiao Luo 2014-05-12 05:15:55 UTC
According to my testing, this issue is not a regression issue.

qemu-kvm-1.5.3-60.el7_0.1.x86_64 - hit it
qemu-kvm-1.5.3-60.el7.x86_64     - hit it
qemu-kvm-1.5.3-55.el7.x86_64     - hit it
qemu-kvm-1.5.3-49.el7.x86_64     - hit it
qemu-kvm-1.5.3-38.el7.x86_64     - hit it
qemu-kvm-1.5.3-30.el7.x86_64     - hit it
qemu-kvm-1.5.3-10.el7.x86_64     - hit it


Best Regards,
sluo

Comment 2 Sibiao Luo 2014-05-12 05:33:17 UTC
(In reply to Sibiao Luo from comment #0)
> Description of problem:
> QEMU core dumped when boot up two scsi-hd disk on the same virtio-scsi-pci
> controller in Intel host specified Intel/AMD cpu type.
> If the two scsi-hd disks on the different virtio-scsi-pci controller which
> did not hit such issue.
> The AMD host did not hit such issue if using the AMD cpu type, but can hit
> it if specified the Intel cpu type.
> 
Also tried other Intel host (provided by juli) with the same testing as comment #0 which also can hit this issue.

Best Regards,
sluo

Comment 3 Sibiao Luo 2014-06-10 05:52:57 UTC
(In reply to Sibiao Luo from comment #0)
> Steps to Reproduce:
> 1.boot up a KVM guest with two scsi-hd disks on the same virtio-scsi-pci
> controller on the Intel host.
> # /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 4096 -smp
> 4,sockets=2,cores=2,threads=1 ...-drive
> file=/home/RHEL-7.0-20140409.0_Server_x86_64.qcow2bk,if=none,id=drive-system-
> disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device
> virtio-scsi-pci,bus=pci.0,addr=0x4,id=scsi0 -device
> scsi-hd,drive=drive-system-disk,id=system-disk,bus=scsi0.0,bootindex=1
> -drive file=/home/my-data-disk.raw,if=none,id=drive-hostdev0 -device
> scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,
> id=hostdev0,bus=scsi0.0
Thanks famz who point out my double 'bus' specified in qemu-kvm command line, but I still hit it with qemu-kvm-1.5.3-60.el7.x86_64 after i remove the duplicate 'bus'.

e.g:...-drive file=/home/RHEL-Server-7.0-64-virtio.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x4,id=scsi0 -device scsi-hd,drive=drive-system-disk,id=system-disk,bus=scsi0.0,bootindex=1 -drive file=/home/my-data-disk.raw,if=none,id=drive-hostdev0 -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0

host info:
# uname -r && rpm -q qemu-kvm
3.10.0-127.el7.x86_64
qemu-kvm-1.5.3-60.el7.x86_64
guest info:
# uname -r
3.10.0-127.el7.x86_64

(gdb) bt
#0  qdev_get_fw_dev_path_helper (dev=0x7f43a16440c0, p=p@entry=0x7fffb65a8410 "0", size=128) at hw/core/qdev.c:506
#1  0x00007f43a020a6d3 in qdev_get_fw_dev_path (dev=<optimized out>) at hw/core/qdev.c:525
#2  0x00007f43a02dc775 in get_boot_devices_list (size=size@entry=0x7fffb65a8510) at vl.c:1229
#3  0x00007f43a023a040 in fw_cfg_machine_ready (n=0x7f43a158c460, data=<optimized out>) at hw/nvram/fw_cfg.c:503
#4  0x00007f43a03ed647 in notifier_list_notify (list=list@entry=0x7f43a0bf92c8 <machine_init_done_notifiers>, 
    data=data@entry=0x0) at util/notify.c:39
#5  0x00007f43a019adc4 in qemu_run_machine_init_done_notifiers () at vl.c:2692
#6  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4333
(gdb) q

Comment 5 Fam Zheng 2014-06-11 06:10:42 UTC
*** Bug 1096575 has been marked as a duplicate of this bug. ***

Comment 6 Markus Armbruster 2014-07-17 09:34:03 UTC
*** Bug 1095606 has been marked as a duplicate of this bug. ***

Comment 7 Jeff Nelson 2014-08-08 16:55:07 UTC
Fix included in qemu-kvm-1.5.3-67.el7

Comment 9 Sibiao Luo 2014-10-11 05:51:08 UTC
Verify this issue on qemu-kvm-1.5.3-75.el7.x86_64.

host info:
# uname -r && rpm -q qemu-kvm
3.10.0-171.el7.x86_64
qemu-kvm-1.5.3-75.el7.x86_64

e.g:...-drive file=/home/RHEL-7.0-Server-Released_x86_64.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x4,id=scsi0 -device scsi-hd,drive=drive-system-disk,id=system-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,bootindex=1...-drive file=/home/my-data-disk.raw,if=none,id=drive-hostdev0 -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0
(qemu) qemu-kvm: -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0: lun already used by 'system-disk'
qemu-kvm: -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0: Device initialization failed.
qemu-kvm: -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0: Device 'scsi-hd' could not be initialized

##################################################

Also try qemu-kvm-rhev-2.1.2-1.el7.x86_64 version which also did not hit hit this issue.
host info:
# uname -r && rpm -q qemu-kvm-rhev
3.10.0-171.el7.x86_64
qemu-kvm-rhev-2.1.2-1.el7.x86_64

e.g:...-drive file=/home/RHEL-7.0-Server-Released_x86_64.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x4,id=scsi0 -device scsi-hd,drive=drive-system-disk,id=system-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,bootindex=1...-drive file=/home/my-data-disk.raw,if=none,id=drive-hostdev0 -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0Warning: option deprecated, use lost_tick_policy property of kvm-pit instead.
QEMU 2.1.2 monitor - type 'help' for more information
(qemu) qemu-kvm: -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0: lun already used by 'system-disk'
qemu-kvm: -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0: Device initialization failed.
qemu-kvm: -device scsi-hd,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0,bus=scsi0.0: Device 'scsi-hd' could not be initialized

Base on above, this issue has been fixed correctly, move to verified status.

Best Regards,
sluo

Comment 12 errata-xmlrpc 2015-03-05 08:08:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0349.html