Bug 881486

Summary: [abrt] qemu-system-x86-1.2.0-23.fc18: object_finalize: Process /usr/bin/qemu-kvm was killed by signal 6 (SIGABRT)
Product: [Fedora] Fedora Reporter: Lucas Meneghel Rodrigues <lmr>
Component: qemuAssignee: Fedora Virtualization Maintainers <virt-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: amit.shah, areis, berrange, cfergeau, crobinso, dwmw2, itamar, knoel, pbonzini, rjones, scottt.tw, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: abrt_hash:1e5717065185ee7e2aa3f26ce6a7bedbfbe3b436
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-01-14 19:48:56 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Attachments:
Description Flags
File: backtrace
none
File: cgroup
none
File: core_backtrace
none
File: dso_list
none
File: environ
none
File: limits
none
File: maps
none
File: open_fds
none
File: proc_pid_status
none
File: smolt_data
none
File: var_log_messages none

Description Lucas Meneghel Rodrigues 2012-11-28 19:35:24 EST
Description of problem:
I was testing qemu-kvm with the virt-tests, executing virtio-console tests:

21:40:43 INFO | Context: Executing test: test_hotplug_virtio_pci
21:40:43 DEBUG| (monitor hmp1) Sending command 'device_add virtio-serial-pci,id=virtio_serial_pci1' 
21:40:53 DEBUG| (monitor hmp1) Sending command 'device_del virtio_serial_pci1' 
21:40:53 INFO | [qemu output] **
21:40:53 INFO | [qemu output] ERROR:qom/object.c:386:object_finalize: assertion failed: (obj->ref == 0)
21:40:59 INFO | [qemu output] /bin/sh: line 1: 10095 Aborted                 (core dumped) /usr/bin/qemu-kvm -S -name 'vm1' -nodefaults -chardev socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20121128-212634-OmvndKRl,server,nowait -mon chardev=hmp_id_hmp1,mode=readline -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20121128-212634-OmvndKRl,server,nowait -device isa-serial,chardev=serial_id_serial1 -device virtio-serial-pci,id=virtio_serial_pci0 -chardev socket,id=devvc1,path=/tmp/virtio_port-vc1-20121128-212634-OmvndKRl,server,nowait -device virtconsole,chardev=devvc1,name=vc1,id=vc1 -chardev socket,id=devvc2,path=/tmp/virtio_port-vc2-20121128-212634-OmvndKRl,server,nowait -device virtconsole,chardev=devvc2,name=vc2,id=vc2 -chardev socket,id=devvc3,path=/tmp/virtio_port-vc3-20121128-212634-OmvndKRl,server,nowait -device virtconsole,chardev=devvc3,name=vc3,id=vc3 -chardev socket,id=devvc4,path=/tmp/virtio_port-vc4-20121128-212634-OmvndKRl,server,nowait -device virtconsole,chardev=devvc4,name=vc4,id=vc4 -chardev socket,id=devvs1,path=/tmp/virtio_port-vs1-20121128-212634-OmvndKRl,server,nowait -device virtserialport,chardev=devvs1,name=vs1,id=vs1 -chardev socket,id=devvs2,path=/tmp/virtio_port-vs2-20121128-212634-OmvndKRl,server,nowait -device virtserialport,chardev=devvs2,name=vs2,id=vs2 -chardev socket,id=devvs3,path=/tmp/virtio_port-vs3-20121128-212634-OmvndKRl,server,nowait -device virtserialport,chardev=devvs3,name=vs3,id=vs3 -chardev socket,id=devvs4,path=/tmp/virtio_port-vs4-20121128-212634-OmvndKRl,server,nowait -device virtserialport,chardev=devvs4,name=vs4,id=vs4 -chardev socket,id=seabioslog_id_20121128-212634-OmvndKRl,path=/tmp/seabios-20121128-212634-OmvndKRl,server,nowait -device isa-debugcon,chardev=seabioslog_id_20121128-212634-OmvndKRl,iobase=0x402 -device ich9-usb-uhci1,id=usb1 -drive file='/home/lmr/Code/virt-test.git/shared/data/images/jeos-17-64.qcow2',if=none,cache=none,id=virtio0 -device virtio-blk-pci,drive=virtio0 -device virtio-net-pci,netdev=idnUVXHv,mac='9a:be:bf:c0:c1:c2',id='idVim6Y5' -netdev user,id=idnUVXHv,hostfwd=tcp::5000-:22 -m 512 -smp 2,cores=1,threads=1,sockets=2 -cpu 'Penryn' -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -vga std -rtc base=utc,clock=host,driftfix=none -boot order=cdn,once=c,menu=off -enable-kvm
21:40:59 INFO | [qemu output] (Process terminated with status 134)

Version-Release number of selected component:
qemu-system-x86-1.2.0-23.fc18

Additional info:
backtrace_rating: 4
cmdline:        /usr/bin/qemu-kvm -S -name vm1 -nodefaults -chardev socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20121128-212634-OmvndKRl,server,nowait -mon chardev=hmp_id_hmp1,mode=readline -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20121128-212634-OmvndKRl,server,nowait -device isa-serial,chardev=serial_id_serial1 -device virtio-serial-pci,id=virtio_serial_pci0 -chardev socket,id=devvc1,path=/tmp/virtio_port-vc1-20121128-212634-OmvndKRl,server,nowait -device virtconsole,chardev=devvc1,name=vc1,id=vc1 -chardev socket,id=devvc2,path=/tmp/virtio_port-vc2-20121128-212634-OmvndKRl,server,nowait -device virtconsole,chardev=devvc2,name=vc2,id=vc2 -chardev socket,id=devvc3,path=/tmp/virtio_port-vc3-20121128-212634-OmvndKRl,server,nowait -device virtconsole,chardev=devvc3,name=vc3,id=vc3 -chardev socket,id=devvc4,path=/tmp/virtio_port-vc4-20121128-212634-OmvndKRl,server,nowait -device virtconsole,chardev=devvc4,name=vc4,id=vc4 -chardev socket,id=devvs1,path=/tmp/virtio_port-vs1-20121128-212634-OmvndKRl,server,nowait -device virtserialport,chardev=devvs1,name=vs1,id=vs1 -chardev socket,id=devvs2,path=/tmp/virtio_port-vs2-20121128-212634-OmvndKRl,server,nowait -device virtserialport,chardev=devvs2,name=vs2,id=vs2 -chardev socket,id=devvs3,path=/tmp/virtio_port-vs3-20121128-212634-OmvndKRl,server,nowait -device virtserialport,chardev=devvs3,name=vs3,id=vs3 -chardev socket,id=devvs4,path=/tmp/virtio_port-vs4-20121128-212634-OmvndKRl,server,nowait -device virtserialport,chardev=devvs4,name=vs4,id=vs4 -chardev socket,id=seabioslog_id_20121128-212634-OmvndKRl,path=/tmp/seabios-20121128-212634-OmvndKRl,server,nowait -device isa-debugcon,chardev=seabioslog_id_20121128-212634-OmvndKRl,iobase=0x402 -device ich9-usb-uhci1,id=usb1 -drive file=/home/lmr/Code/virt-test.git/shared/data/images/jeos-17-64.qcow2,if=none,cache=none,id=virtio0 -device virtio-blk-pci,drive=virtio0 -device virtio-net-pci,netdev=idnUVXHv,mac=9a:be:bf:c0:c1:c2,id=idVim6Y5 -netdev user,id=idnUVXHv,hostfwd=tcp::5000-:22 -m 512 -smp 2,cores=1,threads=1,sockets=2 -cpu Penryn -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -vga std -rtc base=utc,clock=host,driftfix=none -boot order=cdn,once=c,menu=off -enable-kvm
crash_function: object_finalize
executable:     /usr/bin/qemu-kvm
kernel:         3.6.7-5.fc18.x86_64
remote_result:  NOTFOUND
uid:            1000
xsession_errors: 

Truncated backtrace:
Thread no. 1 (10 frames)
 #4 object_finalize at qom/object.c:386
 #5 qbus_free at hw/qdev.c:476
 #6 device_finalize at hw/qdev.c:701
 #7 object_deinit at qom/object.c:370
 #8 object_finalize at qom/object.c:383
 #9 object_delete at qom/object.c:414
 #10 qdev_free at hw/qdev.c:270
 #11 acpi_piix_eject_slot at /usr/src/debug/qemu-kvm-1.2.0/hw/acpi_piix4.c:314
 #12 kvm_handle_io at /usr/src/debug/qemu-kvm-1.2.0/kvm-all.c:1461
 #13 kvm_cpu_exec at /usr/src/debug/qemu-kvm-1.2.0/kvm-all.c:1603
Comment 1 Lucas Meneghel Rodrigues 2012-11-28 19:35:32 EST
Created attachment 653862 [details]
File: backtrace
Comment 2 Lucas Meneghel Rodrigues 2012-11-28 19:35:34 EST
Created attachment 653864 [details]
File: cgroup
Comment 3 Lucas Meneghel Rodrigues 2012-11-28 19:35:36 EST
Created attachment 653865 [details]
File: core_backtrace
Comment 4 Lucas Meneghel Rodrigues 2012-11-28 19:35:39 EST
Created attachment 653866 [details]
File: dso_list
Comment 5 Lucas Meneghel Rodrigues 2012-11-28 19:35:42 EST
Created attachment 653867 [details]
File: environ
Comment 6 Lucas Meneghel Rodrigues 2012-11-28 19:35:44 EST
Created attachment 653868 [details]
File: limits
Comment 7 Lucas Meneghel Rodrigues 2012-11-28 19:35:47 EST
Created attachment 653869 [details]
File: maps
Comment 8 Lucas Meneghel Rodrigues 2012-11-28 19:35:49 EST
Created attachment 653870 [details]
File: open_fds
Comment 9 Lucas Meneghel Rodrigues 2012-11-28 19:35:52 EST
Created attachment 653871 [details]
File: proc_pid_status
Comment 10 Lucas Meneghel Rodrigues 2012-11-28 19:35:54 EST
Created attachment 653872 [details]
File: smolt_data
Comment 11 Lucas Meneghel Rodrigues 2012-11-28 19:35:57 EST
Created attachment 653873 [details]
File: var_log_messages
Comment 12 Lucas Meneghel Rodrigues 2012-12-10 10:35:46 EST
I was running the virtualization test suite, during a sequence of device add a virtio serial device, followed by its removal about 12 seconse later.

13:08:56 INFO | Context: Executing test: test_hotplug_virtio_pci
13:08:56 DEBUG| (monitor hmp1) Sending command 'device_add virtio-serial-pci,id=virtio_serial_pci1' 
13:09:06 DEBUG| (monitor hmp1) Sending command 'device_del virtio_serial_pci1' 
13:09:06 INFO | [qemu output] **
13:09:06 INFO | [qemu output] ERROR:qom/object.c:386:object_finalize: assertion failed: (obj->ref == 0)
13:09:09 INFO | [qemu output] /bin/sh: line 1: 16358 Aborted                 (core dumped) /bin/qemu-kvm -S -name 'vm1' -nodefaults -chardev socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20121210-125737-7qOH7IXP,server,nowait -mon chardev=hmp_id_hmp1,mode=readline -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20121210-125737-7qOH7IXP,server,nowait -device isa-serial,chardev=serial_id_serial1 -device virtio-serial-pci,id=virtio_serial_pci0 -chardev socket,id=devvc1,path=/tmp/virtio_port-vc1-20121210-125737-7qOH7IXP,server,nowait -device virtconsole,chardev=devvc1,name=vc1,id=vc1 -chardev socket,id=devvc2,path=/tmp/virtio_port-vc2-20121210-125737-7qOH7IXP,server,nowait -device virtconsole,chardev=devvc2,name=vc2,id=vc2 -chardev socket,id=devvc3,path=/tmp/virtio_port-vc3-20121210-125737-7qOH7IXP,server,nowait -device virtconsole,chardev=devvc3,name=vc3,id=vc3 -chardev socket,id=devvc4,path=/tmp/virtio_port-vc4-20121210-125737-7qOH7IXP,server,nowait -device virtconsole,chardev=devvc4,name=vc4,id=vc4 -chardev socket,id=devvs1,path=/tmp/virtio_port-vs1-20121210-125737-7qOH7IXP,server,nowait -device virtserialport,chardev=devvs1,name=vs1,id=vs1 -chardev socket,id=devvs2,path=/tmp/virtio_port-vs2-20121210-125737-7qOH7IXP,server,nowait -device virtserialport,chardev=devvs2,name=vs2,id=vs2 -chardev socket,id=devvs3,path=/tmp/virtio_port-vs3-20121210-125737-7qOH7IXP,server,nowait -device virtserialport,chardev=devvs3,name=vs3,id=vs3 -chardev socket,id=devvs4,path=/tmp/virtio_port-vs4-20121210-125737-7qOH7IXP,server,nowait -device virtserialport,chardev=devvs4,name=vs4,id=vs4 -chardev socket,id=seabioslog_id_20121210-125737-7qOH7IXP,path=/tmp/seabios-20121210-125737-7qOH7IXP,server,nowait -device isa-debugcon,chardev=seabioslog_id_20121210-125737-7qOH7IXP,iobase=0x402 -device ich9-usb-uhci1,id=usb1 -drive file='/home/lmr/Code/virt-test.git/shared/data/images/jeos-17-64.qcow2',if=none,cache=none,id=virtio0 -device virtio-blk-pci,drive=virtio0 -device virtio-net-pci,netdev=idyeh55u,mac='9a:a4:a5:a6:a7:a8',id='idBCG9j1' -netdev user,id=idyeh55u,hostfwd=tcp::5002-:22 -m 512 -smp 2,cores=1,threads=1,sockets=2 -cpu 'SandyBridge' -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -vga std -rtc base=utc,clock=host,driftfix=none -boot order=cdn,once=c,menu=off -enable-kvm
13:09:09 INFO | [qemu output] (Process terminated with status 134)


backtrace_rating: 4
Package: qemu-system-x86-1.2.0-24.fc18
OS Release: Fedora release 18 (Spherical Cow)
Comment 13 Cole Robinson 2012-12-14 17:24:11 EST
I think this might be fixed by

commit f05a3da4e00d24c4540811e6fff2c4f0484771bd
Author: Paolo Bonzini <pbonzini@redhat.com>
Date:   Fri Nov 23 09:47:12 2012 +0100

    qom: fix refcount of non-heap-allocated objects
    
    The reference count for embedded objects is always one too low, because
    object_initialize_with_type returns with zero references to the object.
    This causes premature finalization of the object (or an assertion failure)
    after calling object_ref to add an extra reference and object_unref to
    remove it.
    
    The fix is to move the initial object_ref call from object_new_with_type
    to object_initialize_with_type.
    
    Acked-by: Andreas Färber <afaerber@suse.de>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
    (cherry picked from commit 764b63125a77dab54ed405d493452a4e05679c2e)
    
    Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>

Which is in 1.2.2 stable
Comment 14 Fedora Update System 2012-12-16 20:27:33 EST
qemu-1.2.2-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/qemu-1.2.2-1.fc18
Comment 15 Paolo Bonzini 2012-12-17 08:37:33 EST
I agree, either that or another patch later in the same series.
Comment 16 Lucas Meneghel Rodrigues 2012-12-17 09:17:20 EST
Executing the test case mentioned in the current 1.2.0 shipped on F18 gives the error reported:

$ ./run -t kvm --tests virtio_console.spread_linear.unspecifiable.hotplug_virtio_pci
SETUP: PASS (1.20 s)
DATA DIR: /path/to/virt_test/
DEBUG LOG: /path/to/Code/virt-test.git/logs/run-2012-12-17-11.59.17/debug.log
TESTS: 1
(1/1) virtio_console.spread_linear.unspecifiable.hotplug_virtio_pci: FAIL (26.49 s)

The updated 1.2.2 did not get to the updates-testing repos yet. What I did was to download a clean copy of 1.2.2, build it and repeat the test. The test still fails, with the same segmentation fault:

$ ./run -t kvm --tests virtio_console.spread_linear.unspecifiable.hotplug_virtio_pci --qemu-bin /path/to/Code/qemu-1.2.2/x86_64-softmmu/qemu-system-x86_64 
SETUP: PASS (1.31 s)
DATA DIR: /path/to/virt_test/
DEBUG LOG: /path/to/Code/virt-test.git/logs/run-2012-12-17-12.10.07/debug.log
TESTS: 1
(1/1) virtio_console.spread_linear.unspecifiable.hotplug_virtio_pci: FAIL (25.81 s)

... in the debug log ...

12:10:08 INFO | Context: Executing test: test_hotplug_virtio_pci
12:10:08 DEBUG| (monitor hmp1) Sending command 'device_add virtio-serial-pci,id=virtio_serial_pci1' 
12:10:18 DEBUG| (monitor hmp1) Sending command 'device_del virtio_serial_pci1' 
12:10:18 INFO | [qemu output] **
12:10:18 INFO | [qemu output] ERROR:qom/object.c:387:object_finalize: assertion failed: (obj->ref == 0)
12:10:21 INFO | [qemu output] /bin/sh: line 1:  7962 Aborted                 (core dumped) /path/to/Code/qemu-1.2.2/x86_64-softmmu/qemu-system-x86_64 -S -name 'vm1' -n
odefaults -chardev socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20121217-121007-bptwZQCu,server,nowait -mon chardev=hmp_id_hmp1,mode=readline -chardev socket,id=serial_
id_serial1,path=/tmp/serial-serial1-20121217-121007-bptwZQCu,server,nowait -device isa-serial,chardev=serial_id_serial1 -device virtio-serial-pci,id=virtio_serial_pci0 
-chardev socket,id=devvc1,path=/tmp/virtio_port-vc1-20121217-121007-bptwZQCu,server,nowait -device virtconsole,chardev=devvc1,name=vc1,id=vc1 -chardev socket,id=devvc2,
path=/tmp/virtio_port-vc2-20121217-121007-bptwZQCu,server,nowait -device virtconsole,chardev=devvc2,name=vc2,id=vc2 -chardev socket,id=devvc3,path=/tmp/virtio_port-vc3-
20121217-121007-bptwZQCu,server,nowait -device virtconsole,chardev=devvc3,name=vc3,id=vc3 -chardev socket,id=devvc4,path=/tmp/virtio_port-vc4-20121217-121007-bptwZQCu,s
erver,nowait -device virtconsole,chardev=devvc4,name=vc4,id=vc4 -chardev socket,id=devvs1,path=/tmp/virtio_port-vs1-20121217-121007-bptwZQCu,server,nowait -device virts
erialport,chardev=devvs1,name=vs1,id=vs1 -chardev socket,id=devvs2,path=/tmp/virtio_port-vs2-20121217-121007-bptwZQCu,server,nowait -device virtserialport,chardev=devvs
2,name=vs2,id=vs2 -chardev socket,id=devvs3,path=/tmp/virtio_port-vs3-20121217-121007-bptwZQCu,server,nowait -device virtserialport,chardev=devvs3,name=vs3,id=vs3 -char
dev socket,id=devvs4,path=/tmp/virtio_port-vs4-20121217-121007-bptwZQCu,server,nowait -device virtserialport,chardev=devvs4,name=vs4,id=vs4 -chardev socket,id=seabioslo
g_id_20121217-121007-bptwZQCu,path=/tmp/seabios-20121217-121007-bptwZQCu,server,nowait -device isa-debugcon,chardev=seabioslog_id_20121217-121007-bptwZQCu,iobase=0x402 
-device ich9-usb-uhci1,id=usb1 -drive file='/home/lmr/Code/virt-test.git/shared/data/images/jeos-17-64.qcow2',if=none,id=virtio0 -device virtio-blk-pci,drive=virtio0,bo
otindex=1 -device virtio-net-pci,netdev=id8G6KnY,mac='9a:46:47:48:49:4a',id='idFlU8hb' -netdev user,id=id8G6KnY,hostfwd=tcp::5000-:22 -m 512 -smp 2,cores=1,threads=1,so
ckets=2 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -vga std -rtc base=utc,clock=host,driftfix=none -boot order=cdn,once=c,menu=off -enable-kvm
12:10:21 INFO | [qemu output] (Process terminated with status 134)

So, this problem is quite reproducible, if you wish to use the test suite to reproduce it so you can work on it, please git clone what's specified in

https://github.com/autotest/virt-test

The README.rst file at the top of the suite contains info on how to bootstrap the tests.
Comment 17 Paolo Bonzini 2012-12-17 11:39:12 EST
FWIW, I tried just these two commands

device_add virtio-serial-pci,id=virtio_serial_pci1
device_del virtio_serial_pci1

with QEMU origin/master and it worked.  I'll try autotest tomorrow.
Comment 18 Lucas Meneghel Rodrigues 2012-12-17 11:53:53 EST
Ok Paolo, indeed, I just updated my local qemu tree and ran the test against origin master, the test passed:

$ ./run -t kvm --tests virtio_console.spread_linear.unspecifiable.hotplug_virtio_pci --qemu-bin /path/to/qemu/x86_64-softmmu/qemu-system-x86_64 
SETUP: PASS (1.60 s)
DATA DIR: /path/to/virt_test/
DEBUG LOG: /path/to/Code/virt-test.git/logs/run-2012-12-17-14.50.24/debug.log
TESTS: 1
(1/1) virtio_console.spread_linear.unspecifiable.hotplug_virtio_pci: PASS (48.04 s)

So this issue is fixed in origin/master, pending a backport to the 1.2 series, I suppose.
Comment 19 Fedora Update System 2013-01-11 18:55:00 EST
qemu-1.2.2-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.