Bug 1221425

Summary: qemu crash when hot-plug a memory device
Product: Red Hat Enterprise Linux 7 Reporter: Luyao Huang <lhuang>
Component: qemu-kvm-rhevAssignee: Igor Mammedov <imammedo>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 7.2CC: drjones, dyuan, hhuang, honzhang, huding, juzhang, mzhan, virt-maint, xfu, xwei
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.3.0-3.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-04 16:41:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Luyao Huang 2015-05-14 01:35:58 UTC
Description of problem:
qemu crash when hot-plug a memory device when set maxmem with 9765625KiB

Version-Release number of selected component (if applicable):
libvirt-1.2.15-2.el7.x86_64
qemu-kvm-rhev-2.3.0-1.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.
# virsh dumpxml rhel7.0
...
  <maxMemory slots='16' unit='KiB'>9765625</maxMemory>
  <memory unit='KiB'>1024000</memory>
  <currentMemory unit='KiB'>1024000</currentMemory>
...
     <numa>
      <cell id='0' cpus='0-1' memory='512000' unit='KiB'/>
      <cell id='1' cpus='2-3' memory='512000' unit='KiB'/>
    </numa>
...

2. attach a memory device
# cat memdevice.xml 
    <memory model='dimm'>
      <target>
        <size unit='m'>500</size>
        <node>1</node>
      </target>
    </memory>




3.
# virsh attach-device rhel7.0 memdevice.xml
error: Failed to attach device from memdevice.xml
error: Unable to read from monitor: Connection reset by peer


Actual results:
qemu crash when hot-plug a memory device

Expected results:
no crash

Additional info:
cannot reproduce when set maxmem to 2560000

vm log:

ERROR:hw/mem/pc-dimm.c:214:pc_dimm_get_free_addr: assertion failed: (QEMU_ALIGN_UP(address_space_size, align) == address_space_size)


vm qemu CLI:

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name rhel7.0 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Opteron_G5 -m size=1024000k,slots=16,maxmem=9765888k -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -numa node,nodeid=0,cpus=0-1,mem=500 -numa node,nodeid=1,cpus=2-3,mem=500 -uuid 881f3b5b-210f-49a4-b689-d22174642f25 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/rhel7.0.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/fs/r7_ext4.raw,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:af:19:fb,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on
char device redirected to /dev/pts/7 (label charserial0)

backtrace:

Program received signal SIGABRT, Aborted.
0x00007f6b0178d5d7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56	  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) bt
#0  0x00007f6b0178d5d7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007f6b0178ecc8 in __GI_abort () at abort.c:90
#2  0x00007f6b073468c5 in g_assertion_message (domain=domain@entry=0x0, file=file@entry=0x7f6b091b7658 "hw/mem/pc-dimm.c", line=line@entry=214, 
    func=func@entry=0x7f6b091b7a40 <__FUNCTION__.25603> "pc_dimm_get_free_addr", message=message@entry=0x7f6b0be7de20 "assertion failed: (QEMU_ALIGN_UP(address_space_size, align) == address_space_size)")
    at gtestutils.c:2291
#3  0x00007f6b0734695a in g_assertion_message_expr (domain=domain@entry=0x0, file=file@entry=0x7f6b091b7658 "hw/mem/pc-dimm.c", line=line@entry=214, 
    func=func@entry=0x7f6b091b7a40 <__FUNCTION__.25603> "pc_dimm_get_free_addr", expr=expr@entry=0x7f6b091b78f0 "QEMU_ALIGN_UP(address_space_size, align) == address_space_size") at gtestutils.c:2306
#4  0x00007f6b0906edca in pc_dimm_get_free_addr (address_space_start=<optimized out>, address_space_size=<optimized out>, hint=hint@entry=0x0, align=align@entry=2097152, size=size@entry=524288000, 
    errp=errp@entry=0x7fff19d14728) at hw/mem/pc-dimm.c:214
#5  0x00007f6b08f794c8 in pc_dimm_plug (errp=0x7fff19d14780, dev=0x7f6b0bb63700, hotplug_dev=<optimized out>) at /usr/src/debug/qemu-2.3.0/hw/i386/pc.c:1617
#6  pc_machine_device_plug_cb (hotplug_dev=<optimized out>, dev=0x7f6b0bb63700, errp=0x7fff19d14780) at /usr/src/debug/qemu-2.3.0/hw/i386/pc.c:1715
#7  0x00007f6b0903a334 in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fff19d148b8) at hw/core/qdev.c:1069
#8  0x00007f6b090c674e in property_set_bool (obj=0x7f6b0bb63700, v=<optimized out>, opaque=0x7f6b0b7f0f10, name=<optimized out>, errp=0x7fff19d148b8) at qom/object.c:1514
#9  0x00007f6b090c92d7 in object_property_set_qobject (obj=obj@entry=0x7f6b0bb63700, value=value@entry=0x7f6b0bb748a0, name=name@entry=0x7f6b091af97d "realized", errp=errp@entry=0x7fff19d148b8)
    at qom/qom-qobject.c:24
#10 0x00007f6b090c7d60 in object_property_set_bool (obj=obj@entry=0x7f6b0bb63700, value=value@entry=true, name=name@entry=0x7f6b091af97d "realized", errp=errp@entry=0x7fff19d148b8) at qom/object.c:905
#11 0x00007f6b08fe9404 in qdev_device_add (opts=opts@entry=0x7f6b0b4123d0) at qdev-monitor.c:574
#12 0x00007f6b08fe982a in do_device_add (mon=<optimized out>, qdict=<optimized out>, ret_data=<optimized out>) at qdev-monitor.c:754
#13 0x00007f6b08f33551 in qmp_call_cmd (cmd=<optimized out>, params=0x7f6b0c0a9c10, mon=0x7f6b0ac57d60) at /usr/src/debug/qemu-2.3.0/monitor.c:5047
#14 handle_qmp_command (parser=<optimized out>, tokens=<optimized out>) at /usr/src/debug/qemu-2.3.0/monitor.c:5109
#15 0x00007f6b0916c692 in json_message_process_token (lexer=0x7f6b0ac57e50, token=0x7f6b0bb74e90, type=JSON_OPERATOR, x=119, y=97) at qobject/json-streamer.c:87
#16 0x00007f6b0917ea7f in json_lexer_feed_char (lexer=lexer@entry=0x7f6b0ac57e50, ch=<optimized out>, flush=flush@entry=false) at qobject/json-lexer.c:303
#17 0x00007f6b0917eb4e in json_lexer_feed (lexer=0x7f6b0ac57e50, buffer=<optimized out>, size=<optimized out>) at qobject/json-lexer.c:356
#18 0x00007f6b0916c829 in json_message_parser_feed (parser=<optimized out>, buffer=<optimized out>, size=<optimized out>) at qobject/json-streamer.c:110
#19 0x00007f6b08f3190f in monitor_control_read (opaque=<optimized out>, buf=<optimized out>, size=<optimized out>) at /usr/src/debug/qemu-2.3.0/monitor.c:5130
#20 0x00007f6b08fed0f0 in qemu_chr_be_write (len=<optimized out>, buf=0x7fff19d14b10 "}K\321\031\377\177", s=0x7f6b0ac482f0) at qemu-char.c:305
#21 tcp_chr_read (chan=<optimized out>, cond=<optimized out>, opaque=0x7f6b0ac482f0) at qemu-char.c:2870
#22 0x00007f6b073219ba in g_main_dispatch (context=0x7f6b0ac3ac30) at gmain.c:3061
#23 g_main_context_dispatch (context=context@entry=0x7f6b0ac3ac30) at gmain.c:3660
#24 0x00007f6b091020b8 in glib_pollfds_poll () at main-loop.c:200
#25 os_host_main_loop_wait (timeout=<optimized out>) at main-loop.c:245
#26 main_loop_wait (nonblocking=<optimized out>) at main-loop.c:494
#27 0x00007f6b08f0518e in main_loop () at vl.c:1798
#28 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4373

Comment 1 Igor Mammedov 2015-06-12 14:52:35 UTC
Backported upstream commit: b5d3b039
"pc-dimm: don't assert if pc-dimm alignment != hotpluggable mem range size"

Comment 2 Miroslav Rezanina 2015-06-19 11:08:30 UTC
Fix included in qemu-kvm-rhev-2.3.0-3.el7

Comment 3 Xiaoqing Wei 2015-07-02 10:12:47 UTC
per vm xml in https://bugzilla.redhat.com/show_bug.cgi?id=1221425#c0

======================= fixed

# virsh attach-device rhel7.0-ovmf-q35 memdevice.xml 
Device attached successfully

[root@dhcp-11-50 ~]# echo $?
0
[root@dhcp-11-50 qemu-kvm-rhev7]# rpm -q qemu-img-rhev
qemu-img-rhev-2.3.0-6.el7.x86_64


---- in guest:
[root@localhost ~]# dmesg | tail -5
[   22.689781] systemd-journald[486]: Received request to flush runtime journal from PID 1
[   27.594335] ACPI: \_SB_.MP00: ACPI_NOTIFY_DEVICE_CHECK event
[   27.594873] init_memory_mapping: [mem 0x100000000-0x11f3fffff]
[   27.597123]  [mem 0x100000000-0x11f3fffff] page 2M
[   27.608114]  [ffffea0004000000-ffffea00043fffff] PMD -> [ffff880034400000-ffff8800347fffff] on node 1

======================= reproduced 
[root@dhcp-11-50 ~]# virsh start rhel7.0-ovmf-q35
Domain rhel7.0-ovmf-q35 started

[root@dhcp-11-50 ~]# virsh attach-device rhel7.0-ovmf-q35 memdevice.xml 
error: Failed to attach device from memdevice.xml
error: Unable to read from monitor: Connection reset by peer

[root@dhcp-11-50 ~]# rpm -q qemu-kvm-rhev
qemu-kvm-rhev-2.3.0-2.el7.x86_64















moving to VERIFIED.

Comment 5 errata-xmlrpc 2015-12-04 16:41:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html