Hide Forgot
Created attachment 1208313 [details] ppc64le Description of problem: Boot a guest(ppc64 or ppc64le),execute "balloon 1024" command,then shutdown guest,but the qemu monitor and vncviewer can't quit automatically Version-Release number of selected component (if applicable): host:3.10.0-512.el7.ppc64le guest: 3.10.0-512.el7.ppc64le 3.10.0-512.el7.ppc64 How reproducible: 4/5 Steps to Reproduce: 1.boot a guest with command"-device virtio-balloon-pci,disable-legacy=on,disable-modern=off \",open vncviewer; 2.check the memory in guest with command"head -n 3 /proc/meminfo"; 3.execute command "balloon 1024"in qemu monitor; 4.check the memory in guest against; 5.shutdown guest with command "shutdown -h now"in guest; Actual results: 4.the memory result changed right; 5.the guest was shutdown, while the qemu monitor and vncviewer quit automatically Expected results: 4.the memory result changed right; 5.the guest was shutdown, but the qemu monitor and vncviewer don't quit automatically Additional info: 1.How reproducible: ppc64le:4/5 ppc64:4/5 x86_64:0/10 2.the full qume command is: /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -sandbox off \ -nodefaults \ -machine pseries-rhel7.3.0 \ -vga std \ -device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=03 \ -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x4 \ -chardev socket,id=devorg.qemu.guest_agent.0,path=/tmp/virtio_port-org.qemu.guest_agent.0-20160516-164929-dHQ00mMM,server,nowait \ -device virtserialport,chardev=devorg.qemu.guest_agent.0,name=org.qemu.guest_agent.0,id=org.qemu.guest_agent.0,bus=virtio_serial_pci0.0 \ -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pci.0 \ -drive file=/root/RHEL7.3.qcow2,if=none,id=blk1 \ -device virtio-blk-pci,scsi=off,drive=blk1,id=blk-disk1,bootindex=1 \ -drive id=drive_cd1,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/root/RHEL7.3.iso \ -device scsi-cd,id=cd1,drive=drive_cd1,bootindex=2 \ -device virtio-net-pci,mac=9a:7b:7c:7d:7e:71,id=idtlLxAk,vectors=4,netdev=idlkwV8e,bus=pci.0,addr=05 \ -netdev tap,id=idlkwV8e,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \ -device virtio-balloon-pci,disable-legacy=on,disable-modern=off \ -m 4G \ -smp 8 \ -cpu host \ -device usb-kbd \ -device usb-mouse \ -qmp tcp:0:8881,server,nowait \ -vnc :1 \ -msg timestamp=on \ -rtc base=localtime,clock=vm,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -monitor stdio \ -enable-kvm
(In reply to xianwang from comment #0) > Created attachment 1208313 [details] > ppc64le > > Description of problem: > Boot a guest(ppc64 or ppc64le),execute "balloon 1024" command,then shutdown > guest,but the qemu monitor and vncviewer can't quit automatically > > Version-Release number of selected component (if applicable): > host:3.10.0-512.el7.ppc64le > guest: > 3.10.0-512.el7.ppc64le > 3.10.0-512.el7.ppc64 > > How reproducible: > 4/5 > > Steps to Reproduce: > 1.boot a guest with command"-device > virtio-balloon-pci,disable-legacy=on,disable-modern=off \",open vncviewer; > 2.check the memory in guest with command"head -n 3 /proc/meminfo"; > 3.execute command "balloon 1024"in qemu monitor; > 4.check the memory in guest against; > 5.shutdown guest with command "shutdown -h now"in guest; > > Actual results: > 4.the memory result changed right; > 5.the guest was shutdown, while the qemu monitor and vncviewer quit > automatically > correction:the "Actual results" should be as follows: > Actual results: > 4.the memory result changed right; > 5.the guest was shutdown, but the qemu monitor and vncviewer don't quit > automatically > Expected results: > 4.the memory result changed right; > 5.the guest was shutdown, but the qemu monitor and vncviewer don't quit > automatically correction:the "Expected results"should be as follows: > Expected results: > 4.the memory result changed right; > 5.the guest was shutdown, while the qemu monitor and vncviewer quit > automatically > > Additional info: > 1.How reproducible: > ppc64le:4/5 > ppc64:4/5 > x86_64:0/10 > 2.the full qume command is: > > /usr/libexec/qemu-kvm \ > -name 'avocado-vt-vm1' \ > -sandbox off \ > -nodefaults \ > -machine pseries-rhel7.3.0 \ > -vga std \ > -device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=03 \ > -device virtio-scsi-pci,id=scsi1,bus=pci.0,addr=0x4 \ > -chardev > socket,id=devorg.qemu.guest_agent.0,path=/tmp/virtio_port-org.qemu. > guest_agent.0-20160516-164929-dHQ00mMM,server,nowait \ > -device > virtserialport,chardev=devorg.qemu.guest_agent.0,name=org.qemu.guest_agent.0, > id=org.qemu.guest_agent.0,bus=virtio_serial_pci0.0 \ > -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pci.0 \ > -drive file=/root/RHEL7.3.qcow2,if=none,id=blk1 \ > -device virtio-blk-pci,scsi=off,drive=blk1,id=blk-disk1,bootindex=1 \ > -drive > id=drive_cd1,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/ > root/RHEL7.3.iso \ > -device scsi-cd,id=cd1,drive=drive_cd1,bootindex=2 \ > -device > virtio-net-pci,mac=9a:7b:7c:7d:7e:71,id=idtlLxAk,vectors=4,netdev=idlkwV8e, > bus=pci.0,addr=05 \ > -netdev > tap,id=idlkwV8e,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \ > -device virtio-balloon-pci,disable-legacy=on,disable-modern=off \ > -m 4G \ > -smp 8 \ > -cpu host \ > -device usb-kbd \ > -device usb-mouse \ > -qmp tcp:0:8881,server,nowait \ > -vnc :1 \ > -msg timestamp=on \ > -rtc base=localtime,clock=vm,driftfix=slew \ > -boot order=cdn,once=c,menu=off,strict=off \ > -monitor stdio \ > -enable-kvm
Created attachment 1208454 [details] guestppc64le
the shutdown log is as follows: Stopping D-Bus System Message Bus... Stopping Authorization Manager... [ OK ] Stopped Authorization Manager. [ OK ] Stopped D-Bus System Message Bus. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Closed RPCbind Server Activation Socket. [ OK ] Closed Open-iSCSI iscsiuio Socket. [ OK ] Closed CUPS Printing Service Sockets. [ OK ] Closed Avahi mDNS/DNS-SD Stack Activation Socket. [ OK ] Closed Open-iSCSI iscsid Socket. [ OK ] Stopped target Slices. [ OK ] Removed slice User and Session Slice. [ OK ] Stopped target Paths. [ OK ] Closed D-Bus System Message Bus Socket. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. Stopping Apply Kernel Variables... [ OK ] Stopped Setup Virtual Console. Stopping Setup Virtual Console... Stopping Load/Save Random Seed... Stopping Update UTMP about System Boot/Shutdown... [ OK ] Stopped target Encrypted Volumes. [ OK ] Stopped target Swap. [ OK ] Stopped Load/Save Random Seed. [ OK ] Stopped Update UTMP about System Boot/Shutdown. Stopping Security Auditing Service... [ OK ] Stopped Security Auditing Service. [ OK ] Stopped Create Volatile Files and Directories. Stopping Create Volatile Files and Directories... [ OK ] Stopped Import network configuration from initramfs. Stopping Import network configuration from initramfs... [ OK ] Stopped target Local File Systems. [ OK ] Stopped Configure read-only root support. Stopping Configure read-only root support... Unmounting /boot... Unmounting /run/user/42... Unmounting Configuration File System... [ OK ] Unmounted /run/user/42. [ OK ] Unmounted Configuration File System. [ OK ] Unmounted /boot. [ OK ] Stopped target Local File Systems (Pre). Stopping Monitoring of LVM2 mirrors... dmeventd or progress polling... [ OK ] Stopped Create Static Device Nodes in /dev. Stopping Create Static Device Nodes in /dev... [ OK ] Stopped Remount Root and Kernel File Systems. Stopping Remount Root and Kernel File Systems... [ OK ] Reached target Shutdown. ............hang.........
check the guest status in qemu monitor with command: info status, the result is:"VM status: running"
I tried to reproduce this problem and ran my guest > 5 times, but so far, I did not hit this problem once. Could you please provide some more information: 1) Which version of qemu-kvm are you using on the host? 2) How big is the hard disk image of the guest (i.e. parameters used for "qemu-img create")? 3) Which version of RHEL is installed in the guest? Final RHEL 7.3? 4) Can you reproduce the problem also without VNC? E.g. run QEMU without "-vga std" and without "-monitor stdio", but use "-serial mon:stdio" instead. Then, when the guest is booted, you can switch between QEMU monitor and guest serial output by pressing "CTRL-a c".
Another thing to check: Is your guest using a crashkernel? (i.e. is there something like "crashkernel=auto" configured in the kernel parameters)
OK, after some more tries, I was now able to reproduce the problem by using "balloon 800" instead of "balloon 1024". The problem seems not related to VNC, it also occurs if I start the guest in plain serial console mode, without vncviewer. After booting the kernel with the following systemd debug parameters: systemd.log_level=debug systemd.log_target=console console=hvc0 systemd.journald.forward_to_console=1 ... I get a little bit more output: [ OK ] Reached target Shutdown. Got cgroup empty notification for: /system.slice/boot.mount [ 41.402709] systemd-udevd[387]: cleanup idle workers [ 41.403398] systemd-udevd[387]: Validate module index [ 41.403715] systemd-udevd[387]: Check if link configuration needs reloading. [ 41.404047] systemd-udevd[387]: Unload module index [ 41.404371] systemd-udevd[387]: Unload module index [ 41.405298] systemd-udevd[387]: Unloaded link configuration context. [ 41.405855] systemd-udevd[387]: Unloaded link configuration context. [ 41.406739] systemd-udevd[387]: worker [2544] exit [ 41.407082] systemd-udevd[387]: worker [2544] cleaned up [ 41.407407] systemd-udevd[387]: worker [2572] exit [ 41.407727] systemd-udevd[387]: worker [2572] cleaned up and after a while, there are additionally these messages: Got notification message for unit systemd-journald.service systemd-journald.service: Got notification message from PID 365 (WATCHDOG=1) systemd-journald.service: got WATCHDOG=1 So it seems to me that systemd is somehow failing to finish the shutdown of the system when it is low of memory? Maybe something similar like https://github.com/systemd/systemd/issues/2930 ? ... anyway, since I don't know how to debug systemd issues and this is likely not a qemu bug, I'm reassigning the bug to the systemd component now for further investigation.
By the way, I can also reproduce this problem on x86 - I just got to use an even lower value for the balloon command, e.g. "balloon 500" ("head -n 1 /proc/meminfo" should then show something about 300 MB of total memory left). Looks like the problem occurs if the guest is forced into swapping this way, so that swapping is still taking place when shutdown target is reached.
*** Bug 1469981 has been marked as a duplicate of this bug. ***
*** Bug 1458866 has been marked as a duplicate of this bug. ***
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.