Bug 867787

Summary: Do S3 will comeback(wakup) automatically after migration and fail to shutdown guest
Product: Red Hat Enterprise Linux 6 Reporter: Sibiao Luo <sluo>
Component: qemu-kvmAssignee: Hai Huang <hhuang>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.4CC: acathrow, amit.shah, areis, bsarathy, chayang, flang, juzhang, kraxel, mdeng, michen, mkenneth, quintela, qzhang, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 908651 915139 915140 1074901 (view as bug list) Environment:
Last Closed: 2014-06-05 22:16:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 761491, 912287, 915139, 915140, 1074901, 1074906    
Attachments:
Description Flags
guest will not quit after shutdown and stay at ’Power down‘ none

Description Sibiao Luo 2012-10-18 09:56:31 UTC
Description of problem:
boot a guest in src and do S3 in guest, then migrate from src to dest, and do S3 in guest, it will comeback(wakup) automatically after migration without clicking ps/2 mouse/keyboards, or send "system_wakeup" qemu command. and then shutdown guest, it will fail to shutdown successfully, guest will stay at 'Power down'.

Version-Release number of selected component (if applicable):
host info:
# uname -r && rpm -q qemu-kvm
2.6.32-331.el6.x86_64
qemu-kvm-0.12.1.2-2.327.el6.x86_64
# rpm -qa | grep seabios
seabios-0.6.1.2-25.el6.x86_64
# rpm -qa | grep spice-server
spice-server-0.12.0-1.el6.x86_64
guest info:
# uname -r
2.6.32-331.el6.x86_64

How reproducible:
almost every time

Steps to Reproduce:
1.Sync the src and dest host time with clock.redhat.com 
2.boot a guest in src and sync guest time.
eg:(src)# /usr/libexec/qemu-kvm -M rhel6.4.0 -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -usb -device usb-tablet,id=input0 -name sluo_migration -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/home/mnt/RHEL-Server-6.3-64-sluo.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,id=hostnet0,vhost=on,fd=6 6<>/dev/tap6 -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=08:2E:5F:0A:0D:B1,bus=pci.0,addr=0x5 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5931,disable-ticketing,seamless-migration=off -vga qxl -global qxl-vga.vram_size=67108864 -device intel-hda,id=sound0,bus=pci.0,addr=0x6 -device hda-duplex -device usb-ehci,id=ehci,addr=0x7 -chardev spicevmc,name=usbredir,id=usbredirchardev1 -device usb-redir,chardev=usbredirchardev1,id=usbredirdev1,bus=ehci.0,debug=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -nodefaults -serial unix:/tmp/ttyS0,server,nowait -qmp tcp:0:4444,server,nowait -boot menu=on -monitor stdio -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0
  (dest)...-incoming tcp:0:5888,server,nowait
3.do S3 in guest.
# pm-suspend
4.migrate from src to dest.
5.do S3 in guest.
6.shutdown guest.

Actual results:

1.after step 5, guest will comeback(wakup) automatically after migration without clicking ps/2 mouse/keyboards, or send "system_wakeup" qemu command.

2.after step 6,
fail to shutdown successfully, guest will stay at 'Power down'. I will attach the detail log later.

Expected results:
after step 5, guest should not comeback(wakup) automatically, it can wakup by clicking ps/2 mouse/keyboards, or send "system_wakeup" qemu command.

after step 6,
shutdown geust successfully.

Additional info:

Comment 1 Sibiao Luo 2012-10-18 09:59:07 UTC
(In reply to comment #0)
> 
> Actual results:
> 
> 2.after step 6,
> fail to shutdown successfully, guest will stay at 'Power down'. I will
> attach the detail log later.
> 
# nc -U /tmp/ttyS0 
Stopping certmonger: [  OK  ]
Shutting down Avahi daemon: [  OK  ]
Stopping rhsmcertd[  OK  ]
Stopping atd: [  OK  ]
Stopping cups: [  OK  ]
Stopping abrt daemon: [  OK  ]
Stopping sshd: [  OK  ]
Shutting down postfix: [  OK  ]
Stopping spice-vdagentd: [  OK  ]
Stopping crond: [  OK  ]
Stopping automount: [  OK  ]
Stopping acpi daemon: [  OK  ]
Stopping HAL daemon: [  OK  ]
Stopping NetworkManager daemon: [  OK  ]
Stopping system message bus: [  OK  ]
Stopping rpcbind: [  OK  ]
Shutting down system logger: [  OK  ]
Shutting down loopback interface:  [  OK  ]
ip6tables: Flushing firewall rules: [  OK  ]
ip6tables: Setting chains to policy ACCEPT: filter [  OK  ]
ip6tables: Unloading modules: [  OK  ]
iptables: Flushing firewall rules: [  OK  ]
iptables: Setting chains to policy ACCEPT: filter [  OK  ]
iptables: Unloading modules: [  OK  ]
Stopping monitoring for VG VolGroup:   2 logical volume(s) in volume group "VolGroup" unmonitored
[  OK  ]
Sending all processes the TERM signal... [  OK  ]
Sending all processes the KILL signal... [  OK  ]
Saving random seed:  [  OK  ]
Syncing hardware clock to system time [  OK  ]
Turning off swap:  [  OK  ]
Turning off quotas:  [  OK  ]
Unmounting pipe file systems:  [  OK  ]
Unmounting file systems:  [  OK  ]
init: Re-executing /sbin/init
Halting system...
ACPI: Preparing to enter system sleep state S5
Disabling non-boot CPUs ...
SMP alternatives: switching to UP code
Power down.

Comment 2 Sibiao Luo 2012-10-18 10:00:34 UTC
My two host cpu info:
processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping	: 7
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
bogomips	: 6782.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

Comment 3 Sibiao Luo 2012-10-18 10:03:56 UTC
(In reply to comment #0)
> 
> Actual results:
> 
> 2.after step 6,
> fail to shutdown successfully, guest will stay at 'Power down'. I will
> attach the detail log later.
> 
I wait more than 15 min, it stay there. and at this time, i check the status is running, and the qemu-kvm CPU consumed as following:
(qemu) info status 
VM status: running

# top
top - 17:01:21 up  5:46,  4 users,  load average: 0.05, 0.03, 0.00
Tasks: 362 total,   1 running, 361 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8017644k total,  1659884k used,  6357760k free,    31820k buffers
Swap:  8159224k total,        0k used,  8159224k free,   264816k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                        
 7202 root      20   0 5129m 888m 6048 S  3.0 11.3   0:39.32 qemu-kvm                                                                                                       
 2058 root      20   0     0    0    0 S  0.3  0.0   0:01.51 kondemand/0                                                                                                    
 2060 root      20   0     0    0    0 S  0.3  0.0   0:00.99 kondemand/2                                                                                                    
 7204 root      20   0     0    0    0 S  0.3  0.0   0:00.04 vhost-7202 
...

Comment 5 Sibiao Luo 2012-10-18 10:17:27 UTC
(In reply to comment #0)
> Description of problem:
> boot a guest in src and do S3 in guest, then migrate from src to dest, and
> do S3 in guest, it will comeback(wakup) automatically after migration
> without clicking ps/2 mouse/keyboards, or send "system_wakeup" qemu command.
> and then shutdown guest, it will fail to shutdown successfully, guest will
> stay at 'Power down'.
> 
> Actual results:
> 
> 1.after step 5, guest will comeback(wakup) automatically after migration
> without clicking ps/2 mouse/keyboards, or send "system_wakeup" qemu command.
> 
> 2.after step 6,
> fail to shutdown successfully, guest will stay at 'Power down'. I will
> attach the detail log later.
> 
If i use '-cpu SandyBridge,-kvmclock' instead of '-cpu SandyBridge' in qemu-kvm command line, it still can hit this issue. 
guest will comeback(wakup) automatically after migration without clicking ps/2 mouse/keyboards, or send "system_wakeup" qemu command and fail to shutdown successfully, guest will stay at 'Power down'. 
guest kernel log as following:
...
Halting system...
ACPI: Preparing to enter system sleep state S5
Disabling non-boot CPUs ...
Broke affinity for irq 4
Broke affinity for irq 27
SMP alternatives: switching to UP code
Power down.

Comment 6 Sibiao Luo 2012-10-18 11:43:32 UTC
Created attachment 629328 [details]
guest will not quit after shutdown and stay at ’Power down‘

Comment 8 Min Deng 2012-10-23 06:40:40 UTC
(In reply to comment #6)
> Created attachment 629328 [details]
> guest will not quit after shutdown and stay at ’Power down‘

Hi all,
 The following steps also can help to reproduce the issue on windows guest(2k8-R2)
1.boot up guest 
  /usr/libexec/qemu-kvm -m 2G -smp 2 -cpu host,+x2apic -usb -device usb-tablet -drive file=win2k8-R2.qcow2,format=qcow2,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device virtio-blk-pci,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,mac=10:10:1a:75:59:03,bus=pci.0,addr=0x4 -uuid `uuidgen` -rtc base=localtime -no-kvm-pit-reinjection -monitor stdio -name win2k8-64-fun -device virtio-balloon-pci,addr=0x6,bus=pci.0,id=balloon1 -chardev socket,id=111a,path=/tmp/monitor-win2k8-64-balloon-30,server,nowait -mon chardev=111a,mode=readline -cdrom /home/en_windows_server_2008_r2_standard_enterprise_datacenter_and_web_with_sp1_x64_dvd_617601.iso -drive file=disk1.qcow2,format=qcow2,if=none,id=drive-virtio1,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-virtio1,id=virtio-blk-pci1 -drive file=disk2.qcow2,format=qcow2,if=none,id=drive-virtio2,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-virtio2,id=virtio-blk-pci2 -drive file=disk3.qcow2,format=qcow2,if=none,id=drive-virtio3,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-virtio3,id=virtio-blk-pci3 -drive file=disk4.qcow2,format=qcow2,if=none,id=drive-virtio4,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-virtio4,id=virtio-blk-pci4 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -spice disable-ticketing,port=5931 -vga qxl
2.running iozone in the guest 
3.do S3 and resume it back after a while 
4.Shut down the guest
Actual results,the guest stays at "shutting down" status for a long time.
Expected results,the guest can shut down successfully.

Thanks,
Min

Comment 9 Amit Shah 2013-02-21 11:47:04 UTC
There are two problems here:

1. Run state is not checked on destination machine -- if incoming VM is in suspended state, dest. should continue to be in suspended state.

2. ACPI power off doesn't work.

We need two separate bugs for these two issues.

Comment 10 Sibiao Luo 2013-02-25 03:32:46 UTC
(In reply to comment #9)
> There are two problems here:
> 
> 1. Run state is not checked on destination machine -- if incoming VM is in
> suspended state, dest. should continue to be in suspended state.
bug 915139
> 2. ACPI power off doesn't work.
bug 915140
> We need two separate bugs for these two issues.

Thanks for your kindly checking. 
I has separated them for bug 915140 and bug 915139.

Comment 12 Orit Wasserman 2013-05-07 15:18:04 UTC
Fixing this issue will require a change to migration protocol.
postponed due to capacity.

Comment 15 Ademar Reis 2014-06-05 22:16:00 UTC
S3/S4 support is tech-preview in RHEL6 and it'll be promoted to fully supported
at some point, but only in RHEL7.

Therefore we're closing all S3/S4 related bugs in RHEL6. New bugs will be
considered only if they're regressions or break some important use-case or
certification.

RHEL7 is being more extensively tested and effort from QE is underway in
certifying that this particular bug is not present there.

Please reopen with a justification if you believe this bug should not be
closed. We'll consider them on a case-by-case basis following a best effort
approach.


Thank you.