Bug 994877

Summary: fail to shutdown RHEL7.0 guest in destination after migration
Product: Red Hat Enterprise Linux 7 Reporter: Jun Li <juli>
Component: qemu-kvmAssignee: Gerd Hoffmann <kraxel>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: medium    
Version: 7.0CC: acathrow, hhuang, juli, juzhang, michen, qzhang, rhod, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-05-23 08:53:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jun Li 2013-08-08 07:40:53 UTC
Description of problem:
After migration, run "halt -p" inside the guest, guest can't be shutdowned. Even 10 minites later, guest can not be shutdowned yet.

Version-Release number of selected component (if applicable):
qemu-kvm-1.5.2-2.el7.x86_64
3.10.0-4.el7.x86_64

How reproducible:
100% 

Steps to Reproduce:
1.Boot two RHEL7.0 guests on two hosts and do migration.
2.After migration, run "halt -p" command inside guest
3.

Actual results:
guest console output:
# nc -U /tmp/ttyS0
[root@localhost ~]# [  223.549400] systemd[1]: bus_connection_get_unix_fd failed Broken pipe
[  233.842033] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 3 5 6 7} (detected by 0, t=60002 jiffies, g=831, c=830, q=22782)
[  346.959056] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 3 5 6 7} (detected by 0, t=60002 jiffies, g=848, c=847, q=12051)
[  423.757055] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 3 4 5 6 7} (detected by 0, t=60002 jiffies, g=855, c=854, q=12021)
[  487.746056] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 3 4 5 6 7} (detected by 2, t=60003 jiffies, g=857, c=856, q=12021)
[  551.734058] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 3 4 5 6 7} (detected by 2, t=60003 jiffies, g=860, c=859, q=12337)
[  622.142055] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 3 4 5 6 7} (detected by 0, t=60002 jiffies, g=864, c=863, q=12337)
----------------------------

qeu-kvm process status:
# ps -ef |grep qemu-kvm
# top -p 21400
top - 11:45:23 up 2 days,  1:30,  5 users,  load average: 0.01, 0.04, 0.05
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni, 99.6 id,  0.2 wa,  0.2 hi,  0.0 si,  0.0 st
KiB Mem:   7594624 total,  3856348 used,  3738276 free,    86636 buffers
KiB Swap:  8142844 total,   173196 used,  7969648 free,  2604348 cached

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND    
21400 root      20   0 5788964 533136   7996 S 8.988 7.020   1:15.61  qemu-kvm

---------------------------------

# kvm_stat
kvm statistics

 kvm_exit                                     31422    2033
 kvm_entry                                    31422    2033
 kvm_msr                                      15420    1006
 kvm_inj_virq                                 15325    1006
 kvm_eoi                                      15324    1006
 kvm_apic_accept_irq                          15324    1006
 kvm_pv_eoi                                   15200    1004
 kvm_fpu                                        258	 16
 kvm_pio                                        249	 12
 kvm_userspace_exit                             249	 12
 kvm_emulate_insn                               191	  9
 kvm_set_irq                                     67	  3
 kvm_pic_set_irq                                 67	  3
 kvm_ioapic_set_irq                              67	  3
 kvm_msi_set_irq                                 31	  2
 kvm_apic                                       126	  1
 kvm_ack_irq                                     16	  1
 kvm_exit(HLT)                                   33	  0
 kvm_exit(MSR_WRITE)                             33	  0
 kvm_exit(EXCEPTION_NMI)                          1	  0


Expected results:
After migration, guest can be shutdowned as normal.

Additional info:

Comment 4 zhonglinzhang 2013-12-20 10:12:59 UTC
Retest this issue with kernel: 3.10.0-61.el7.x86_64 and qemu_kvm: qemu-kvm-1.5.3-30.el7.x86_64

Steps:
1. Boot up a RHEL7.0 guest in source host and listening with "incoming" in des host:

2. Migrate from src host to des host:

3. After migration, inside guest run "halt -p", can shutdown the guest.

Comment 8 Gerd Hoffmann 2014-05-23 08:53:53 UTC
Given that the original report says 100% reproduce rate and both attempts to reproduce (comments 4+5) failed, with more recent versions than the original report, I suspect this one is simply fixed meanwhile.

Should it show up again please reopen the bug.