Attachment 494775 Details for Bug 699556 – serial output (it contains panic info)

serial output (it contains panic info)

serial-vm1.log (text/x-log), 56.67 KB, created by Amos Kong on 2011-04-26 00:13:40 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Amos Kong

Created: 2011-04-26 00:13:40 UTC

Size: 56.67 KB

patch

obsolete

>2011-04-21 10:57:56: Shutting down Avahi daemon:
>2011-04-21 10:57:56: [  OK  ]
>2011-04-21 10:57:56: Stopping rhsmcertd
>2011-04-21 10:57:56: [  OK  ]
>2011-04-21 10:57:56: Stopping atd:
>2011-04-21 10:57:57: [  OK  ]
>2011-04-21 10:57:57: Stopping abrt daemon:
>2011-04-21 10:57:57: [  OK  ]
>2011-04-21 10:57:57: Stopping sshd:
>2011-04-21 10:57:57: [  OK  ]
>2011-04-21 10:57:57: Shutting down postfix:
>2011-04-21 10:57:57: [  OK  ]
>2011-04-21 10:57:57: Stopping crond:
>2011-04-21 10:57:57: [  OK  ]
>2011-04-21 10:57:58: Stopping acpi daemon:
>2011-04-21 10:57:58: [  OK  ]
>2011-04-21 10:57:58: Stopping HAL daemon: [  OK  ]
>2011-04-21 10:57:58: Stopping NetworkManager daemon:
>2011-04-21 10:57:58: [  OK  ]
>2011-04-21 10:57:58: Killing mdmonitor:
>2011-04-21 10:57:58: [  OK  ]
>2011-04-21 10:57:58: Stopping system message bus: [  OK  ]
>2011-04-21 10:57:59: Stopping rpcbind:
>2011-04-21 10:57:59: [  OK  ]
>2011-04-21 10:57:59: Stopping auditd: type=1305 audit(1303354677.344:61): audit_pid=0 old=1218 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1
>2011-04-21 10:57:59: [  OK  ]
>2011-04-21 10:57:59: type=1400 audit(1303354677.692:62): avc:  denied  { sys_module } for  pid=1961 comm="ip" capability=16  scontext=system_u:system_r:ifconfig_t:s0 tcontext=system_u:system_r:ifconfig_t:s0 tclass=capability
>2011-04-21 10:57:59: type=1300 audit(1303354677.692:62): arch=40000003 syscall=54 success=no exit=-19 a0=4 a1=8933 a2=bfe8bd9c a3=4 items=0 ppid=1960 pid=1961 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip" exe="/sbin/ip" subj=system_u:system_r:ifconfig_t:s0 key=(null)
>2011-04-21 10:57:59: Shutting down loopback interface:
>2011-04-21 10:58:00: [  OK  ]
>2011-04-21 10:58:00: ip6tables: Flushing firewall rules: type=1325 audit(1303354678.520:63): table=filter family=10 entries=10
>2011-04-21 10:58:00: type=1300 audit(1303354678.520:63): arch=40000003 syscall=102 success=yes exit=0 a0=e a1=bfa01900 a2=5adcb0 a3=5ab974 items=0 ppid=2027 pid=2035 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/sbin/ip6tables-multi" subj=system_u:system_r:iptables_t:s0 key=(null)
>2011-04-21 10:58:00: type=1325 audit(1303354678.547:64): table=filter family=10 entries=4
>2011-04-21 10:58:00: type=1300 audit(1303354678.547:64): arch=40000003 syscall=102 success=yes exit=0 a0=e a1=bf851960 a2=358cb0 a3=356974 items=0 ppid=2027 pid=2037 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/sbin/ip6tables-multi" subj=system_u:system_r:iptables_t:s0 key=(null)
>2011-04-21 10:58:00: [  OK  ]
>2011-04-21 10:58:00: ip6tables: Setting chains to policy ACCEPT: filter type=1325 audit(1303354678.580:65): table=filter family=10 entries=4
>2011-04-21 10:58:00: type=1300 audit(1303354678.580:65): arch=40000003 syscall=102 success=yes exit=0 a0=e a1=bfee2010 a2=e2ccb0 a3=e2a974 items=0 ppid=2027 pid=2040 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/sbin/ip6tables-multi" subj=system_u:system_r:iptables_t:s0 key=(null)
>2011-04-21 10:58:00: [  OK  ]
>2011-04-21 10:58:00: ip6tables: Unloading modules:
>2011-04-21 10:58:00: [  OK  ]
>2011-04-21 10:58:01: iptables: Flushing firewall rules: [  OK  ]
>2011-04-21 10:58:01: iptables: Setting chains to policy ACCEPT: filter [  OK  ]
>2011-04-21 10:58:01: iptables: Unloading modules:
>2011-04-21 10:58:01: [  OK  ]
>2011-04-21 10:58:02: Stopping monitoring for VG VolGroup:
>2011-04-21 10:58:02:   2 logical volume(s) in volume group "VolGroup" unmonitored
>2011-04-21 10:58:02: [  OK  ]
>2011-04-21 10:58:02: Sending all processes the TERM signal...
>2011-04-21 10:58:02: [  OK  ]
>2011-04-21 10:58:04: Sending all processes the KILL signal...
>2011-04-21 10:58:04: [  OK  ]
>2011-04-21 10:58:04: Saving random seed:  [  OK  ]
>2011-04-21 10:58:04: Syncing hardware clock to system time
>2011-04-21 10:58:06: __ratelimit: 42 callbacks suppressed
>2011-04-21 10:58:06: type=1111 audit(1303354684.502:73): user pid=2171 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:hwclock_t:s0 msg='changing system time: exe="/sbin/hwclock" hostname=? addr=? terminal=? res=success'
>2011-04-21 10:58:06: [  OK  ]
>2011-04-21 10:58:06: Turning off swap:  [  OK  ]
>2011-04-21 10:58:06: Turning off quotas:  [  OK  ]
>2011-04-21 10:58:06: Unmounting pipe file systems:  [  OK  ]
>2011-04-21 10:58:06: Unmounting file systems:  [  OK  ]
>2011-04-21 10:58:06: init: Re-executing /sbin/init
>2011-04-21 10:58:07: Halting system...
>2011-04-21 10:58:07: type=1128 audit(1303354685.444:74): user pid=1802 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:initrc_t:s0 msg='init: exe="/sbin/reboot" hostname=? addr=? terminal=console res=success'
>2011-04-21 10:58:07: md: stopping all md devices.
>2011-04-21 10:58:08: ACPI: Preparing to enter system sleep state S5
>2011-04-21 10:58:08: Disabling non-boot CPUs ...
>2011-04-21 10:59:23: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 10:59:23: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 10:59:23: 
>2011-04-21 10:59:23: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 10:59:23: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 10:59:23: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 10:59:23: 
>2011-04-21 10:59:23: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 10:59:23: EIP: 0060:[<c04a14aa>] EFLAGS: 00000293 CPU: 3
>2011-04-21 10:59:23: EIP is at stop_machine_cpu_stop+0x5a/0xb0
>2011-04-21 10:59:23: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 10:59:23: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 10:59:23:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 10:59:23: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 10:59:23: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 10:59:23: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 10:59:23: Call Trace:
>2011-04-21 10:59:23:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 10:59:23:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 10:59:23:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 10:59:23:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 10:59:23:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 10:59:23:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 10:59:23:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 10:59:23:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 10:59:23:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 10:59:23: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 10:59:23: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 10:59:23: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 10:59:23: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 10:59:23:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 10:59:23: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 10:59:23: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 10:59:23: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 10:59:23: Call Trace:
>2011-04-21 10:59:23:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 10:59:23:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 10:59:23:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 10:59:23:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 10:59:23:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 10:59:23:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 10:59:23:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 10:59:23:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 10:59:23:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:00:31: BUG: soft lockup - CPU#0 stuck for 61s! [migration/0:5]
>2011-04-21 11:00:31: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:00:31: 
>2011-04-21 11:00:31: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:00:31: EIP: 0060:[<c04a14aa>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:00:31: EIP is at stop_machine_cpu_stop+0x5a/0xb0
>2011-04-21 11:00:31: BUG: soft lockup - CPU#3 stuck for 61s! [migration/3:16]
>2011-04-21 11:00:31: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:00:31: 
>2011-04-21 11:00:31: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:00:31: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:00:31: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:00:31: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:00:31: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:00:31:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:00:31: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:00:31: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:00:31: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:00:31: Call Trace:
>2011-04-21 11:00:31:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:00:31:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:00:31:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:00:31:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:00:31:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:00:31:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:00:31:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:00:31:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:00:31:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:00:31: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:00:31: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:00:31:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:00:31: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:00:31: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:00:31: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:00:31: Call Trace:
>2011-04-21 11:00:31:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:00:31:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:00:31:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:00:31:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:00:31:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:00:31:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:00:31:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:00:31:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:00:31:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:01:42: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:01:42: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:01:42: 
>2011-04-21 11:01:42: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:01:42: EIP: 0060:[<c04a14ac>] EFLAGS: 00000246 CPU: 0
>2011-04-21 11:01:42: EIP is at stop_machine_cpu_stop+0x5c/0xb0
>2011-04-21 11:01:42: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:01:42: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:01:42: 
>2011-04-21 11:01:42: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:01:42: EIP: 0060:[<c04a14a3>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:01:42: EIP is at stop_machine_cpu_stop+0x53/0xb0
>2011-04-21 11:01:42: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:01:42: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:01:42:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:01:42: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:01:42: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:01:42: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:01:42: Call Trace:
>2011-04-21 11:01:42:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:01:42:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:01:42:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:01:42:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:01:42:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:01:42:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:01:42:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:01:42:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:01:42:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:01:42: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:01:42: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:01:42:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:01:42: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:01:42: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:01:42: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:01:42: Call Trace:
>2011-04-21 11:01:42:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:01:42:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:01:42:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:01:42:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:01:42:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:01:42:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:01:42:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:01:42:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:01:42:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:02:51: BUG: soft lockup - CPU#0 stuck for 61s! [migration/0:5]
>2011-04-21 11:02:51: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:02:51: 
>2011-04-21 11:02:51: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:02:51: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:02:51: BUG: soft lockup - CPU#3 stuck for 61s! [migration/3:16]
>2011-04-21 11:02:51: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:02:51: 
>2011-04-21 11:02:51: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:02:51: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:02:51: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:02:51: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:02:51: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:02:51:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:02:51: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:02:51: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:02:51: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:02:51: Call Trace:
>2011-04-21 11:02:51:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:02:51:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:02:51:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:02:51:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:02:51:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:02:51:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:02:51:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:02:51:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:02:51:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:02:51: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:02:51: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:02:51: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:02:51:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:02:51: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:02:51: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:02:51: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:02:51: Call Trace:
>2011-04-21 11:02:51:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:02:51:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:02:51:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:02:51:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:02:51:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:02:51:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:02:51:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:02:51:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:02:51:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:04:02: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:04:02: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:04:02: 
>2011-04-21 11:04:02: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:04:02: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:04:02: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:04:02: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:04:02: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:04:02: 
>2011-04-21 11:04:02: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:04:02: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:04:02: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:04:02: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:04:02: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:04:02:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:04:02: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:04:02: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:04:02: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:04:02: Call Trace:
>2011-04-21 11:04:02:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:04:02:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:04:02:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:04:02:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:04:02:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:04:02:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:04:02:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:04:02:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:04:02:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:04:02: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:04:02: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:04:02:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:04:02: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:04:02: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:04:02: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:04:02: Call Trace:
>2011-04-21 11:04:02:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:04:02:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:04:02:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:04:02:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:04:02:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:04:02:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:04:02:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:04:02:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:04:02:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:05:14: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:05:14: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:05:14: 
>2011-04-21 11:05:14: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:05:14: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:05:14: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:05:14: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:05:14: 
>2011-04-21 11:05:14: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:05:14: EIP: 0060:[<c04a14a3>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:05:14: EIP is at stop_machine_cpu_stop+0x53/0xb0
>2011-04-21 11:05:14: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:05:14: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:05:14:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:05:14: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:05:14: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:05:14: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:05:14: Call Trace:
>2011-04-21 11:05:14:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:05:14:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:05:14:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:05:14:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:05:14:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:05:14:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:05:14:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:05:14:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:05:14:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:05:14: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:05:14: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:05:14: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:05:14:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:05:14: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:05:14: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:05:14: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:05:14: Call Trace:
>2011-04-21 11:05:14:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:05:14:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:05:14:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:05:14:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:05:14:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:05:14:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:05:14:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:05:14:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:05:14:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:06:22: BUG: soft lockup - CPU#0 stuck for 61s! [migration/0:5]
>2011-04-21 11:06:22: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:06:22: 
>2011-04-21 11:06:22: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:06:22: EIP: 0060:[<c04a14a3>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:06:22: BUG: soft lockup - CPU#3 stuck for 61s! [migration/3:16]
>2011-04-21 11:06:22: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:06:22: 
>2011-04-21 11:06:22: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:06:22: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:06:22: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:06:22: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:06:22: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:06:22:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:06:22: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:06:22: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:06:22: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:06:22: Call Trace:
>2011-04-21 11:06:22:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:06:22:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:06:22:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:06:22:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:06:22:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:06:22:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:06:22:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:06:22:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:06:22:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:06:22: EIP is at stop_machine_cpu_stop+0x53/0xb0
>2011-04-21 11:06:22: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:06:22: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:06:22:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:06:22: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:06:22: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:06:22: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:06:22: Call Trace:
>2011-04-21 11:06:22:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:06:22:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:06:22:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:06:22:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:06:22:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:06:22:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:06:22:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:06:22:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:06:22:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:07:34: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:07:34: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:07:34: 
>2011-04-21 11:07:34: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:07:34: EIP: 0060:[<c04a14aa>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:07:34: EIP is at stop_machine_cpu_stop+0x5a/0xb0
>2011-04-21 11:07:34: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:07:34: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:07:34: 
>2011-04-21 11:07:34: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:07:34: EIP: 0060:[<c04a14aa>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:07:34: EIP is at stop_machine_cpu_stop+0x5a/0xb0
>2011-04-21 11:07:34: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:07:34: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:07:34:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:07:34: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:07:34: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:07:34: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:07:34: Call Trace:
>2011-04-21 11:07:34:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:07:34:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:07:34:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:07:34:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:07:34:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:07:34:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:07:34:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:07:34:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:07:34:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:07:34: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:07:34: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:07:34:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:07:34: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:07:34: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:07:34: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:07:34: Call Trace:
>2011-04-21 11:07:34:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:07:34:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:07:34:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:07:34:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:07:34:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:07:34:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:07:34:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:07:34:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:07:34:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:08:45: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:08:45: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:08:45: 
>2011-04-21 11:08:45: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:08:45: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:08:45: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:08:45: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:08:45: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:08:45: 
>2011-04-21 11:08:45: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:08:45: EIP: 0060:[<c04a14ac>] EFLAGS: 00000246 CPU: 3
>2011-04-21 11:08:45: EIP is at stop_machine_cpu_stop+0x5c/0xb0
>2011-04-21 11:08:45: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:08:45: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:08:45:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:08:45: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:08:45: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:08:45: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:08:45: Call Trace:
>2011-04-21 11:08:45:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:08:45:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:08:45:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:08:45:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:08:45:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:08:45:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:08:45:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:08:45:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:08:45:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:08:45: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:08:45: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:08:45:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:08:45: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:08:45: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:08:45: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:08:45: Call Trace:
>2011-04-21 11:08:45:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:08:45:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:08:45:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:08:45:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:08:45:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:08:45:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:08:45:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:08:45:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:08:45:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:09:53: BUG: soft lockup - CPU#0 stuck for 61s! [migration/0:5]
>2011-04-21 11:09:53: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:09:53: 
>2011-04-21 11:09:53: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:09:53: EIP: 0060:[<c04a14a3>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:09:53: EIP is at stop_machine_cpu_stop+0x53/0xb0
>2011-04-21 11:09:53: BUG: soft lockup - CPU#3 stuck for 61s! [migration/3:16]
>2011-04-21 11:09:53: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:09:53: 
>2011-04-21 11:09:53: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:09:53: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:09:53: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:09:53: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:09:53: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:09:53:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:09:53: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:09:53: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:09:53: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:09:53: Call Trace:
>2011-04-21 11:09:53:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:09:53:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:09:53:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:09:53:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:09:53:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:09:53:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:09:53:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:09:53:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:09:53:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:09:53: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:09:53: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:09:53:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:09:53: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:09:53: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:09:54: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:09:54: Call Trace:
>2011-04-21 11:09:54:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:09:54:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:09:54:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:09:54:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:09:54:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:09:54:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:09:54:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:09:54:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:09:54:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:11:05: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:11:05: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:11:05: 
>2011-04-21 11:11:05: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:11:05: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:11:05: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:11:05: 
>2011-04-21 11:11:05: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:11:05: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:11:05: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:11:05: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:11:05: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:11:05:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:11:05: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:11:05: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:11:05: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:11:05: Call Trace:
>2011-04-21 11:11:05:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:11:05:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:11:05:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:11:05:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:11:05:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:11:05:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:11:05:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:11:05:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:11:05:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:11:05: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:11:05: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:11:05: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:11:05: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:11:05:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:11:05: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:11:05: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:11:05: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:11:05: Call Trace:
>2011-04-21 11:11:05:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:11:05:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:11:05:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:11:05:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:11:05:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:11:05:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:11:05:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:11:05:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:11:05:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:12:13: BUG: soft lockup - CPU#0 stuck for 61s! [migration/0:5]
>2011-04-21 11:12:13: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:12:13: 
>2011-04-21 11:12:13: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:12:13: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:12:13: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:12:13: BUG: soft lockup - CPU#3 stuck for 61s! [migration/3:16]
>2011-04-21 11:12:13: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:12:13: 
>2011-04-21 11:12:13: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:12:13: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:12:13: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:12:13: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:12:13: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:12:13:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:12:13: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:12:13: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:12:13: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:12:13: Call Trace:
>2011-04-21 11:12:13:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:12:13:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:12:13:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:12:13:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:12:13:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:12:13:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:12:13:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:12:13:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:12:13:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:12:13: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:12:13: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:12:13:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:12:13: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:12:13: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:12:13: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:12:13: Call Trace:
>2011-04-21 11:12:13:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:12:13:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:12:13:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:12:13:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:12:13:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:12:13:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:12:13:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:12:13:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:12:13:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:13:25: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:13:25: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:13:25: 
>2011-04-21 11:13:25: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:13:25: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:13:25: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:13:25: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:13:25: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:13:25: 
>2011-04-21 11:13:25: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:13:25: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:13:25: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:13:25: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:13:25: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:13:25:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:13:25: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:13:25: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:13:25: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:13:25: Call Trace:
>2011-04-21 11:13:25:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:13:25:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:13:25:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:13:25:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:13:25:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:13:25:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:13:25:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:13:25:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:13:25:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:13:25: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:13:25: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:13:25:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:13:25: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:13:25: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:13:25: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:13:25: Call Trace:
>2011-04-21 11:13:25:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:13:25:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:13:25:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:13:25:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:13:25:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:13:25:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:13:25:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:13:25:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:13:25:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:14:36: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:14:36: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:14:36: 
>2011-04-21 11:14:36: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:14:36: EIP: 0060:[<c04a14aa>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:14:36: EIP is at stop_machine_cpu_stop+0x5a/0xb0
>2011-04-21 11:14:36: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:14:36: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:14:36: 
>2011-04-21 11:14:36: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:14:36: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:14:37: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:14:37: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:14:37: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:14:37:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:14:37: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:14:37: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:14:37: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:14:37: Call Trace:
>2011-04-21 11:14:37:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:14:37:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:14:37:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:14:37:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:14:37:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:14:37:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:14:37:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:14:37:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:14:37:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:14:37: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:14:37: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:14:37:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:14:37: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:14:37: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:14:37: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:14:37: Call Trace:
>2011-04-21 11:14:37:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:14:37:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:14:37:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:14:37:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:14:37:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:14:37:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:14:37:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:14:37:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:14:37:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:15:45: BUG: soft lockup - CPU#0 stuck for 61s! [migration/0:5]
>2011-04-21 11:15:45: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:15:45: 
>2011-04-21 11:15:45: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:15:45: BUG: soft lockup - CPU#3 stuck for 61s! [migration/3:16]
>2011-04-21 11:15:45: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:15:45: 
>2011-04-21 11:15:45: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:15:45: EIP: 0060:[<c04a14aa>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:15:45: EIP is at stop_machine_cpu_stop+0x5a/0xb0
>2011-04-21 11:15:45: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:15:45: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:15:45:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:15:45: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:15:45: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:15:45: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:15:45: Call Trace:
>2011-04-21 11:15:45:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:15:45:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:15:45:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:15:45:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:15:45:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:15:45:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:15:45:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:15:45:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:15:45:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:15:45: EIP: 0060:[<c04a14a3>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:15:45: EIP is at stop_machine_cpu_stop+0x53/0xb0
>2011-04-21 11:15:45: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:15:45: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:15:45:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:15:45: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:15:45: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:15:45: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:15:45: Call Trace:
>2011-04-21 11:15:45:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:15:45:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:15:45:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:15:45:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:15:45:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:15:45:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:15:45:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:15:45:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:15:45:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:16:56: BUG: soft lockup - CPU#0 stuck for 63s! [migration/0:5]
>2011-04-21 11:16:56: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:16:56: 
>2011-04-21 11:16:56: Pid: 5, comm: migration/0 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:16:56: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 0
>2011-04-21 11:16:56: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:16:56: BUG: soft lockup - CPU#3 stuck for 63s! [migration/3:16]
>2011-04-21 11:16:56: Modules linked in: sunrpc ipt_REJECT ip6t_REJECT ipv6 dm_mirror dm_region_hash dm_log ppdev parport_pc parport microcode sg virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 sr_mod cdrom virtio_blk ata_generic pata_acpi ata_piix virtio_pci virtio_ring virtio dm_mod [last unloaded: nf_conntrack]
>2011-04-21 11:16:56: 
>2011-04-21 11:16:56: Pid: 16, comm: migration/3 Not tainted (2.6.32-131.0.1.el6.i686 #1) KVM
>2011-04-21 11:16:56: EIP: 0060:[<c04a14a7>] EFLAGS: 00000293 CPU: 3
>2011-04-21 11:16:56: EIP is at stop_machine_cpu_stop+0x57/0xb0
>2011-04-21 11:16:56: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f709b030
>2011-04-21 11:16:56: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f70b3ef8
>2011-04-21 11:16:56:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:16:56: CR0: 8005003b CR2: 0842e378 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:16:56: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:16:56: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:16:56: Call Trace:
>2011-04-21 11:16:56:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:16:56:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:16:56:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:16:56:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:16:56:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:16:56:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:16:56:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:16:56:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:16:56:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:16:56: EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: f705b570
>2011-04-21 11:16:56: ESI: ecf99e10 EDI: ecf99e24 EBP: c0833200 ESP: f7077ef8
>2011-04-21 11:16:56:  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>2011-04-21 11:16:56: CR0: 8005003b CR2: bfc3e5a0 CR3: 01563000 CR4: 000006f0
>2011-04-21 11:16:56: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>2011-04-21 11:16:56: DR6: ffff0ff0 DR7: 00000400
>2011-04-21 11:16:56: Call Trace:
>2011-04-21 11:16:56:  [<c04a1450>] ? stop_machine_cpu_stop+0x0/0xb0
>2011-04-21 11:16:56:  [<c04a1373>] ? cpu_stopper_thread+0x93/0x170
>2011-04-21 11:16:56:  [<c044bbfa>] ? try_to_wake_up+0x9a/0x3b0
>2011-04-21 11:16:56:  [<c043a2b7>] ? __wake_up_common+0x47/0x70
>2011-04-21 11:16:56:  [<c043e640>] ? complete+0x40/0x60
>2011-04-21 11:16:56:  [<c04a12e0>] ? cpu_stopper_thread+0x0/0x170
>2011-04-21 11:16:56:  [<c0473c84>] ? kthread+0x74/0x80
>2011-04-21 11:16:56:  [<c0473c10>] ? kthread+0x0/0x80
>2011-04-21 11:16:56:  [<c040a03f>] ? kernel_thread_helper+0x7/0x10
>2011-04-21 11:17:56: (Process terminated with status 0)

Actions: View

Attachments on bug 699556: 494775