Bug 1378395

Summary: 'handle_exit: unexpected exit_ini_info...' shows repeatedly in L1 rhel6.8 guest on AMD host
Product: Red Hat Enterprise Linux 7 Reporter: Pei Zhang <pezhang>
Component: qemu-kvm-rhevAssignee: Bandan Das <bdas>
Status: CLOSED CURRENTRELEASE QA Contact: FuXiangChun <xfu>
Severity: low Docs Contact:
Priority: medium    
Version: 7.3CC: chayang, choma, juzhang, knoel, michen, pezhang, theophanis_kontogiannis, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-02-08 18:14:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Pei Zhang 2016-09-22 10:37:12 UTC
Description of problem:

On rhel7.3 host, boot L1 guest(rhel6.8), then boot L2 guest(any guest) in L1. L1 rhel6.8 guest will repeatedly showing below info at user's terminal and #dmesg.
"
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x80000051 exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
..."

Version-Release number of selected component (if applicable):
Host: rhel7.3
3.10.0-510.el7.x86_64
qemu-kvm-rhev-2.6.0-26.el7.x86_64

L1 Guest: rhel 6.8
2.6.32-642.el6.x86_64
qemu-kvm-0.12.1.2-2.491.el6.x86_64

L2 Guest: rhel6.8 /rhel7.3 / windows 
2.6.32-642.el6.x86_64


How reproducible:
100%


Steps to Reproduce:
1. In host, set kvm_amd with nest enabled
# modprobe -r kvm_amd
# modprobe kvm_amd nested=1 

2. Boot L1 guest with '-cpu host'
# /usr/libexec/qemu-kvm -name rhel6.8 \
-cpu host -m 4G,slots=10,maxmem=10G \
-smp 8,sockets=8,cores=1,threads=1 \
-netdev tap,id=hostnet0 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=22:54:00:5c:87:61,rx_queue_size=256 \
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16 \
-spice port=5902,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on \
-monitor stdio \
-serial unix:/tmp/monitor,server,nowait \
-qmp tcp:0:5551,server,nowait \
-drive file=/home/pezhang/rhel6.8.qcow2,format=qcow2,if=none,id=drive-virtio-blk0,werror=stop,rerror=stop,cache=none \
-device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0 \
-usbdevice tablet \

3. Boot L2 guest with '-cpu Opteron_G5'
# /usr/libexec/qemu-kvm -name rhel7.3 \
-cpu Opteron_G5 -m 2G -numa node \
-smp 2,sockets=2,cores=1,threads=1 \
-spice port=5901,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on \
-monitor stdio \
-drive file=/home/pezhang/rhel6.8.qcow2,format=qcow2,if=none,id=drive-virtio-blk0,werror=stop,rerror=stop \
-device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0 \
-netdev tap,id=hostnet0 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=22:54:00:5c:77:62 \

4. L2 guest works well, but below info will many times in L1 guest #dmesg.
# dmesg
...
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x80000051 exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
handle_exit: unexpected exit_ini_info 0x800000ef exit_code 0x60
...

Actual results:
Warning info repeatedly showing in L1 guest.

Expected results:
In L1 guest, the warning message should not show repeatedly as it may cause a bad user experience.

Additional info:
1. Intel doesn't hit this issue.

Comment 6 Bandan Das 2017-11-17 21:51:47 UTC
I tested this on the same system and I see something different. Trying to run a 6.8 guest, my 6.8 L1 crashes with the following trace (in L1): 
-----------[ cut here ]------------
kernel BUG at arch/x86/kvm/../../../virt/kvm/kvm_main.c:3167!
invalid opcode: 0000 [#1] SMP 
last sysfs file: /sys/module/kvm_amd/initstate
CPU 0 
Modules linked in: ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_co]

Pid: 2452, comm: qemu-kvm Not tainted 2.6.32-696.el6.x86_64 #1 Red Hat KVM
RIP: 0010:[<ffffffffa018d3c9>]  [<ffffffffa018d3c9>] kvm_spurious_fault+0x9/0x10 [kvm]
RSP: 0018:ffff880079eebc20  EFLAGS: 00010046
RAX: ffff880078b10000 RBX: ffff88007d3a8038 RCX: 0000000002200000
RDX: 0000000000000000 RSI: ffff880079eebc54 RDI: 0000000002200000
RBP: ffff880079eebc20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000001 R12: ffff88007d3a8098
R13: 0000000000000000 R14: 0000000000000000 R15: ffff880037083520
FS:  00007fe15316b700(0000) GS:ffff880002200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe15d2c7000 CR3: 0000000036d2f000 CR4: 00000000000407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process qemu-kvm (pid: 2452, threadinfo ffff880079ee8000, task ffff880037083520)
Stack:
 ffff880079eebc88 ffffffffa0201849 ffffffffa01fdec6 000000000000bc88
<d> ffff88007d3a8038 ffff880079eebc68 0000000000000000 ffff880079eebc88
<d> ffff88007d3a8038 ffff88007d3a8098 0000000000000000 0000000000000000
Call Trace:
 [<ffffffffa0201849>] svm_hardware_unsetup+0xcd/0x172 [kvm_amd]
 [<ffffffffa01fdec6>] ? svm_vcpu_run+0x156/0x520 [kvm_amd]
 [<ffffffffa01a88fa>] kvm_arch_vcpu_ioctl_run+0x40a/0x1060 [kvm]
 [<ffffffffa018f034>] kvm_vcpu_ioctl+0x434/0x580 [kvm]
 [<ffffffff8109402d>] ? __sigqueue_free+0x3d/0x50
 [<ffffffff81097702>] ? __dequeue_signal+0x102/0x200
 [<ffffffff811af602>] vfs_ioctl+0x22/0xa0
 [<ffffffff811afaca>] do_vfs_ioctl+0x3aa/0x580
 [<ffffffff811afd21>] sys_ioctl+0x81/0xa0
 [<ffffffff810ee3ce>] ? __audit_syscall_exit+0x25e/0x290
 [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
Code: 00 00 55 48 89 e5 0f 1f 44 00 00 31 c0 48 c7 86 88 00 00 00 c0 f0 1b a0 c9 c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 0f 1f 44 00 00 <0f> 0b eb fe 0f 1f 00 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 
RIP  [<ffffffffa018d3c9>] kvm_spurious_fault+0x9/0x10 [kvm]
 RSP <ffff880079eebc20>


This looks lilke the virt instruction resulted in a fault for some reason. Seems to be like a hardware bug ? I will run my setup with an upstream kernel and check what the behavior is.

Comment 7 Theophanis Kontogiannis 2017-12-28 10:43:14 UTC
Same here with L1 6.8 L2 6.8 and host AMD A4-5000 APU with Radeon(TM) HD Graphic

How can I be of help?

Comment 8 Bandan Das 2018-02-06 20:54:45 UTC
(In reply to Theophanis Kontogiannis from comment #7)
> Same here with L1 6.8 L2 6.8 and host AMD A4-5000 APU with Radeon(TM) HD
> Graphic
> 
> How can I be of help?

Are you seeing the same trace as above ? What's the kernel version on the host ?