Bug 1002794 - KVM internal error. Suberror: 1 when doing system_reset
KVM internal error. Suberror: 1 when doing system_reset
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
6.5
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: Radim Krčmář
Virtualization Bugs
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-08-29 23:08 EDT by xhan
Modified: 2016-01-12 14:14 EST (History)
14 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 923653
Environment:
Last Closed: 2016-01-12 12:59:59 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Comment 2 xhan 2013-08-30 02:27:03 EDT
version:
host: intel-i72600-03
cpu:  Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
kernel:  2.6.32-358.18.1.el6.x86_64
qemu-kvm: qemu-kvm-rhev-0.12.1.2-2.355.el6_4.8.x86_64

guest:
RHEL.6.4-64

how producable
once

steps:

1. start vm
cmd line:
qemu -name 'vm1' -nodefaults \
-chardev socket,id=qmp_monitor_id_qmpmonitor1,path=/tmp/monitor \
-qmpmonitor1-20130829-095622-AtVA,server,nowait \
-mon chardev=qmp_monitor_id_qmpmonitor1,mode=control \
-chardev socket,id=serial_id_20130829-095622-AtVA,path=/tmp/serial-20130829-095622-AtVA,server,nowait \
-device isa-serial,chardev=serial_id_20130829-095622-AtVA \
-device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x4 \
-drive file='RHEL-Server-6.4-64-virtio.qcow2',if=none,id=drive-virtio-disk1,media=disk,cache=none,boot=off,snapshot=off,format=qcow2,aio=native \
-device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 \
-device virtio-net-pci,netdev=id6dr0XB,mac=9a:e4:cd:ec:3d:8a,id=ndev00id6dr0XB,bus=pci.0,addr=0x3 \
-netdev tap,id=id6dr0XB,vhost=on,fd=21 \
-m 4096 -smp 4,cores=2,threads=1,sockets=2 \
-cpu 'SandyBridge' \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-vnc :0 -vga cirrus -rtc base=utc,clock=host,driftfix=slew \
-M rhel6.4.0 -boot order=cdn,once=c,menu=off    \
-no-kvm-pit-reinjection -enable-kvm 
2. execute cmd line via human monitor
   system_reset
Comment 3 Xiaoqing Wei 2013-08-30 03:02:37 EDT
(In reply to xhan from comment #2)
> version:
> host: intel-i72600-03
> cpu:  Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
> kernel:  2.6.32-358.18.1.el6.x86_64
> qemu-kvm: qemu-kvm-rhev-0.12.1.2-2.355.el6_4.8.x86_64
> 

Hi Xhan,
pls try on latset rhel6.5 pkgs too, and update here, thx.
Comment 4 xhan 2013-09-02 20:45:27 EDT
did not meet this problem on rhel6.5 qemu-kvm-0.12.1.2-2.398.el6.x86_64_kernel_2.6.32-414.el6.x86_64 with running 50 times on the same host.
Comment 5 Xiaoqing Wei 2013-09-02 22:14:26 EDT
(In reply to xhan from comment #4)
> did not meet this problem on rhel6.5
> qemu-kvm-0.12.1.2-2.398.el6.x86_64_kernel_2.6.32-414.el6.x86_64 with running
> 50 times on the same host.

So you mean this is a Regression or not ?
Comment 6 xhan 2013-09-02 22:28:11 EDT
Although run it 50 times on rhel6.5 with no error, I don't think it is a regression problem. 
I would retest with qemu-kvm-355_4.7 to see if it can happen.
Comment 7 Qunfang Zhang 2013-09-04 05:32:12 EDT
(In reply to xhan from comment #6)
> Although run it 50 times on rhel6.5 with no error, I don't think it is a
> regression problem. 
> I would retest with qemu-kvm-355_4.7 to see if it can happen.

Hi, Xiangming

What is the result on qemu-kvm-355_4.7? And what is the reproducible rate on  qemu-kvm-355_4.8?
Comment 8 xhan 2013-09-04 06:12:07 EDT
Test on host intel-i72600-03 with qemu-kvm-355_4.7 and qemu-kvm-355_4.8 each 50 times. Did not meet this issue. 

This issue is found once in Acceptance testing for qemu-kvm-rhev-0.12.1.2-2.355.el6_4.8. And it seems could not be duplicated.

In view of the above, this problem is not a regression.
Comment 9 Ademar Reis 2013-09-05 14:03:32 EDT
(In reply to xhan from comment #8)
> Test on host intel-i72600-03 with qemu-kvm-355_4.7 and qemu-kvm-355_4.8 each
> 50 times. Did not meet this issue. 
> 
> This issue is found once in Acceptance testing for
> qemu-kvm-rhev-0.12.1.2-2.355.el6_4.8. And it seems could not be duplicated.

Closing then. Please reopen if you manage to reproduce it.
Comment 13 Xu Han 2014-08-11 01:06:38 EDT
Reproduced this bug:

seabios-0.6.1.2-28.el6.x86_64
kernel-2.6.32-494.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.434.el6.x86_64

Result:
KVMInternalError: KVM internal error.Suberror: 1
rax 0000000000000040 rbx 000000000000000a rcx 000000000000c2b8 rdx 0000000000000402
rsi 0000000000000012 rdi 0000000000000008 rsp 0000000000000368 rbp 0000000000000000
r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
rip 0000000000000004 rflags 00010046
cs 0600 (00006000/0000ffff p 1 dpl 0 db 0 s 1 type b l 0 g 0 avl 0)
ds 0040 (00000400/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
es 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
ss 9dc0 (0009dc00/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
fs 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
gs 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
tr 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
gdt 0/ffff
idt 0/ffff
cr0 10 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
emulation failure, check dmesg for details

Recent logs:
08/04 16:58:11 DEBUG|qemu_monit:1372| Send command: {'execute': 'system_reset', 'id': 'DvYbu1ei'}
08/04 16:58:11 DEBUG|system_res:0053| Reset the system by monitor cmd after 12secs
08/04 16:58:13 DEBUG|qemu_monit:0267| (monitor qmpmonitor1) Sending command 'info registers' (via Human Monitor)
08/04 16:58:18 DEBUG|qemu_monit:0267| (monitor qmpmonitor1) Sending command 'info registers' (via Human Monitor)
08/04 16:58:22 DEBUG| utils_env:0081| (address cache) DHCP lease OK: 00:15:5d:6a:49:1e --> 10.66.110.168
08/04 16:58:23 DEBUG|qemu_monit:0267| (monitor qmpmonitor1) Sending command 'info registers' (via Human Monitor)
08/04 16:58:23 INFO |     error:0085| Context: Reset guest system for 10 times
08/04 16:58:23 DEBUG|qemu_monit:0267| (monitor qmpmonitor1) Sending command 'system_reset' 
08/04 16:58:23 DEBUG|qemu_monit:1372| Send command: {'execute': 'system_reset', 'id': 'qA5wqF2c'}
08/04 16:58:23 DEBUG|system_res:0053| Reset the system by monitor cmd after 6secs
08/04 16:58:23 INFO |   aexpect:0968| [qemu output] KVM internal error.Suberror: 1
08/04 16:58:23 INFO |   aexpect:0968| [qemu output] rax 0000000000000040 rbx 000000000000000a rcx 000000000000c2b8 rdx 0000000000000402
08/04 16:58:23 INFO |   aexpect:0968| [qemu output] rsi 0000000000000012 rdi 0000000000000008 rsp 0000000000000368 rbp 0000000000000000
...

Seabios logs:
2014-08-04 16:58:11: Start bios (version seabios-0.6.1.2-28.el6)
2014-08-04 16:58:11: Ram Size=0xe0000000 (0x0000000320000000 high)
2014-08-04 16:58:11: CPU Mhz=2503
2014-08-04 16:58:11: PCI: pci_bios_init_bus_rec bus = 0x0
2014-08-04 16:58:11: PIIX3/PIIX4 init: elcr=00 0c
2014-08-04 16:58:11: PCI: bus=0 devfn=0x00: vendor_id=0x8086 device_id=0x1237
2014-08-04 16:58:11: PCI: bus=0 devfn=0x08: vendor_id=0x8086 device_id=0x7000
2014-08-04 16:58:11: PCI: bus=0 devfn=0x09: vendor_id=0x8086 device_id=0x7010
2014-08-04 16:58:11: region 4: 0x0000c000
2014-08-04 16:58:11: PCI: bus=0 devfn=0x0b: vendor_id=0x8086 device_id=0x7113
2014-08-04 16:58:11: Using pmtimer, ioport 0xb008, freq 3579 kHz
2014-08-04 16:58:11: PCI: bus=0 devfn=0x10: vendor_id=0x1b36 device_id=0x0100
2014-08-04 16:58:11: region 0: 0xf0000000
2014-08-04 16:58:11: region 1: 0xf4000000
2014-08-04 16:58:11: region 2: 0xf6000000
2014-08-04 16:58:11: region 3: 0x0000c020
2014-08-04 16:58:11: region 6: 0xf6010000
2014-08-04 16:58:11: PCI: bus=0 devfn=0x18: vendor_id=0x8086 device_id=0x2415
2014-08-04 16:58:11: region 0: 0x0000c400
2014-08-04 16:58:11: region 1: 0x0000c800
2014-08-04 16:58:11: PCI: bus=0 devfn=0x20: vendor_id=0x8086 device_id=0x2934
2014-08-04 16:58:11: region 4: 0x0000c900
2014-08-04 16:58:11: PCI: bus=0 devfn=0x28: vendor_id=0x1af4 device_id=0x1001
2014-08-04 16:58:11: region 0: 0x0000c940
2014-08-04 16:58:11: region 1: 0xf6020000
2014-08-04 16:58:11: PCI: bus=0 devfn=0x30: vendor_id=0x1af4 device_id=0x1000
2014-08-04 16:58:11: region 0: 0x0000c980
2014-08-04 16:58:11: region 1: 0xf6021000
2014-08-04 16:58:11: region 6: 0xf6030000
2014-08-04 16:58:11: Found 8 cpu(s) max supported 8 cpu(s)
2014-08-04 16:58:11: MP table addr=0x000fda30 MPC table addr=0x000fda40 size=268
2014-08-04 16:58:12: SMBIOS ptr=0x000fda10 table=0xdffffd60
2014-08-04 16:58:12: ACPI tables: RSDP=0x000fd9e0 RSDT=0xdfffd210
2014-08-04 16:58:12: Scan for VGA option rom
2014-08-04 16:58:12: Running option rom at c000:0003
2014-08-04 16:58:12: Turning on vga text mode console
2014-08-04 16:58:12: SeaBIOS (version seabios-0.6.1.2-28.el6)
2014-08-04 16:58:12: UHCI init on dev 00:04.0 (io=c900)
2014-08-04 16:58:12: Found 0 lpt ports
2014-08-04 16:58:12: Found 1 serial ports
2014-08-04 16:58:12: ATA controller 0 at 1f0/3f4/0 (irq 14 dev 9)
2014-08-04 16:58:12: ATA controller 1 at 170/374/0 (irq 15 dev 9)
2014-08-04 16:58:12: found virtio-blk at 0:5
2014-08-04 16:58:12: ebda moved from 9fc00 to 9dc00
2014-08-04 16:58:12: Searching bootorder for: /pci@i0cf8/*@5
2014-08-04 16:58:12: PS2 keyboard initialized
2014-08-04 16:58:12: All threads complete.
2014-08-04 16:58:12: Scan for option roms
2014-08-04 16:58:12: Running option rom at ca00:0003
2014-08-04 16:58:12: pnp call arg1=60
2014-08-04 16:58:12: pmm call arg1=0
2014-08-04 16:58:12: pmm call arg1=2
2014-08-04 16:58:12: pmm call arg1=0
2014-08-04 16:58:16: Searching bootorder for: /pci@i0cf8/*@6
2014-08-04 16:58:16: Searching bootorder for: /rom@genroms/vapic.bin
2014-08-04 16:58:16: Searching bootorder for: HALT
2014-08-04 16:58:16: drive 0x000fd990: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=41943040
2014-08-04 16:58:16: Running option rom at ca80:0003
2014-08-04 16:58:16: Returned 53248 bytes of ZoneHigh
2014-08-04 16:58:16: e820 map has 7 items:
2014-08-04 16:58:16:   0: 0000000000000000 - 000000000009dc00 = 1
2014-08-04 16:58:16:   1: 000000000009dc00 - 00000000000a0000 = 2
2014-08-04 16:58:16:   2: 00000000000f0000 - 0000000000100000 = 2
2014-08-04 16:58:16:   3: 0000000000100000 - 00000000dfffd000 = 1
2014-08-04 16:58:16:   4: 00000000dfffd000 - 00000000e0000000 = 2
2014-08-04 16:58:16:   5: 00000000fffbc000 - 0000000100000000 = 2
2014-08-04 16:58:16:   6: 0000000100000000 - 0000000420000000 = 1
2014-08-04 16:58:16: enter handle_19:
2014-08-04 16:58:16:   NULL
2014-08-04 16:58:16: Booting from Hard Disk...
2014-08-04 16:58:16: Booting from 0000:7c00
2014-08-04 16:58:23: In resume (status=10)
2014-08-04 17:17:00: (Process terminated with status 0)
Comment 14 Xu Han 2014-08-11 01:10:53 EDT
Command line:
/usr/bin/qemu-kvm \
    -S  \
    -name 'virt-tests-vm1' \
    -M rhel6.5.0  \
    -nodefaults  \
    -vga qxl  \
    -global qxl-vga.vram_size=33554432 \
    -device AC97,bus=pci.0,addr=03  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140804-164242-BJUVNWRm,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140804-164242-BJUVNWRm,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20140804-164242-BJUVNWRm,path=/tmp/seabios-20140804-164242-BJUVNWRm,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20140804-164242-BJUVNWRm,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=04 \
    -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/root/autotest-devel/client/tests/virt/shared/data/images/RHEL-Server-5.11-64-virtio.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=05 \
    -device virtio-net-pci,mac=9a:44:45:46:47:48,id=idJ1Owg5,vectors=4,netdev=idXXdZKM,bus=pci.0,addr=06  \
    -netdev tap,id=idXXdZKM,vhost=on,vhostfd=26,fd=20  \
    -m 16384  \
    -smp 8,maxcpus=8,cores=4,threads=1,sockets=2  \
    -cpu 'SandyBridge' \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -spice port=3000,password=123456,addr=0,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off  \
    -no-kvm-pit-reinjection \
    -enable-kvm
Comment 18 ShupingCui 2014-10-28 20:21:29 EDT
Reproduced this bug on:
# uname -r
2.6.32-504.1.2.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.448.el6_6.x86_64
# rpm -q seabios
seabios-0.6.1.2-28.el6.x86_64

[qemu output] KVM internal error.Suberror: 1
[qemu output] rax 0000000000000040 rbx 000000000000000a rcx 000000000000c2b8 rdx 0000000000000402
[qemu output] rsi 0000000000000012 rdi 0000000000000008 rsp 0000000000000368 rbp 0000000000000000
[qemu output] r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
[qemu output] r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
[qemu output] rip 0000000000000004 rflags 00010046
[qemu output] cs 0600 (00006000/0000ffff p 1 dpl 0 db 0 s 1 type b l 0 g 0 avl 0)
[qemu output] ds 0040 (00000400/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] es 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] ss 9dc0 (0009dc00/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] fs 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] gs 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] tr 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
[qemu output] ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
[qemu output] gdt 0/ffff
[qemu output] idt 0/ffff
[qemu output] cr0 10 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
[qemu output] emulation failure, check dmesg for details
Comment 19 Radim Krčmář 2014-11-11 16:34:30 EST
(In reply to ShupingCui from comment #18)
> Reproduced this bug on:

How long does the reproducer have run?

What is the host configuration? (CPU, KVM parameters)

Thanks.
Comment 23 ShupingCui 2014-11-11 22:37:36 EST
(In reply to Radim Krčmář from comment #19)
> (In reply to ShupingCui from comment #18)
> > Reproduced this bug on:
> 
> How long does the reproducer have run?
> 
> What is the host configuration? (CPU, KVM parameters)
> 
> Thanks.

For KVM parameters:

# cat allow_unsafe_assigned_interrupts 
N
# cat /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts 
N
# cat /sys/module/kvm/parameters/ignore_msrs 
N
# cat /sys/module/kvm/parameters/min_timer_period_us 
500
# cat /sys/module/kvm/parameters/oos_shadow 
Y

# cat /sys/module/kvm_intel/parameters/bypass_guest_pf
N
# cat /sys/module/kvm_intel/parameters/emulate_invalid_guest_state
N
# cat /sys/module/kvm_intel/parameters/ept
Y
# cat /sys/module/kvm_intel/parameters/eptad 
N
# cat /sys/module/kvm_intel/parameters/flexpriority 
Y
# cat /sys/module/kvm_intel/parameters/ple_gap 
0
# cat /sys/module/kvm_intel/parameters/ple_window
4096
# cat /sys/module/kvm_intel/parameters/ple_window_grow 
2
# cat /sys/module/kvm_intel/parameters/ple_window_max 
1073741823
# cat /sys/module/kvm_intel/parameters/ple_window_shrink 
0
# cat /sys/module/kvm_intel/parameters/unrestricted_guest 
Y
# cat /sys/module/kvm_intel/parameters/vmm_exclusive 
Y
# cat /sys/module/kvm_intel/parameters/vpid 
Y
# cat /sys/module/kvm_intel/parameters/yield_on_hlt 
Y
Comment 24 Radim Krčmář 2015-01-28 09:01:30 EST
Sorry for the delay ... flags look normal and I had no luck reproducing with a very similar setup,

please lend me the host machine if you hit it again.

Thank you.

(The bug is not critical, so I'm deferring it to 6.8.)
Comment 26 Radim Krčmář 2016-01-12 12:59:59 EST
It wasn't reproduced since comment 24, the reproducer relies on operation that is very rare in real-world scenarios and RHEL6.8 is coming to an end.  It's unlikely that we'll fix this in RHEL6.

Note You need to log in before you can comment on or make changes to this bug.