1002794 – KVM internal error. Suberror: 1 when doing system_reset

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1002794 - KVM internal error. Suberror: 1 when doing system_reset

Summary: KVM internal error. Suberror: 1 when doing system_reset

Keywords:
Status:	CLOSED WONTFIX
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	qemu-kvm
Sub Component:
Version:	6.5
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Radim Krčmář
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2013-08-30 03:08 UTC by xhan
Modified:	2016-01-12 19:14 UTC (History)
CC List:	14 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:	923653
Environment:
Last Closed:	2016-01-12 17:59:59 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Comment 2 xhan 2013-08-30 06:27:03 UTC

version:
host: intel-i72600-03
cpu:  Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
kernel:  2.6.32-358.18.1.el6.x86_64
qemu-kvm: qemu-kvm-rhev-0.12.1.2-2.355.el6_4.8.x86_64

guest:
RHEL.6.4-64

how producable
once

steps:

1. start vm
cmd line:
qemu -name 'vm1' -nodefaults \
-chardev socket,id=qmp_monitor_id_qmpmonitor1,path=/tmp/monitor \
-qmpmonitor1-20130829-095622-AtVA,server,nowait \
-mon chardev=qmp_monitor_id_qmpmonitor1,mode=control \
-chardev socket,id=serial_id_20130829-095622-AtVA,path=/tmp/serial-20130829-095622-AtVA,server,nowait \
-device isa-serial,chardev=serial_id_20130829-095622-AtVA \
-device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x4 \
-drive file='RHEL-Server-6.4-64-virtio.qcow2',if=none,id=drive-virtio-disk1,media=disk,cache=none,boot=off,snapshot=off,format=qcow2,aio=native \
-device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 \
-device virtio-net-pci,netdev=id6dr0XB,mac=9a:e4:cd:ec:3d:8a,id=ndev00id6dr0XB,bus=pci.0,addr=0x3 \
-netdev tap,id=id6dr0XB,vhost=on,fd=21 \
-m 4096 -smp 4,cores=2,threads=1,sockets=2 \
-cpu 'SandyBridge' \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-vnc :0 -vga cirrus -rtc base=utc,clock=host,driftfix=slew \
-M rhel6.4.0 -boot order=cdn,once=c,menu=off    \
-no-kvm-pit-reinjection -enable-kvm 
2. execute cmd line via human monitor
   system_reset

Comment 3 Xiaoqing Wei 2013-08-30 07:02:37 UTC

(In reply to xhan from comment #2)
> version:
> host: intel-i72600-03
> cpu:  Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
> kernel:  2.6.32-358.18.1.el6.x86_64
> qemu-kvm: qemu-kvm-rhev-0.12.1.2-2.355.el6_4.8.x86_64
> 

Hi Xhan,
pls try on latset rhel6.5 pkgs too, and update here, thx.

Comment 4 xhan 2013-09-03 00:45:27 UTC

did not meet this problem on rhel6.5 qemu-kvm-0.12.1.2-2.398.el6.x86_64_kernel_2.6.32-414.el6.x86_64 with running 50 times on the same host.

Comment 5 Xiaoqing Wei 2013-09-03 02:14:26 UTC

(In reply to xhan from comment #4)
> did not meet this problem on rhel6.5
> qemu-kvm-0.12.1.2-2.398.el6.x86_64_kernel_2.6.32-414.el6.x86_64 with running
> 50 times on the same host.

So you mean this is a Regression or not ?

Comment 6 xhan 2013-09-03 02:28:11 UTC

Although run it 50 times on rhel6.5 with no error, I don't think it is a regression problem. 
I would retest with qemu-kvm-355_4.7 to see if it can happen.

Comment 7 Qunfang Zhang 2013-09-04 09:32:12 UTC

(In reply to xhan from comment #6)
> Although run it 50 times on rhel6.5 with no error, I don't think it is a
> regression problem. 
> I would retest with qemu-kvm-355_4.7 to see if it can happen.

Hi, Xiangming

What is the result on qemu-kvm-355_4.7? And what is the reproducible rate on  qemu-kvm-355_4.8?

Comment 8 xhan 2013-09-04 10:12:07 UTC

Test on host intel-i72600-03 with qemu-kvm-355_4.7 and qemu-kvm-355_4.8 each 50 times. Did not meet this issue. 

This issue is found once in Acceptance testing for qemu-kvm-rhev-0.12.1.2-2.355.el6_4.8. And it seems could not be duplicated.

In view of the above, this problem is not a regression.

Comment 9 Ademar Reis 2013-09-05 18:03:32 UTC

(In reply to xhan from comment #8)
> Test on host intel-i72600-03 with qemu-kvm-355_4.7 and qemu-kvm-355_4.8 each
> 50 times. Did not meet this issue. 
> 
> This issue is found once in Acceptance testing for
> qemu-kvm-rhev-0.12.1.2-2.355.el6_4.8. And it seems could not be duplicated.

Closing then. Please reopen if you manage to reproduce it.

Comment 13 Xu Han 2014-08-11 05:06:38 UTC

Reproduced this bug:

seabios-0.6.1.2-28.el6.x86_64
kernel-2.6.32-494.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.434.el6.x86_64

Result:
KVMInternalError: KVM internal error.Suberror: 1
rax 0000000000000040 rbx 000000000000000a rcx 000000000000c2b8 rdx 0000000000000402
rsi 0000000000000012 rdi 0000000000000008 rsp 0000000000000368 rbp 0000000000000000
r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
rip 0000000000000004 rflags 00010046
cs 0600 (00006000/0000ffff p 1 dpl 0 db 0 s 1 type b l 0 g 0 avl 0)
ds 0040 (00000400/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
es 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
ss 9dc0 (0009dc00/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
fs 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
gs 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
tr 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
gdt 0/ffff
idt 0/ffff
cr0 10 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
emulation failure, check dmesg for details

Recent logs:
08/04 16:58:11 DEBUG|qemu_monit:1372| Send command: {'execute': 'system_reset', 'id': 'DvYbu1ei'}
08/04 16:58:11 DEBUG|system_res:0053| Reset the system by monitor cmd after 12secs
08/04 16:58:13 DEBUG|qemu_monit:0267| (monitor qmpmonitor1) Sending command 'info registers' (via Human Monitor)
08/04 16:58:18 DEBUG|qemu_monit:0267| (monitor qmpmonitor1) Sending command 'info registers' (via Human Monitor)
08/04 16:58:22 DEBUG| utils_env:0081| (address cache) DHCP lease OK: 00:15:5d:6a:49:1e --> 10.66.110.168
08/04 16:58:23 DEBUG|qemu_monit:0267| (monitor qmpmonitor1) Sending command 'info registers' (via Human Monitor)
08/04 16:58:23 INFO |     error:0085| Context: Reset guest system for 10 times
08/04 16:58:23 DEBUG|qemu_monit:0267| (monitor qmpmonitor1) Sending command 'system_reset' 
08/04 16:58:23 DEBUG|qemu_monit:1372| Send command: {'execute': 'system_reset', 'id': 'qA5wqF2c'}
08/04 16:58:23 DEBUG|system_res:0053| Reset the system by monitor cmd after 6secs
08/04 16:58:23 INFO |   aexpect:0968| [qemu output] KVM internal error.Suberror: 1
08/04 16:58:23 INFO |   aexpect:0968| [qemu output] rax 0000000000000040 rbx 000000000000000a rcx 000000000000c2b8 rdx 0000000000000402
08/04 16:58:23 INFO |   aexpect:0968| [qemu output] rsi 0000000000000012 rdi 0000000000000008 rsp 0000000000000368 rbp 0000000000000000
...

Seabios logs:
2014-08-04 16:58:11: Start bios (version seabios-0.6.1.2-28.el6)
2014-08-04 16:58:11: Ram Size=0xe0000000 (0x0000000320000000 high)
2014-08-04 16:58:11: CPU Mhz=2503
2014-08-04 16:58:11: PCI: pci_bios_init_bus_rec bus = 0x0
2014-08-04 16:58:11: PIIX3/PIIX4 init: elcr=00 0c
2014-08-04 16:58:11: PCI: bus=0 devfn=0x00: vendor_id=0x8086 device_id=0x1237
2014-08-04 16:58:11: PCI: bus=0 devfn=0x08: vendor_id=0x8086 device_id=0x7000
2014-08-04 16:58:11: PCI: bus=0 devfn=0x09: vendor_id=0x8086 device_id=0x7010
2014-08-04 16:58:11: region 4: 0x0000c000
2014-08-04 16:58:11: PCI: bus=0 devfn=0x0b: vendor_id=0x8086 device_id=0x7113
2014-08-04 16:58:11: Using pmtimer, ioport 0xb008, freq 3579 kHz
2014-08-04 16:58:11: PCI: bus=0 devfn=0x10: vendor_id=0x1b36 device_id=0x0100
2014-08-04 16:58:11: region 0: 0xf0000000
2014-08-04 16:58:11: region 1: 0xf4000000
2014-08-04 16:58:11: region 2: 0xf6000000
2014-08-04 16:58:11: region 3: 0x0000c020
2014-08-04 16:58:11: region 6: 0xf6010000
2014-08-04 16:58:11: PCI: bus=0 devfn=0x18: vendor_id=0x8086 device_id=0x2415
2014-08-04 16:58:11: region 0: 0x0000c400
2014-08-04 16:58:11: region 1: 0x0000c800
2014-08-04 16:58:11: PCI: bus=0 devfn=0x20: vendor_id=0x8086 device_id=0x2934
2014-08-04 16:58:11: region 4: 0x0000c900
2014-08-04 16:58:11: PCI: bus=0 devfn=0x28: vendor_id=0x1af4 device_id=0x1001
2014-08-04 16:58:11: region 0: 0x0000c940
2014-08-04 16:58:11: region 1: 0xf6020000
2014-08-04 16:58:11: PCI: bus=0 devfn=0x30: vendor_id=0x1af4 device_id=0x1000
2014-08-04 16:58:11: region 0: 0x0000c980
2014-08-04 16:58:11: region 1: 0xf6021000
2014-08-04 16:58:11: region 6: 0xf6030000
2014-08-04 16:58:11: Found 8 cpu(s) max supported 8 cpu(s)
2014-08-04 16:58:11: MP table addr=0x000fda30 MPC table addr=0x000fda40 size=268
2014-08-04 16:58:12: SMBIOS ptr=0x000fda10 table=0xdffffd60
2014-08-04 16:58:12: ACPI tables: RSDP=0x000fd9e0 RSDT=0xdfffd210
2014-08-04 16:58:12: Scan for VGA option rom
2014-08-04 16:58:12: Running option rom at c000:0003
2014-08-04 16:58:12: Turning on vga text mode console
2014-08-04 16:58:12: SeaBIOS (version seabios-0.6.1.2-28.el6)
2014-08-04 16:58:12: UHCI init on dev 00:04.0 (io=c900)
2014-08-04 16:58:12: Found 0 lpt ports
2014-08-04 16:58:12: Found 1 serial ports
2014-08-04 16:58:12: ATA controller 0 at 1f0/3f4/0 (irq 14 dev 9)
2014-08-04 16:58:12: ATA controller 1 at 170/374/0 (irq 15 dev 9)
2014-08-04 16:58:12: found virtio-blk at 0:5
2014-08-04 16:58:12: ebda moved from 9fc00 to 9dc00
2014-08-04 16:58:12: Searching bootorder for: /pci@i0cf8/*@5
2014-08-04 16:58:12: PS2 keyboard initialized
2014-08-04 16:58:12: All threads complete.
2014-08-04 16:58:12: Scan for option roms
2014-08-04 16:58:12: Running option rom at ca00:0003
2014-08-04 16:58:12: pnp call arg1=60
2014-08-04 16:58:12: pmm call arg1=0
2014-08-04 16:58:12: pmm call arg1=2
2014-08-04 16:58:12: pmm call arg1=0
2014-08-04 16:58:16: Searching bootorder for: /pci@i0cf8/*@6
2014-08-04 16:58:16: Searching bootorder for: /rom@genroms/vapic.bin
2014-08-04 16:58:16: Searching bootorder for: HALT
2014-08-04 16:58:16: drive 0x000fd990: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=41943040
2014-08-04 16:58:16: Running option rom at ca80:0003
2014-08-04 16:58:16: Returned 53248 bytes of ZoneHigh
2014-08-04 16:58:16: e820 map has 7 items:
2014-08-04 16:58:16:   0: 0000000000000000 - 000000000009dc00 = 1
2014-08-04 16:58:16:   1: 000000000009dc00 - 00000000000a0000 = 2
2014-08-04 16:58:16:   2: 00000000000f0000 - 0000000000100000 = 2
2014-08-04 16:58:16:   3: 0000000000100000 - 00000000dfffd000 = 1
2014-08-04 16:58:16:   4: 00000000dfffd000 - 00000000e0000000 = 2
2014-08-04 16:58:16:   5: 00000000fffbc000 - 0000000100000000 = 2
2014-08-04 16:58:16:   6: 0000000100000000 - 0000000420000000 = 1
2014-08-04 16:58:16: enter handle_19:
2014-08-04 16:58:16:   NULL
2014-08-04 16:58:16: Booting from Hard Disk...
2014-08-04 16:58:16: Booting from 0000:7c00
2014-08-04 16:58:23: In resume (status=10)
2014-08-04 17:17:00: (Process terminated with status 0)

Comment 14 Xu Han 2014-08-11 05:10:53 UTC

Command line:
/usr/bin/qemu-kvm \
    -S  \
    -name 'virt-tests-vm1' \
    -M rhel6.5.0  \
    -nodefaults  \
    -vga qxl  \
    -global qxl-vga.vram_size=33554432 \
    -device AC97,bus=pci.0,addr=03  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140804-164242-BJUVNWRm,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140804-164242-BJUVNWRm,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20140804-164242-BJUVNWRm,path=/tmp/seabios-20140804-164242-BJUVNWRm,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20140804-164242-BJUVNWRm,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=04 \
    -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/root/autotest-devel/client/tests/virt/shared/data/images/RHEL-Server-5.11-64-virtio.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=05 \
    -device virtio-net-pci,mac=9a:44:45:46:47:48,id=idJ1Owg5,vectors=4,netdev=idXXdZKM,bus=pci.0,addr=06  \
    -netdev tap,id=idXXdZKM,vhost=on,vhostfd=26,fd=20  \
    -m 16384  \
    -smp 8,maxcpus=8,cores=4,threads=1,sockets=2  \
    -cpu 'SandyBridge' \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -spice port=3000,password=123456,addr=0,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off  \
    -no-kvm-pit-reinjection \
    -enable-kvm

Comment 18 ShupingCui 2014-10-29 00:21:29 UTC

Reproduced this bug on:
# uname -r
2.6.32-504.1.2.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.448.el6_6.x86_64
# rpm -q seabios
seabios-0.6.1.2-28.el6.x86_64

[qemu output] KVM internal error.Suberror: 1
[qemu output] rax 0000000000000040 rbx 000000000000000a rcx 000000000000c2b8 rdx 0000000000000402
[qemu output] rsi 0000000000000012 rdi 0000000000000008 rsp 0000000000000368 rbp 0000000000000000
[qemu output] r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
[qemu output] r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
[qemu output] rip 0000000000000004 rflags 00010046
[qemu output] cs 0600 (00006000/0000ffff p 1 dpl 0 db 0 s 1 type b l 0 g 0 avl 0)
[qemu output] ds 0040 (00000400/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] es 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] ss 9dc0 (0009dc00/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] fs 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] gs 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
[qemu output] tr 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
[qemu output] ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
[qemu output] gdt 0/ffff
[qemu output] idt 0/ffff
[qemu output] cr0 10 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
[qemu output] emulation failure, check dmesg for details

Comment 19 Radim Krčmář 2014-11-11 21:34:30 UTC

(In reply to ShupingCui from comment #18)
> Reproduced this bug on:

How long does the reproducer have run?

What is the host configuration? (CPU, KVM parameters)

Thanks.

Comment 23 ShupingCui 2014-11-12 03:37:36 UTC

(In reply to Radim Krčmář from comment #19)
> (In reply to ShupingCui from comment #18)
> > Reproduced this bug on:
> 
> How long does the reproducer have run?
> 
> What is the host configuration? (CPU, KVM parameters)
> 
> Thanks.

For KVM parameters:

# cat allow_unsafe_assigned_interrupts 
N
# cat /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts 
N
# cat /sys/module/kvm/parameters/ignore_msrs 
N
# cat /sys/module/kvm/parameters/min_timer_period_us 
500
# cat /sys/module/kvm/parameters/oos_shadow 
Y

# cat /sys/module/kvm_intel/parameters/bypass_guest_pf
N
# cat /sys/module/kvm_intel/parameters/emulate_invalid_guest_state
N
# cat /sys/module/kvm_intel/parameters/ept
Y
# cat /sys/module/kvm_intel/parameters/eptad 
N
# cat /sys/module/kvm_intel/parameters/flexpriority 
Y
# cat /sys/module/kvm_intel/parameters/ple_gap 
0
# cat /sys/module/kvm_intel/parameters/ple_window
4096
# cat /sys/module/kvm_intel/parameters/ple_window_grow 
2
# cat /sys/module/kvm_intel/parameters/ple_window_max 
1073741823
# cat /sys/module/kvm_intel/parameters/ple_window_shrink 
0
# cat /sys/module/kvm_intel/parameters/unrestricted_guest 
Y
# cat /sys/module/kvm_intel/parameters/vmm_exclusive 
Y
# cat /sys/module/kvm_intel/parameters/vpid 
Y
# cat /sys/module/kvm_intel/parameters/yield_on_hlt 
Y

Comment 24 Radim Krčmář 2015-01-28 14:01:30 UTC

Sorry for the delay ... flags look normal and I had no luck reproducing with a very similar setup,

please lend me the host machine if you hit it again.

Thank you.

(The bug is not critical, so I'm deferring it to 6.8.)

Comment 26 Radim Krčmář 2016-01-12 17:59:59 UTC

It wasn't reproduced since comment 24, the reproducer relies on operation that is very rare in real-world scenarios and RHEL6.8 is coming to an end.  It's unlikely that we'll fix this in RHEL6.

Note You need to log in before you can comment on or make changes to this bug.