Bug 757660

Summary: Kernel panic - not syncing: IO-APIC + timer doesn't work! was shown when starting kvm_guest installation
Product: Red Hat Enterprise Linux 5 Reporter: Xiaoqing Wei <xwei>
Component: kvmAssignee: Eduardo Habkost <ehabkost>
Status: CLOSED NEXTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.8CC: akong, amit.shah, areis, juzhang, michen, mkenneth, rhod, shuang, sradvan, tburke, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
We recommend the divider=10 guest kernel parameter for RHEL5 guests
 Story Points: ---
Clone Of: Environment:
Last Closed: 2012-04-18 13:41:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 807971    
Attachments:
Description Flags
detail info
none
full output of serial none

Description Xiaoqing Wei 2011-11-28 10:12:45 UTC 
Description of problem:

Kernel panic - not syncing: IO-APIC + timer doesn't work!   was shown when starting kvm_guest installation
Version-Release number of selected component (if applicable):

kvm-83-246.el5
kernel-2.6.18-298.el5

How reproducible:
met it only twice

Steps to Reproduce:
1.start install RHEL-5.7-32 using ide / e1000 / spice

/usr/local/staf/test/RHEV/kvm-new/autotest/client/tests/kvm/qemu -name 'vm1'
-monitor unix:'/tmp/monitor-humanmonitor1-20111125-170421-tsWQ',server,nowait
-serial unix:'/tmp/serial-20111125-170421-tsWQ',server,nowait \
-drive
file='RHEL-Server-5.7-32.qcow2',index=0,if=ide,media=disk,cache=none,format=qcow2
\
-net nic,vlan=0,model=e1000,macaddr='9a:2b:06:a6:6a:ae' \
-net tap,vlan=0,ifname='t0-170421-tsWQ' -m 4096 -smp
2,cores=1,threads=1,sockets=2 \
-drive file='RHEL-Server-5.7-i386-DVD.iso',media=cdrom,index=1 -drive
file='ks.iso',media=cdrom,index=2 \
 -cpu qemu64,+sse2 -soundhw ac97 -kernel 'vmlinuz' -initrd 'initrd.img' \
-spice port=8000,disable-ticketing -qxl 1 -rtc-td-hack -boot d   \
-no-kvm-pit-reinjection \
--append 'ks=cdrom nicdelay=60 console=ttyS0,115200 console=tty0'  -M rhel5.6.0
-usbdevice tablet 


below is the last few lines of guest serial output:

2011-11-26 03:07:09: Calibrating delay using timer specific routine.. 5816.46
BogoMIPS (lpj=2908231)
2011-11-26 03:07:09: CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64
bytes/line)
2011-11-26 03:07:09: CPU: L2 Cache: 512K (64 bytes/line)
2011-11-26 03:07:09: Intel machine check architecture supported.
2011-11-26 03:07:09: Intel machine check reporting enabled on CPU#1.
2011-11-26 03:07:09: CPU1: AMD QEMU Virtual CPU version 0.9.1 stepping 03
2011-11-26 03:07:09: Total of 2 processors activated (7820.79 BogoMIPS).
2011-11-26 03:07:09: WARNING: This combination of AMD processors is not
suitable for SMP.
2011-11-26 03:07:09: ENABLING IO-APIC IRQs
2011-11-26 03:07:09: ..TIMER: vector=0x31 apic1=0 pin1=0 apic2=-1 pin2=-1
2011-11-26 03:07:09: ..MP-BIOS bug: 8254 timer not connected to IO-APIC
2011-11-26 03:07:09: ...trying to set up timer (IRQ0) through the 8259A ... 
failed.
2011-11-26 03:07:09: ...trying to set up timer as Virtual Wire IRQ... failed.
2011-11-26 03:07:10: ...trying to set up timer as ExtINT IRQ... failed :(.
2011-11-26 03:07:10: Kernel panic - not syncing: IO-APIC + timer doesn't work! 
Boot with apic=debug and send a report.  Then try booting with the 'noapic'
option
2011-11-26 03:07:10: 


  
Actual results:

guest kernel panic
Expected results:

guest installation successfully


Additional info:
Host cpu:

processor	: 1
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 107
model name	: AMD Athlon(tm) 64 X2 Dual Core Processor 5600+
stepping	: 2
cpu MHz		: 1000.000
cache size	: 512 KB
Comment 1 Xiaoqing Wei 2011-11-28 10:25:56 UTC 
Created attachment 537424 [details]
detail info
Comment 2 Dor Laor 2011-11-29 14:48:37 UTC 
Can you retest with -smp 2 without the complex topology settings?
Comment 4 Xiaoqing Wei 2011-11-30 08:07:35 UTC 
(In reply to comment #2)
> Can you retest with -smp 2 without the complex topology settings?

Hi Dor,

using this cmd, met it 2 times during 12 attempts (I am testing with 100 rounds, if needed, I will update bz after the whole 100 tests completed. )

/home/staf-kvm-devel/autotest-devel/client/tests/kvm/qemu -name 'vm1' -monitor unix:'/tmp/monitor-humanmonitor1-20111130-111255-Uapa',server,nowait -serial unix:'/tmp/serial-20111130-111255-Uapa',server,nowait -drive file='/home/staf-kvm-devel/autotest-devel/client/tests/kvm/images/RHEL-Server-5.7-32.qcow2',index=0,if=ide,media=disk,cache=none,format=qcow2 -net nic,vlan=0,model=e1000,macaddr='9a:2b:06:a6:95:23' -net tap,vlan=0,fd=21 \
\
\
 -m 2048 -smp 2 \
\
\
-drive file='/home/staf-kvm-devel/autotest-devel/client/tests/kvm/isos/linux/RHEL-Server-5.7-i386-DVD.iso',media=cdrom,index=1 -drive file='/home/staf-kvm-devel/autotest-devel/client/tests/kvm/images/rhel57-32/ks.iso',media=cdrom,index=2 -cpu qemu64,+sse2 -soundhw ac97 -kernel '/home/staf-kvm-devel/autotest-devel/client/tests/kvm/images/rhel57-32/vmlinuz' -initrd '/home/staf-kvm-devel/autotest-devel/client/tests/kvm/images/rhel57-32/initrd.img' -spice port=8000,disable-ticketing -qxl 1 -rtc-td-hack -boot d   -no-kvm-pit-reinjection --append 'ks=cdrom nicdelay=60 console=ttyS0,115200 console=tty0'  -M rhel5.6.0 -usbdevice tablet
Comment 5 Siyuan Wang 2012-01-18 04:49:54 UTC 
I also encountered this bug. 
part of the serial output is:

2012-01-17 15:52:47: ACPI: Core revision 20060707
2012-01-17 15:52:47: CPU0: AMD QEMU Virtual CPU version 0.9.1 stepping 03
2012-01-17 15:52:47: Total of 1 processors activated (2004.81 BogoMIPS).
2012-01-17 15:52:47: ENABLING IO-APIC IRQs
2012-01-17 15:52:47: ..TIMER: vector=0x31 apic1=0 pin1=0 apic2=-1 pin2=-1
2012-01-17 15:52:47: ..MP-BIOS bug: 8254 timer not connected to IO-APIC
2012-01-17 15:52:47: ...trying to set up timer (IRQ0) through the 8259A ...  failed.
2012-01-17 15:52:47: ...trying to set up timer as Virtual Wire IRQ... failed.
2012-01-17 15:52:47: ...trying to set up timer as ExtINT IRQ... failed :(.
2012-01-17 15:52:47: Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option

I add an attachment of the full serial output.
Comment 6 Siyuan Wang 2012-01-18 04:50:59 UTC 
Created attachment 555912 [details]
full output of serial
Comment 7 RHEL Program Management 2012-04-02 10:42:49 UTC 
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux release.  Product Management has
requested further review of this request by Red Hat Engineering, for
potential inclusion in a Red Hat Enterprise Linux release for currently
deployed products.  This request is not yet committed for inclusion in
a release.
Comment 8 Ademar Reis 2012-04-09 22:35:25 UTC 
Did it ever happened on RHEL6? Since this happens on about 10% of the tries on a specific hardware, I recommend closing it as WONTFIX for RHEL5.

Please confirm if this bug happens on RHEL6 (using the same hardware and same command-line options just to make sure).
Comment 11 Amos Kong 2012-04-18 05:05:06 UTC 
talked with xwei, this problem only exists in amd host, and could not reproduce on rhel6 host.
Comment 13 Xiaoqing Wei 2012-04-18 07:59:42 UTC 
(In reply to comment #11)
> talked with xwei, this problem only exists in amd host, and could not reproduce
> on rhel6 host.

Yes, till now, I only met it on AMD RHEL5 host.
Comment 14 Ronen Hod 2012-04-18 13:41:01 UTC 
Closing this bug, since it specific to AMD RHEL5, and can (and should) be avoided (maybe almost avoided) by using a divider.
Added a technical note.

Thanks everybody, Ronen.
Comment 15 Ronen Hod 2012-04-18 13:41:01 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
We always recommend divider=10 and it is a must when there the guest doesn't have a kvm clock support.
Comment 18 Amit Shah 2012-04-19 08:05:56 UTC 
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-We always recommend divider=10 and it is a must when there the guest doesn't have a kvm clock support.+We always recommend the divider=10 guest kernel parameter, and it is required when the guest doesn't have kvm clock support.
Comment 20 Ronen Hod 2012-04-19 08:48:52 UTC 
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-We always recommend the divider=10 guest kernel parameter, and it is required when the guest doesn't have kvm clock support.+We recommend the divider=10 guest kernel parameter for newer RHEL5 guests, starting from RHEL5.8.
Comment 21 Dor Laor 2012-04-19 14:28:01 UTC 
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-We recommend the divider=10 guest kernel parameter for newer RHEL5 guests, starting from RHEL5.8.+We recommend the divider=10 guest kernel parameter for RHEL5 guests