Bug 701251 - kernel panic when trying to install xen guest
Summary: kernel panic when trying to install xen guest
Keywords:
Status: CLOSED CANTFIX
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel-xen
Version: 5.7
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Xen Maintainance List
QA Contact: Virtualization Bugs
URL:
Whiteboard:
: 750741 (view as bug list)
Depends On:
Blocks: rhel57rtt
TreeView+ depends on / blocked
 
Reported: 2011-05-02 09:51 UTC by Alexander Todorov
Modified: 2011-11-02 09:54 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-06-09 11:57:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Alexander Todorov 2011-05-02 09:51:22 UTC
Description of problem:
I get kernel panic when trying to create a Xen guest:

# xencomm_privcmd_domctl: unknown domctl cmd 45
(XEN) ia64_fault, vector=0x4, ifa=0xf300001e21e1df78, iip=0xf00000000407da30, ipsr=0x0000121008226038, isr=0x00000a0400000000
(XEN) Alt DTLB.
(XEN) d 0xf000000007c88080 domid 0
(XEN) vcpu 0xf000000007c68000 vcpu 0
(XEN) 
(XEN) CPU 0
(XEN) psr : 0000121008226038 ifs : 800000000000040d ip  : [<f00000000407da31>]
(XEN) ip is at domain_page_flush_and_put+0x441/0x500
(XEN) unat: 0000000000000000 pfs : 000000000000040d rsc : 0000000000000003
(XEN) rnat: 0000000000000000 bsps: f0000000043a4b50 pr  : 000000000069c199
(XEN) ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c0270033f
(XEN) csd : 0000000000000000 ssd : 0000000000000000
(XEN) b0  : f00000000407da30 b6  : f0000000040b98d0 b7  : f000000004002e50
(XEN) f6  : 0ffff8000000000000000 f7  : 000000000000000000000
(XEN) f8  : 000000000000000000000 f9  : 000000000000000000000
(XEN) f10 : 000000000000000000000 f11 : 000000000000000000000
(XEN) r1  : f0000000043a4b50 r2  : 0000000000000000 r3  : 000000000000003f
(XEN) r8  : 0000000000000000 r9  : 0000000000000000 r10 : a000000100a91e50
(XEN) r11 : 0000000000000008 r12 : f000000007c6fdd0 r13 : f000000007c68000
(XEN) r14 : f000000007c60018 r15 : f300001e21e1df78 r16 : 0000000000000000
(XEN) r17 : f000000004c10480 r18 : 0000000000000000 r19 : 0000000000000001
(XEN) r20 : 0000000000000001 r21 : 0000000000000000 r22 : 0000001008226038
(XEN) r23 : 0000000000000000 r24 : 0000000000000020 r25 : 0000000000000000
(XEN) r26 : e0000100b68da0c0 r27 : 0000000000000000 r28 : a000000201150010
(XEN) r29 : 0000000000000000 r30 : 0000000000000000 r31 : f0000000041bc580
(XEN) 
(XEN) Call Trace:
(XEN)  [<f0000000040c0530>] show_stack+0x80/0xa0
(XEN)                                 sp=f000000007c6fa00 bsp=f000000007c695e8
(XEN)  [<f000000004088430>] ia64_fault+0x9e0/0xbb0
(XEN)                                 sp=f000000007c6fbd0 bsp=f000000007c695a8
(XEN)  [<f0000000040b9320>] ia64_leave_kernel+0x0/0x300
(XEN)                                 sp=f000000007c6fbd0 bsp=f000000007c695a8
(XEN)  [<f00000000407da30>] domain_page_flush_and_put+0x440/0x500
(XEN)                                 sp=f000000007c6fdd0 bsp=f000000007c69540
(XEN)  [<f000000004081970>] __dom0vp_add_physmap+0x330/0x630
(XEN)                                 sp=f000000007c6fde0 bsp=f000000007c694d8
(XEN)  [<f00000000405fa70>] do_dom0vp_op+0x1f0/0x560
(XEN)                                 sp=f000000007c6fdf0 bsp=f000000007c69498
(XEN)  [<f000000004002e80>] fast_hypercall+0x170/0x2f0
(XEN)                                 sp=f000000007c6fe00 bsp=f000000007c69498
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Fault in Xen.
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...


Version-Release number of selected component (if applicable):
kernel-xen-2.6.18-259.el5
xen-3.0.3-130.el5


How reproducible:
2/2 on particular hardware

Steps to Reproduce:
1. Create a Xen FV guest from boot.iso on particular hardware
2. Leave default selection in virt-manager and proceed to the last step
3.
  
Actual results:
Kernel panic

Expected results:
No panic, guest is created.

Additional info:
This looks like bug #431001. 

Also https://bugzilla.redhat.com/show_bug.cgi?id=431001#c20 says:
<quote>
(ia64) If your system has less than 2GB of memory, a kernel panic may occur (on
the host's kernel) if you attempt to create a guest. When this occurs, use the
kernel parameter dom0_mem=1G on the hypervisor kernel before retrying.
</quote>

My system has some more and I haven't changed the boot command line parameters.

[root@hp-rx2660-04 ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          3757        976       2780          0         48        438
-/+ buffers/cache:        488       3268
Swap:         5951          0       5951
[root@hp-rx2660-04 ~]# setenforce 0

Comment 2 Andrew Jones 2011-05-02 10:22:33 UTC
How much memory are you trying to give the guest? Can you try adding dom0_mem=1G to the command line to see what happens? This backtrace really does match the one in the BZ you've pointed to. Maybe the patch for that bug only hid the problem, or we need to make another hacky bump of spare_hv_pages...

Comment 3 Alexander Todorov 2011-05-02 12:03:35 UTC
Test 1:
After the above crash the system rebooted. I logged in and tried to create xen guest:
 - FV, Generic/Generic with 512MB of memory (the default) and the with 1 GB. In both cases I got:

(XEN) No enough contiguous memory(16384KB) for init_domain_vhpt


and virt-manager refused to create the guest.


Test 2: 
Try to create a Xen guest: FV, Linux/RHEL 5.4 or later => kernel panic


Test 3:
Added dom0_mem=1G to the boot command line => successfully created a guest with 512MB of memory.



So I only get the kernel panic when I select for the guest to use the virtio drivers.

Comment 4 Laszlo Ersek 2011-05-02 16:39:23 UTC
Xen sort of never worked on this machine (hp-rx2660-04). Please open bug 661989 and read all the comments that contain the string "hp-rx2660-04" to see my earlier testing.

I don't know why it crashes, but it is not a regression. (See in particular bug 661989 comment 27.) I think it is some hardware trait / config that Xen chokes on (even though it's probably not the NUMA config).

Matt, may I ask if there's anything special about hp-rx2660-04 that prevents Xen from working? Thanks!

Comment 5 Matt Brodeur 2011-05-02 17:13:11 UTC
(In reply to comment #4)
> 
> I don't know why it crashes, but it is not a regression. (See in particular bug
> 661989 comment 27.) I think it is some hardware trait / config that Xen chokes
> on (even though it's probably not the NUMA config).
> 
> Matt, may I ask if there's anything special about hp-rx2660-04 that prevents
> Xen from working? Thanks!

Not as far as I know.  I would expect any rx2660 to be Xen capable, but there could be something wrong with that one.  If you can determine that this panic is specific to that host I'll get our HP expert to dig into it.

Comment 6 Laszlo Ersek 2011-05-02 19:59:59 UTC
(In reply to comment #5)
> (In reply to comment #4)

> > Matt, may I ask if there's anything special about hp-rx2660-04 that prevents
> > Xen from working? Thanks!
> 
> Not as far as I know.  I would expect any rx2660 to be Xen capable, but there
> could be something wrong with that one.  If you can determine that this panic
> is specific to that host I'll get our HP expert to dig into it.

That would be great, thanks!

Based on the

    (XEN) No enough contiguous memory(16384KB) for init_domain_vhpt

messages, perhaps it is still a memory layout problem on hp-rx2660-04.

The problem does seem to be unique to this machine. Under bug 661989 the following machines worked okay with Xen:

hp-rx2660-03.rhts.eng.bos.redhat.com (bug 661989 comment 19)
hp-rx2660-01.rhts.eng.brq.redhat.com (bug 661989 comment 27)

Comment 14 Laszlo Ersek 2011-11-02 09:54:15 UTC
*** Bug 750741 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.