Bug 220132 - Xen kernel does not boot on Altix
Xen kernel does not boot on Altix
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel-xen (Show other bugs)
ia64 Linux
medium Severity medium
: ---
: ---
Assigned To: George Beshers
Martin Jenner
: 446594 (view as bug list)
Depends On:
Blocks: 253733 492570
  Show dependency treegraph
Reported: 2006-12-18 20:30 EST by George Beshers
Modified: 2009-04-27 08:57 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-04-27 08:57:54 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description George Beshers 2006-12-18 20:30:15 EST
Description of problem:
Xen kernel does not boot on Altix -- work is going on up stream.

Version-Release number of selected component (if applicable):

How reproducible:
Extremely :-(

Steps to Reproduce:
1. No known way to boot provided it is the Xen kernel.
Actual results:

Expected results:

Additional info:
Comment 1 George Beshers 2006-12-18 20:41:08 EST
I am putting just IA64, but actually the problems are largely NUMA related.
Comment 2 Stephen Tweedie 2007-03-13 14:50:16 EDT
OK, please update the bug if there's anything concrete you need to propose being
merged at any stage.  Thanks!
Comment 3 Red Hat Bugzilla 2007-07-24 20:43:48 EDT
change QA contact
Comment 4 George Beshers 2008-04-14 15:44:06 EDT
Just confirming that Xen does not boot on Altix, even a small 4cpu system.
Comment 6 Bill Burns 2008-05-15 06:42:11 EDT
*** Bug 446594 has been marked as a duplicate of this bug. ***
Comment 8 Jeff Burke 2008-10-20 10:57:20 EDT
   Can you please give specific details of the issue that surround the Altix platform and Xen code base in RHEL5

Comment 9 Mike Gahagan 2008-10-20 11:12:06 EDT
This issue also shows up on  hp-rx8640-03.rhts.bos.redhat.com. hp-rx8640-02 does not have this problem. My understanding is -03 is configured for numa while -02 is not.

failure log from hp-rx8640-03:

Comment 10 George Beshers 2008-10-20 13:03:30 EDT

    Its been some time since I talked with Jes about Xen as SGI
decided that KVM was in our best interest.

    My understanding is that Xen doesn't understand NUMA and this
leads to unacceptable performance degradation even for relatively
small machines, e.g., 4 nodes w/ 16 cores.  The other part of the
puzzle was handling gaps in physical memory; on Altix each node
starts at a fixed address and so there are holes between nodes
in the memory map.

    If you need more information, I will need to get Jes's help.

Comment 11 Jeff Burke 2008-10-20 13:21:41 EDT
    Thanks for the information. Could you please speak with Jes again. This seems to be more of a performance issue. These systems panic when they try and boot. Also there is conflicting information on why SGI systems can't run the xen kernel.

    It would be good for SGI to give specific reasons on what any/all the issues are.

Comment 12 Jeff Burke 2008-10-31 09:20:21 EDT
   Any update?

Comment 13 George Beshers 2008-11-03 12:16:11 EST
The following Q/As are from an internal SGI document --- please treat
them as such.

 Q: Red Hat have announced Xen on ia64 support, what about Altix?
 A: Xen/ia64 runs on DIG compliant platforms and platforms that are
    almost DIG compliant, such as the HP systems (DIG is the Intel
    reference platform). Altix is _not_ DIG compliant.

 Q: When will Xen be ready for Altix?
 A: Xen will not be ported to Altix - the long term solution /
    replacement is KVM.

 Q: Why are we not porting or providing Xen for Altix?
 A: Xen is based on a micro-kernel design. The architecture of Xen
    means that it is running on the bare hardware, ie. it's effectively
    a separate operating system, which costs almost as much to port and
    support as it costs to support Linux. In addition, Xen's designers
    have decided to re-implement most core operating system components,
    including memory management, the scheduler and hardware device
    management. It has taken years to make these components scalable
    and NUMA aware in Linux - Xen is starting from scratch.

 Q: How does KVM differ from Xen?
 A: The fundamental difference between KVM and Xen is that while Xen is
    a micro-kernel based hypervisor, KVM is a Linux kernel module which
    runs it's virtual machines as tasks under Linux. KVM therefore
    benefits from the NUMA and scalability features already available
    in Linux. The IO model is also more efficient as a KVM virtual
    machine will call straight into the host Linux kernel to have IO
    served. A Xen virtual machine will call into Xen which will then
    hand off the IO request to a privileged guest which has access to
    the physical hardware - this significantly increases IO overhead.

Not mentioned above, but my understanding is that there were political
issues with Xen's designers in that they were simply not interested in
128 processor and larger systems that SGI ships.

Ultimately the issues are scalability and resources.  A determination
was made that KVM offered far better scalability and, because it was
being introduced into the mainline kernel, would require fewer resources
to maintain in the long run.
Comment 14 Chris Lalancette 2009-04-27 08:57:54 EDT
Based on Comment #13, I'm going to close this out.  If you disagree, feel free to re-open.

Chris Lalancette

Note You need to log in before you can comment on or make changes to this bug.