Red Hat Bugzilla – Bug 220132
Xen kernel does not boot on Altix
Last modified: 2009-04-27 08:57:54 EDT
Description of problem:
Xen kernel does not boot on Altix -- work is going on up stream.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. No known way to boot provided it is the Xen kernel.
I am putting just IA64, but actually the problems are largely NUMA related.
OK, please update the bug if there's anything concrete you need to propose being
merged at any stage. Thanks!
change QA contact
Just confirming that Xen does not boot on Altix, even a small 4cpu system.
*** Bug 446594 has been marked as a duplicate of this bug. ***
Can you please give specific details of the issue that surround the Altix platform and Xen code base in RHEL5
This issue also shows up on hp-rx8640-03.rhts.bos.redhat.com. hp-rx8640-02 does not have this problem. My understanding is -03 is configured for numa while -02 is not.
failure log from hp-rx8640-03:
Its been some time since I talked with Jes about Xen as SGI
decided that KVM was in our best interest.
My understanding is that Xen doesn't understand NUMA and this
leads to unacceptable performance degradation even for relatively
small machines, e.g., 4 nodes w/ 16 cores. The other part of the
puzzle was handling gaps in physical memory; on Altix each node
starts at a fixed address and so there are holes between nodes
in the memory map.
If you need more information, I will need to get Jes's help.
Thanks for the information. Could you please speak with Jes again. This seems to be more of a performance issue. These systems panic when they try and boot. Also there is conflicting information on why SGI systems can't run the xen kernel.
It would be good for SGI to give specific reasons on what any/all the issues are.
The following Q/As are from an internal SGI document --- please treat
them as such.
Q: Red Hat have announced Xen on ia64 support, what about Altix?
A: Xen/ia64 runs on DIG compliant platforms and platforms that are
almost DIG compliant, such as the HP systems (DIG is the Intel
reference platform). Altix is _not_ DIG compliant.
Q: When will Xen be ready for Altix?
A: Xen will not be ported to Altix - the long term solution /
replacement is KVM.
Q: Why are we not porting or providing Xen for Altix?
A: Xen is based on a micro-kernel design. The architecture of Xen
means that it is running on the bare hardware, ie. it's effectively
a separate operating system, which costs almost as much to port and
support as it costs to support Linux. In addition, Xen's designers
have decided to re-implement most core operating system components,
including memory management, the scheduler and hardware device
management. It has taken years to make these components scalable
and NUMA aware in Linux - Xen is starting from scratch.
Q: How does KVM differ from Xen?
A: The fundamental difference between KVM and Xen is that while Xen is
a micro-kernel based hypervisor, KVM is a Linux kernel module which
runs it's virtual machines as tasks under Linux. KVM therefore
benefits from the NUMA and scalability features already available
in Linux. The IO model is also more efficient as a KVM virtual
machine will call straight into the host Linux kernel to have IO
served. A Xen virtual machine will call into Xen which will then
hand off the IO request to a privileged guest which has access to
the physical hardware - this significantly increases IO overhead.
Not mentioned above, but my understanding is that there were political
issues with Xen's designers in that they were simply not interested in
128 processor and larger systems that SGI ships.
Ultimately the issues are scalability and resources. A determination
was made that KVM offered far better scalability and, because it was
being introduced into the mainline kernel, would require fewer resources
to maintain in the long run.
Based on Comment #13, I'm going to close this out. If you disagree, feel free to re-open.