Bug 219951
Summary: | Xen performance on NUMA systems is fairly poor | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 5 | Reporter: | Chris Lalancette <clalance> | ||||
Component: | kernel-xen | Assignee: | Xen Maintainance List <xen-maint> | ||||
Status: | CLOSED CURRENTRELEASE | QA Contact: | |||||
Severity: | medium | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | 5.0 | CC: | ddomingo, konradr, tao | ||||
Target Milestone: | --- | ||||||
Target Release: | --- | ||||||
Hardware: | All | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | 5.0.0 | Doc Type: | Bug Fix | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2007-01-19 13:21:08 UTC | Type: | --- | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 197865, 225424, 234654 | ||||||
Attachments: |
|
Description
Chris Lalancette
2006-12-17 17:04:05 UTC
From IBM: I've finished the port of the 4 changesets from xen-unstable needed to enable NUMA support in the Xen hypervisor. The work involved the following: Acquiring the relavent patches from xen-unstable.hg: wget -O xen-numa-01-11893.patch "http://xenbits.xensource.com/xen-unstable.hg?cs=f312c2d01d8b;style=raw" wget -O xen-numa-02-11894.patch "http://xenbits.xensource.com/xen-unstable.hg?cs=cf95c3218a70;style=raw" wget -O xen-numa-03-11895.patch "http://xenbits.xensource.com/xen-unstable.hg?cs=041507e2754c;style=raw" wget -O xen-numa-04-11920.patch "http://xenbits.xensource.com/xen-unstable.hg?cs=3db344ea8070;style=raw" Installing the kernel source rpm for 12/07 drop: rpm -ivh kernel-2.6.18-1.2839.el5.src.rpm Putting the patches in /usr/src/redhat/SOURCES Modifying the spec file to patch in the numa patches Rebuilding the kernel package: rpmbuild --target x86_64 --define='dist el5' -ba kernel-2.6.spec The resultant kernel-xen rpm was installed: rpm -Uvh kernel-xen-2.6.18-1.2840.el5.x86_64.rpm on my test machine (2-way opteron which has SRAT tables) and I've test booted it and it appears to function. I'll be attaching the binary kernel which can be used for testing, the modifications to the specfile which were needed for building xen with the numa patches, the source rpm which has all of the changes contained with in it, and the xm dmesg output which shows the NUMA code functioning. Created attachment 143869 [details]
Spec diff to add NUMA support to RHEL-5
Chris, kindly post the text that IBM needs to be included in the release notes. note that the deadline for Final Release Notes is on 19 December 2006. making this bug block release notes, for tracking purposes. thanks! Cheers, Don Don, Something like the following should work for the release notes: ------------------------------------------------------------------------------- The hypervisor shipped in RHEL-5 GA is not NUMA aware; therefore, it's performance on NUMA machines may be sub-optimal. This issue will be addressed in a future RHEL-5 update. -------------------------------------------------------------------------------- Yeah, yeah, I know it's kind of lame but I don't know what the workaround is right now; maybe setting node interleaving would help, but I haven't tested so I can't say for sure. I'll ask IBM if this is a valid workaround, but I probably won't hear before the deadline. Anyway, the above is at least something. Chris Lalancette We should probably actually recommend that the admins turn on 'memory node interleaving' in the BIOS for NUMA systems. This will ensure there is more consistent performance, by making the memory round-robin interleave in 4k blocks. added to release notes: <quote> At present, the Virtualization feature has the following limitations: <snip> * The hypervisor included in this release of Virtualization is not NUMA-aware; as such, its performance on NUMA machines may be sub-optimal. This will be addressed in a future update of Red Hat Enterprise Linux 4.92. To work around this, you can enable memory node interleaving in the NUMA machine's BIOS. This ensures a more consistent performance. </quote> Closing out . . . please ensure that a RHEL5.1 bug is entered to track resolution of the original issue. This BZ is closed. Is that the correct state it should be in? |