Bug 539684

in kernel-2.6.18-164.7.1.el5

Please do NOT transition this bugzilla state to VERIFIED until our QE team
has sent specific instructions indicating when to do so.

From Comment #8 in
https://bugzilla.redhat.com/show_bug.cgi?id=526051
we know that this bug cannot be reproduced everytime. I try testing it on amd-dinar-04.lab.bos.redhat.com(x86 dom0, numa=on, K10 platform) and cannot reproduce it.

(In reply to comment #3)
> From Comment #8 in
> https://bugzilla.redhat.com/show_bug.cgi?id=526051
> we know that this bug cannot be reproduced everytime. I try testing it on
> amd-dinar-04.lab.bos.redhat.com(x86 dom0, numa=on, K10 platform) and cannot
> reproduce it.  

Hm, I'm not sure we should give up so easily.  I'm under the impression that with the right hardware, this can be reproduced every single time.  Bhavna, what hardware do we need to be able to reproduce this and then confirm the fix?

Chris Lalancette

(In reply to comment #4)

Borrow comment #5 from BZ526051
> Ah, cool, good stuff.  Interestingly, I booted my Barcelona with exactly the
> same parameters this morning (i386 dom0, numa=on), and it booted just fine. 
> Maybe it has to do with how the memory is laid out, I'm not sure.  In any case,
> good that we have the fix now.
> 
> Chris Lalancette  

Just FYI, numa support isn't configured into the i386 brew kernel by default.

(In reply to comment #3)
> From Comment #8 in
> https://bugzilla.redhat.com/show_bug.cgi?id=526051
> we know that this bug cannot be reproduced everytime. I try testing it on
> amd-dinar-04.lab.bos.redhat.com(x86 dom0, numa=on, K10 platform) and cannot
> reproduce it.  

I think I did a wrong test on the machine(amd-dinar-04.lab.bos.redhat.com) for knowing little about numa. Today I retest it on the same machine and reproduced it with kernel-xen-2.6.18-164.6.1.el5.x86_64.rpm.

(In reply to comment #7)
> (In reply to comment #3)
> > From Comment #8 in
> > https://bugzilla.redhat.com/show_bug.cgi?id=526051
> > we know that this bug cannot be reproduced everytime. I try testing it on
> > amd-dinar-04.lab.bos.redhat.com(x86 dom0, numa=on, K10 platform) and cannot
> > reproduce it.  
> 
> I think I did a wrong test on the machine(amd-dinar-04.lab.bos.redhat.com) for
> knowing little about numa. Today I retest it on the same machine and reproduced
> it with kernel-xen-2.6.18-164.6.1.el5.x86_64.rpm.  

OK, great!  Just so you know...

(In reply to comment #6)
> (In reply to comment #4)
> 
> Borrow comment #5 from BZ526051
> > Ah, cool, good stuff.  Interestingly, I booted my Barcelona with exactly the
> > same parameters this morning (i386 dom0, numa=on), and it booted just fine. 
> > Maybe it has to do with how the memory is laid out, I'm not sure.  In any case,
> > good that we have the fix now.
> > 
> > Chris Lalancette  
> 
> Just FYI, numa support isn't configured into the i386 brew kernel by default.  

NUMA support *is* configured into the i386 Xen hypervisor, however.

Chris Lalancette

Some guy checked the configure items for us and it shew:
<caiqian> kernel-2.6.18-i686-PAE.config:# CONFIG_X86_NUMAQ is not set
<caiqian> kernel-2.6.18-i686-PAE.config:# CONFIG_NUMA is not set
<caiqian> kernel-2.6.18-i686-PAE.config:CONFIG_ACPI_NUMA=y
<caiqian> kernel-2.6.18-i686-xen.config:# CONFIG_X86_NUMAQ is not set
<caiqian> kernel-2.6.18-i686-xen.config:# CONFIG_NUMA is not set
<caiqian> kernel-2.6.18-i686-xen.config:CONFIG_ACPI_NUMA=y
<caiqian> kernel-2.6.18-ia64.config:CONFIG_NUMA=y
<caiqian> kernel-2.6.18-ia64.config:CONFIG_ACPI_NUMA=y
...
From that probably we can guess it isn't supported in i386 Xen hypervisor either.

> Just FYI, numa support isn't configured into the i386 brew kernel by default.  

It does not need to be configured in the kernels. NUMA is managed exclusively by the Xen hypervisor, neither Dom0, nor DomU gets to see any NUMA topology when running on the hypervisor.


> <caiqian> kernel-2.6.18-i686-xen.config:# CONFIG_X86_NUMAQ is not set
> <caiqian> kernel-2.6.18-i686-xen.config:# CONFIG_NUMA is not set
> <caiqian> kernel-2.6.18-i686-xen.config:CONFIG_ACPI_NUMA=y
> <caiqian> kernel-2.6.18-ia64.config:CONFIG_NUMA=y
> <caiqian> kernel-2.6.18-ia64.config:CONFIG_ACPI_NUMA=y
> ...
> From that probably we can guess it isn't supported in i386 Xen hypervisor
> either.  

AFAIK, the  .config settings have *no* effect on the way the hypervisor is built. The hypervisor has very few compile config settings, pretty much everything is on by default all the time.

Many thanks to Danie and Chris. I can verify this bus on kernel-2.6.18-164.8.1.el5xen now. After booting successfully, I check numa with virsh cmd. It shows:
[root@amd-dinar-04 boot]# virsh  nodeinfo
CPU model:           x86_64
CPU(s):              24
CPU frequency:       2094 MHz
CPU socket(s):       1
Core(s) per socket:  12
Thread(s) per core:  1
NUMA cell(s):        4
Memory size:         7337984 kB

And I cannot check numa in usual ways.
Then is this ok?

Igor.

(In reply to comment #11)
> Many thanks to Danie and Chris. I can verify this bus on
> kernel-2.6.18-164.8.1.el5xen now. After booting successfully, I check numa with
> virsh cmd. It shows:
> [root@amd-dinar-04 boot]# virsh  nodeinfo
> CPU model:           x86_64
> CPU(s):              24
> CPU frequency:       2094 MHz
> CPU socket(s):       1
> Core(s) per socket:  12
> Thread(s) per core:  1
> NUMA cell(s):        4
> Memory size:         7337984 kB
> 
> And I cannot check numa in usual ways.
> Then is this ok?

That looks reasonable to me.  The fact that the system did not crash, and that it is showing NUMA cell(s): 4 seems to say it is working.  You can do one final test and get the output from "xm info" which will show a bit more detailed information, but I would say this is VERIFIED.

Chris Lalancette

Yes, it doesn't crash. I just want to ensure numa is working normally when it can be bootstraped. Final test shows:
[root@amd-dinar-04 boot]# xm info
host                   : amd-dinar-04.lab.bos.redhat.com
release                : 2.6.18-164.8.1.el5xen
version                : #1 SMP Mon Nov 23 13:10:23 EST 2009
machine                : x86_64
nr_cpus                : 24
nr_nodes               : 4
sockets_per_node       : 0
cores_per_socket       : 12
threads_per_core       : 1
cpu_mhz                : 2094
hw_caps                : 178bfbff:efd3fbff:00000000:00000110:00802009:00000000:000837ff
total_memory           : 7166
free_memory            : 383
node_to_cpu            : node0:0-5
                         node1:6-11
                         node2:18-23
                         node3:12-17
xen_major              : 3
xen_minor              : 1
xen_extra              : .2-164.8.1.el5
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
cc_compiler            : gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)
cc_compile_by          : mockbuild
cc_compile_domain      : redhat.com
cc_compile_date        : Mon Nov 23 13:00:50 EST 2009
xend_config_format     : 2

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-1670.html