Bug 647949

Summary: Wrong /sys/devices/system/cpu/cpu*/cache/index4/shared_cpu_map for L3 and L2 cache on HP Integrity BL870c box with 2 Intel Itanium2 9140N processors
Product: Red Hat Enterprise Linux 5 Reporter: Jiri Hladky <jhladky>
Component: kernelAssignee: Prarit Bhargava <prarit>
Status: CLOSED WONTFIX QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: medium Docs Contact:
Priority: low    
Version: 5.6CC: bmarson, Brice.Goglin, dshaks, jarod, mzywusko, rmusil
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-02-24 13:30:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/sys tree structure for 2.6.18-227.el5 on hp-bl870c-01.rhts.eng.bos.redhat.com box none

Description Jiri Hladky 2010-10-29 22:49:44 UTC 
Created attachment 456575 [details]
/sys tree structure for 2.6.18-227.el5 on hp-bl870c-01.rhts.eng.bos.redhat.com box

Description of problem:

We see wrong CPU topology reported by kernel on the following box:

https://beaker.engineering.redhat.com/view/hp-bl870c-01.rhts.eng.bos.redhat.com

It's HP Integrity BL870c box with two (dual socket) Intel Itanium2 9140N processors with Hyperthreading enabled. Check 
ftp://download.intel.com/products/processor/itanium/dc_prod_brief.pdf

Each socket has L3 cache shared among all cores. However,
/sys/devices/system/cpu/cpu*/cache/index4/shared_cpu_map is empty. It means that kernel does not know that l3 cache is associated with 4 cores.

Same problem with L2 cache. 
/sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_map is empty map. It's wrong.

Another problem is with size of L3 cache. The physical size of L3 is 18MB. Kernel reports only 9MB. It seems like kernel is confused with hyper-threading being enabled.


Version-Release number of selected component (if applicable):

kernel 2.6.18-227.el5

How reproducible:

Install RHEL 5.6 and check
/sys/devices/system/cpu/cpu*/cache/index4/shared_cpu_map => should contain all cores on one Socket
/sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_map => Should contain two cores (hyperthreading is enabled)
/sys/devices/system/cpu/cpu0/cache/index4/size  => should be 18MB

  
Actual results:
/sys/devices/system/cpu/cpu*/cache/index4/shared_cpu_map
Empty list (all 0s)

/sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_map
Empty list (all 0s)

/sys/devices/system/cpu/cpu0/cache/index4/size
9MB (should be 18MB)

Additional info:
Comment 1 Brice Goglin 2010-10-30 08:32:09 UTC 
Actually, I think the L3 cache in this Itanium is not shared between cores. There are two 9MB L3 caches, one per core, 18MB total. So index4/shared_cpu_map should contain 2 bits (2 threads of a core), not 4 bits (4 threads of 2 cores of a socket).
Comment 2 Jiri Hladky 2010-10-30 20:22:52 UTC 
Hi Brice,

thanks for the correction. I have been searching Intel web site for the detailed specification but without success.Finally I have found this link:
http://www.xbitlabs.com/news/cpu/display/20071031210033.html#

There are indeed 2 9MB L3 caches, one per each core, 18MB total. index4/shared_cpu_map should contain 2 bits (2 threads of a core).

Thanks
Jirka
Comment 5 Prarit Bhargava 2011-02-24 13:30:49 UTC 
No customer has complained about this so I'm inclined to leave it as is.  AFAICT, this does not impact performance.

P.