Bug 751205

Summary:	cpu_socket(s) facts value occasionally differs from value reported by lscpu (which is correct?)
Product:	Red Hat Enterprise Linux 7	Reporter:	John Sefler <jsefler>
Component:	subscription-manager	Assignee:	candlepin-bugs
Status:	CLOSED CURRENTRELEASE	QA Contact:	Entitlement Bugs <entitlement-bugs>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	7.0	CC:	alikins, bkearney, jgalipea, jofernan
Target Milestone:	beta
Target Release:	7.0
Hardware:	ppc64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-06-13 11:01:10 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	756082, 863175

Description John Sefler 2011-11-03 20:32:15 UTC

Description of problem:
** This bug appears to be very hardware dependent.  Usually on rhel6, lscpu socket value matches the subscription-manager fact for cpu_socket(s) **

As demonstrated below, subscription-manager facts is reporting a different value for cpu_socket(s) than what is reported by lscpu on a ppc64 provision of rhel62 server using beaker hardware https://beaker.engineering.redhat.com/view/ibm-js22-vios-01-lp1.rhts.eng.bos.redhat.com...

**  **  **  **  **  **  **  **  **  **  **  **  **  **  **  **  **  **
                 This System is reserved by jsefler.

 To return this system early. You can run the command: return2beaker.sh
  Ensure you have your logs off the system before returning to Beaker

 To extend your reservation time. You can run the command:
  extendtesttime.sh
 This is an interactive script. You will be prompted for how many
  hours you would like to extend the reservation.
  Please use this command responsibly, Everyone uses these machines.

 You should verify the watchdog was updated succesfully after
  you extend your reservation.
  https://beaker.engineering.redhat.com/recipes/316993

 For ssh, kvm, serial and power control operations please look here:
  https://beaker.engineering.redhat.com/view/ibm-js22-vios-01-lp1.rhts.eng.bos.redhat.com

      Beaker Test information:
                         HOSTNAME=ibm-js22-vios-01-lp1.rhts.eng.bos.redhat.com
                            JOBID=152032
                         RECIPEID=316993
                    RESULT_SERVER=127.0.0.1:7093
                           DISTRO=RHEL6.2-20111101.n.0
                     ARCHITECTURE=ppc64
**  **  **  **  **  **  **  **  **  **  **  **  **  **  **  **  **  **
[root@ibm-js22-vios-01-lp1 ~]# subscription-manager facts --list | grep cpu_socket
cpu.cpu_socket(s): 1
lscpu.cpu_socket(s): 4


^^^ I would have expected those two values to match.  Subscription-manager uses the "cpu.cpu_socket(s)" for filtering which means that a providing subscription that serves 2 socket hardware would err on the side of subscription availability when it would normally not be available (assuming 4 sockets is correct and real). Hence this failing instance favors the customer.


Version-Release number of selected component (if applicable):
[root@ibm-js22-vios-01-lp1 ~]# rpm -q subscription-manager
subscription-manager-0.96.17-1.el6.ppc64

Repeatability:
does not happen on every hardware instance, but it is happening on ibm-js22-vios-01-lp1.rhts.eng.bos.redhat.com

Additional info:
[root@ibm-js22-vios-01-lp1 ~]# lscpu | grep socket
Core(s) per socket:    1
CPU socket(s):         4
[root@ibm-js22-vios-01-lp1 ~]# lscpu
Architecture:          ppc64
Byte Order:            Big Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    1
CPU socket(s):         4
NUMA node(s):          2
Model:                 IBM,7998-61X
L1d cache:             64K
L1i cache:             64K
NUMA node0 CPU(s):     0-7
NUMA node1 CPU(s):     
[root@ibm-js22-vios-01-lp1 ~]#

Comment 1 Adrian Likins 2011-11-03 20:43:29 UTC

The builtin socket detection code is failing. In particular, it is failing
because the /sys/devices/system/cpu/cpu0/topology/physical_package_id
is always reporting "-1". The code assumes that we are going to get a unique 
physical_package_id for each socket. In this case, it thinks there is just
one socket with an id of -1 and all the cores are in it. 


[root@ibm-js22-vios-01-lp1 ~]# cat /sys/devices/system/cpu/cpu{0,1,2,3,4,5,6,7}/topology/physical_package_id
-1
-1
-1
-1
-1
-1
-1
-1


Looking through lscpu code now to see what it's doing different that it
is handling this correctly.

Comment 3 John Sefler 2011-11-04 17:34:22 UTC

found another system (s390x guest) with the same problem...

[root@ibm-z10-09 ~]# rpm -q subscription-manager
subscription-manager-0.96.17-1.el6.s390x

[root@ibm-z10-09 ~]# subscription-manager facts --list | grep cpu_socket
cpu.cpu_socket(s): 1
lscpu.cpu_socket(s): 2

[root@ibm-z10-09 ~]# ls /sys/devices/system/cpu | grep cpu
cpu0
cpu1

[root@ibm-z10-09 ~]# cat /sys/devices/system/cpu/cpu{0,1}/topology/physical_package_id
-1
-1

^^^ lscpu is reporting 2 sockets and subscription-manager is reporting 1 sockets



Additional Info:

[root@ibm-z10-09 ~]# lscpu
Architecture:          s390x
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Big Endian
CPU(s):                2
On-line CPU(s) list:   0,1
Thread(s) per core:    1
Core(s) per socket:    1
CPU socket(s):         2
Vendor ID:             IBM/S390

[root@ibm-z10-09 ~]# subscription-manager facts --list
cpu.core(s)_per_socket: 2
cpu.cpu(s): 2
cpu.cpu_socket(s): 1
distribution.id: Santiago
distribution.name: Red Hat Enterprise Linux Server
distribution.version: 6.2
lscpu.architecture: s390x
lscpu.byte_order: Big Endian
lscpu.core(s)_per_socket: 1
lscpu.cpu(s): 2
lscpu.cpu_op-mode(s): 32-bit, 64-bit
lscpu.cpu_socket(s): 2
lscpu.on-line_cpu(s)_list: 0,1
lscpu.thread(s)_per_core: 1
lscpu.vendor_id: IBM/S390
memory.memtotal: 503432
memory.swaptotal: 1015800
net.interface.eth0.broadcast: 10.16.71.255
net.interface.eth0.hwaddr: 02:de:ad:be:ef:09
net.interface.eth0.ipaddr: 10.16.66.200
net.interface.eth0.netmask: 255.255.248.0
net.interface.lo.broadcast: 0.0.0.0
net.interface.lo.hwaddr: 00:00:00:00:00:00
net.interface.lo.ipaddr: 127.0.0.1
net.interface.lo.netmask: 255.0.0.0
network.hostname: ibm-z10-09.rhts.eng.bos.redhat.com
network.ipaddr: 10.16.66.200
system.entitlements_valid: False
system.name: ibm-z10-09.rhts.eng.bos.redhat.com
system.uuid: baeaf4ba-1c33-46ed-95b2-d87434da9d9f
uname.machine: s390x
uname.nodename: ibm-z10-09.rhts.eng.bos.redhat.com
uname.release: 2.6.32-215.el6.s390x
uname.sysname: Linux
uname.version: #1 SMP Fri Oct 28 18:21:27 EDT 2011
virt.host_type: ibm_systemz
ibm_systemz-zvm
virt.is_guest: True
virt.uuid: Unknown
[root@ibm-z10-09 ~]#

Comment 7 John Sefler 2012-04-09 22:31:23 UTC

Additional Info: In bug 805415, an s390x machine also demonstrates that a discrepancy in cpu_sockets is sometimes reported...
[root@ibm-z10-30 ~]# subscription-manager facts --list | grep cpu_socket
cpu.cpu_socket(s): 1
lscpu.cpu_socket(s): 2

Comment 9 RHEL Program Management 2012-07-10 06:58:18 UTC

This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Comment 10 RHEL Program Management 2012-07-11 02:08:17 UTC

This request was erroneously removed from consideration in Red Hat Enterprise Linux 6.4, which is currently under development.  This request will be evaluated for inclusion in Red Hat Enterprise Linux 6.4.

Comment 11 RHEL Program Management 2012-12-14 07:24:13 UTC

This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Comment 12 John Sefler 2013-04-23 14:56:25 UTC

Here's another machine with this problem.  (RHEL64 running on ppc64)
https://beaker.engineering.redhat.com/view/ibm-p730-04-lp4.rhts.eng.bos.redhat.com

[root@ibm-p730-04-lp4 ~]# rpm -q subscription-manager
subscription-manager-1.8.6-1.el6.ppc64

[root@ibm-p730-04-lp4 ~]# subscription-manager facts --list | grep "socket(s)"
cpu.cpu_socket(s): 1
lscpu.socket(s): 7

[root@ibm-p730-04-lp4 ~]# lspcu
-bash: lspcu: command not found
[root@ibm-p730-04-lp4 ~]# lscpu
Architecture:          ppc64
Byte Order:            Big Endian
CPU(s):                28
On-line CPU(s) list:   0-27
Thread(s) per core:    4
Core(s) per socket:    1
Socket(s):             7
NUMA node(s):          2
Model:                 IBM,8231-E2B
L1d cache:             32K
L1i cache:             32K
NUMA node0 CPU(s):     0-27
NUMA node1 CPU(s):     

[root@ibm-p730-04-lp4 ~]# for cpu in `ls -1 /sys/devices/system/cpu/ | egrep cpu[[:digit:]]`; do echo "cpu `cat /sys/devices/system/cpu/$cpu/topology/physical_package_id`"; done | grep cpu | uniq | wc -l
1


^^^ lscpu is reporting 7 sockets and subscription-manager is reporting 1 sockets


The consequence of this situation is that systems are being reported as compliant when they should not be.  For example, this system with 7 sockets (as reported by lscpu) shows the system to be "Subscribed" after attaching a "4 socket RHEL subscription" when it should really be "Partially Subscribed".

[root@ibm-p730-04-lp4 ~]# subscription-manager list --consumed
+-------------------------------------------+
   Consumed Subscriptions
+-------------------------------------------+

Subscription Name: Red Hat Enterprise Linux for IBM POWER, Standard (4 sockets) (Up to 30 LPARs) with Smart Management
Provides:          Red Hat Enterprise Linux for IBM POWER
SKU:               RH0380468
Contract:          10014721
Account:           5206749
Serial Number:     3378912489728694267
Pool ID:           Not Available
Active:            True
Quantity Used:     1
Service Level:     Standard
Service Type:      L1-L3
Starts:            12/31/2012
Ends:              12/31/2013

[root@ibm-p730-04-lp4 ~]# subscription-manager list --installed
+-------------------------------------------+
    Installed Product Status
+-------------------------------------------+
Product Name: Red Hat Enterprise Linux for IBM POWER
Product ID:   74
Version:      6.4
Arch:         ppc64
Status:       Subscribed              <============ SHOULD BE "Partially Subscribed"
Starts:       12/31/2012
Ends:         12/31/2013

[root@ibm-p730-04-lp4 ~]#

Comment 13 John Sefler 2013-04-23 14:57:51 UTC

This problem seems to occur more frequently on ppc64 and s390x architectures.

Comment 14 Adrian Likins 2014-01-31 00:46:56 UTC

(In reply to John Sefler from comment #12)
> Here's another machine with this problem.  (RHEL64 running on ppc64)
> https://beaker.engineering.redhat.com/view/ibm-p730-04-lp4.rhts.eng.bos.
> redhat.com
> 
Looks like recent ish subman matches lscpu output.

The beaker page says:  28 cpus, 0 cores, 0 sockets
lscpu:                 28 cpus, 7 cores, 7 sockets
subman-1.9.11-1.el6    28 cpus, 7 cores, 7 sockets (they match now!)

I'm gonna consider that works for me.

Comment 15 John Sefler 2014-01-31 12:59:10 UTC

Re-testing offending s390 machine ibm-z10-30.rhts.eng.bos.redhat.com from comment 7 with rhel70....

[root@ibm-z10-30 ~]# subscription-manager version
server type: This system is currently not registered.
subscription management server: Unknown
subscription-manager: 1.10.11-2.el7
python-rhsm: 1.10.11-2.el7

[root@ibm-z10-30 ~]# subscription-manager facts | grep cpu
cpu.book(s): 2
cpu.book(s)_per_cpu: 1
cpu.core(s)_per_socket: 1
cpu.cpu(s): 2
cpu.cpu_socket(s): 2
cpu.socket(s)_per_book: 1
cpu.thread(s)_per_core: 1
cpu.topology_source: s390 book_siblings_list
lscpu.architecture: s390x
lscpu.bogomips: 2913.00
lscpu.book(s): 2
lscpu.byte_order: Big Endian
lscpu.core(s)_per_socket: 1
lscpu.cpu(s): 2
lscpu.cpu_op-mode(s): 32-bit, 64-bit
lscpu.dispatching_mode: horizontal
lscpu.hypervisor: z/VM 6.1.0
lscpu.hypervisor_vendor: IBM
lscpu.l1d_cache: 96K
lscpu.l1i_cache: 64K
lscpu.l2d_cache: 1024K
lscpu.l2i_cache: 1024K
lscpu.on-line_cpu(s)_list: 0,1
lscpu.socket(s)_per_book: 1
lscpu.thread(s)_per_core: 1
lscpu.vendor_id: IBM/S390
lscpu.virtualization_type: full
[root@ibm-z10-30 ~]# 

VERIFIED: The system fact "cpu.cpu_socket(s): 2" used by subscription-manager for socket compliance matches the value calculated by lscpu which equates to:
  lscpu.book(s):2 * lscpu.socket(s)_per_book:1 = 2

Comment 17 John Sefler 2014-02-11 15:13:24 UTC

Testing another s390 machine ibm-z10-74.rhts.eng.bos.redhat.com with rhel70....

[root@ibm-z10-74 ~]# rpm -q subscription-manager
subscription-manager-1.10.13-1.el7.s390x

[root@ibm-z10-74 ~]# subscription-manager facts | grep cpu
cpu.book(s): 2
cpu.book(s)_per_cpu: 1
cpu.core(s)_per_socket: 1
cpu.cpu(s): 2
cpu.cpu_socket(s): 2
cpu.socket(s)_per_book: 1
cpu.thread(s)_per_core: 1
cpu.topology_source: s390 book_siblings_list
lscpu.architecture: s390x
lscpu.bogomips: 2913.00
lscpu.book(s): 2
lscpu.byte_order: Big Endian
lscpu.core(s)_per_socket: 1
lscpu.cpu(s): 2
lscpu.cpu_op-mode(s): 32-bit, 64-bit
lscpu.dispatching_mode: horizontal
lscpu.hypervisor: z/VM 6.1.0
lscpu.hypervisor_vendor: IBM
lscpu.l1d_cache: 96K
lscpu.l1i_cache: 64K
lscpu.l2d_cache: 1024K
lscpu.l2i_cache: 1024K
lscpu.on-line_cpu(s)_list: 0,1
lscpu.socket(s)_per_book: 1
lscpu.thread(s)_per_core: 1
lscpu.vendor_id: IBM/S390
lscpu.virtualization_type: full

VERIFIED: The system fact "cpu.cpu_socket(s): 2" used by subscription-manager for socket compliance matches the value calculated by lscpu which equates to:
  lscpu.book(s):2 * lscpu.socket(s)_per_book:1 = 2

VERIFIED: The system fact for calculated cores also matches the lscpu cores which is now used for compliance of virtual guest systems using subscriptions defined with a "vcpu" attribute.
  cpu.core(s)_per_socket: 1 * cpu.cpu_socket(s): 2 = lscpu.core(s)_per_socket: 1 * lscpu.socket(s)_per_book: 1 * lscpu.book(s): 2

Comment 18 John Sefler 2014-02-11 15:19:59 UTC

Testing ppc64 machine ibm-p720-01-lp2.rhts.eng.bos.redhat.com with rhel70...

[root@ibm-p720-01-lp2 ~]# rpm -q subscription-manager
subscription-manager-1.10.13-1.el7.ppc64

[root@ibm-p720-01-lp2 ~]# subscription-manager facts | grep cpu
cpu.core(s)_per_socket: 1
cpu.cpu(s): 28
cpu.cpu_socket(s): 7
cpu.thread(s)_per_core: 4
cpu.topology_source: kernel /sys cpu sibling lists
lscpu.architecture: ppc64
lscpu.byte_order: Big Endian
lscpu.core(s)_per_socket: 1
lscpu.cpu(s): 28
lscpu.cpu_op-mode(s): 32-bit, 64-bit
lscpu.l1d_cache: 32K
lscpu.l1i_cache: 32K
lscpu.model: IBM,8202-E4B
lscpu.numa_node(s): 1
lscpu.numa_node0_cpu(s): 0-27
lscpu.on-line_cpu(s)_list: 0-27
lscpu.socket(s): 7
lscpu.thread(s)_per_core: 4

VERIFIED: The system fact "cpu.cpu_socket(s): 7" used by subscription-manager for socket compliance of physical systems matches the value reported by lscpu "lscpu.socket(s): 7"

VERIFIED: The system fact for calculated cores also matches the lscpu cores which equates to:
  cpu.core(s)_per_socket: 1 * cpu.cpu_socket(s): 7 = lscpu.core(s)_per_socket: 1 * lscpu.socket(s): 7

Comment 19 John Sefler 2014-02-11 19:58:35 UTC

Re-testing offending ppc64 machine ibm-p730-04-lp4.rhts.eng.bos.redhat.com from comment 12 with rhel70....

[root@ibm-p730-04-lp4 ~]# rpm -q subscription-manager
subscription-manager-1.10.13-1.el7.ppc64

[root@ibm-p730-04-lp4 ~]# subscription-manager facts | grep cpu
cpu.core(s)_per_socket: 1
cpu.cpu(s): 28
cpu.cpu_socket(s): 7
cpu.thread(s)_per_core: 4
cpu.topology_source: kernel /sys cpu sibling lists
lscpu.architecture: ppc64
lscpu.byte_order: Big Endian
lscpu.core(s)_per_socket: 1
lscpu.cpu(s): 28
lscpu.cpu_op-mode(s): 32-bit, 64-bit
lscpu.l1d_cache: 32K
lscpu.l1i_cache: 32K
lscpu.model: IBM,8231-E2B
lscpu.numa_node(s): 2
lscpu.numa_node0_cpu(s): 0-27
lscpu.numa_node1_cpu(s): Unknown
lscpu.on-line_cpu(s)_list: 0-27
lscpu.socket(s): 7
lscpu.thread(s)_per_core: 4

VERIFIED: The system fact "cpu.cpu_socket(s): 7" used by subscription-manager for socket compliance of physical systems matches the value reported by lscpu "lscpu.socket(s): 7"

VERIFIED: The system fact for calculated cores also matches the lscpu cores which is now used for compliance of virtual guest systems using subscriptions defined with a "vcpu" attribute.
  cpu.core(s)_per_socket: 1 * cpu.cpu_socket(s): 7 = lscpu.core(s)_per_socket: 1 * lscpu.socket(s): 7

Comment 20 Ludek Smid 2014-06-13 11:01:10 UTC

This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.