RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1472167 - Guest OS does not show any NUMA nodes
Summary: Guest OS does not show any NUMA nodes
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm
Version: 7.4
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Eduardo Habkost
QA Contact: Yumei Huang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-07-18 08:24 UTC by Artyom
Modified: 2017-08-26 20:30 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-26 20:30:18 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Artyom 2017-07-18 08:24:33 UTC 
Description of problem:
I run a virtual machine in the RHEVM with next QEMU parameters:
/usr/libexec/qemu-kvm 
-name guest=golden_env_mixed_virtio_0,debug-threads=on 
-S 
-object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-389-golden_env_mixed_vir/master-key.aes 
-machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off 
-cpu Conroe 
-m size=1048576k,slots=16,maxmem=4194304k 
-realtime mlock=off 
-smp 2,maxcpus=16,sockets=16,cores=1,threads=1 
-numa node,nodeid=0,cpus=0,mem=512 
-numa node,nodeid=1,cpus=1,mem=512 
-uuid fcbe94d1-7cea-4170-852e-71627f3b1d30 
-smbios type=1,manufacturer=oVirt,product=oVirt Node,version=7.4-18.el7,serial=55157E90-C28C-11E0-983A-5CF3FC78F458,uuid=fcbe94d1-7cea-4170-852e-71627f3b1d30 
....
but when I start the VM and run numactl command it shows me only one NUMA node
# numactl -H
available: 1 nodes (0)
node 0 cpus: 0
node 0 size: 1023 MB
node 0 free: 674 MB
node distances:
node   0 
  0:  10 


Version-Release number of selected component (if applicable):
VM packages:
# uname -r
3.10.0-691.el7.x86_64
# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.4 (Maipo)
# rpm -qa | grep numa
numactl-libs-2.0.9-6.el7_2.x86_64
numactl-2.0.9-6.el7_2.x86_64

Host packages:
# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.4 (Maipo)
# uname -r
3.10.0-691.el7.x86_64
# rpm -qa | grep qemu
libvirt-daemon-driver-qemu-3.2.0-14.el7.x86_64
ipxe-roms-qemu-20170123-1.git4e85b27.el7.noarch
qemu-kvm-ev-2.6.0-28.el7.10.1.x86_64
qemu-img-ev-2.6.0-28.el7.10.1.x86_64
qemu-kvm-common-ev-2.6.0-28.el7.10.1.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create VM with above parameters
2. Start the VM
3. Check VM NUMA nodes

Actual results:
numactl -H command shows only one NUMA node

Expected results:
numactl -H command must show two NUMA nodes

Additional info:
I checked VM with the same QEMU parameters, but with different kernel and all works as expected
Working configuration:
# uname -r
3.10.0-514.10.2.el7.x86_64
# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.3 (Maipo)
# rpm -qa | grep numa
numactl-libs-2.0.9-6.el7_2.x86_64
numactl-2.0.9-6.el7_2.x86_64
============================================================================
Also I checked dmesg of problematic VM and I can see:
[    0.000000] found SMP MP-table at [mem 0x000f71a0-0x000f71af] mapped at [ffff8800000f71a0]
[    0.000000] Base memory trampoline at [ffff880000099000] 99000 size 24576
[    0.000000] BRK [0x01fe9000, 0x01fe9fff] PGTABLE
[    0.000000] BRK [0x01fea000, 0x01feafff] PGTABLE
[    0.000000] BRK [0x01feb000, 0x01febfff] PGTABLE
[    0.000000] BRK [0x01fec000, 0x01fecfff] PGTABLE
[    0.000000] RAMDISK: [mem 0x35848000-0x36c1bfff]
[    0.000000] Early table checksum verification disabled
[    0.000000] ACPI BIOS Error (bug): A valid RSDP was not found (20130517/tbxfroot-243)
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at [mem 0x0000000000000000-0x000000003fffbfff]
[    0.000000] NODE_DATA(0) allocated [mem 0x3ffd5000-0x3fffbfff]
[    0.000000] crashkernel=auto resulted in zero bytes of reserved memory.
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 0:3ff85001, primary cpu clock
[    0.000000] kvm-clock: using sched offset of 7024018732 cycles
============================================================================
Both kernels have the same NUMA configuration:
# cat config-3.10.0-691.el7.x86_64 | grep -i numa
CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
CONFIG_ARCH_WANTS_PROT_NUMA_PROT_NONE=y
CONFIG_ARCH_USES_NUMA_PROT_NONE=y
CONFIG_NUMA_BALANCING_DEFAULT_ENABLED=y
CONFIG_NUMA_BALANCING=y
# CONFIG_X86_NUMACHIP is not set
CONFIG_NUMA=y
CONFIG_AMD_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
# CONFIG_NUMA_EMU is not set
CONFIG_USE_PERCPU_NUMA_NODE_ID=y
CONFIG_ACPI_NUMA=y

# cat /boot/config-3.10.0-514.10.2.el7.x86_64 | grep -i numa
CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
CONFIG_ARCH_WANTS_PROT_NUMA_PROT_NONE=y
CONFIG_ARCH_USES_NUMA_PROT_NONE=y
CONFIG_NUMA_BALANCING_DEFAULT_ENABLED=y
CONFIG_NUMA_BALANCING=y
# CONFIG_X86_NUMACHIP is not set
CONFIG_NUMA=y
CONFIG_AMD_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
# CONFIG_NUMA_EMU is not set
CONFIG_USE_PERCPU_NUMA_NODE_ID=y
CONFIG_ACPI_NUMA=y

I am pretty sure it does not numactl bug, but I believe you will know to do propogation, because I just do not sure what component is problematic.
Comment 2 Yumei Huang 2017-07-31 08:02:00 UTC 
Hi Artyom, 
I tried and failed to reproduce this bug. Could you please help check my steps? Thanks!

Test details:
host kernel: 3.10.0-691.el7.x86_64
guest kernel: 3.10.0-691.el7.x86_64
numactl-2.0.9-6.el7_2.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.2

CMDline:
# /usr/libexec/qemu-kvm   rhel73-bak.qcow2  \

-m size=1048576k,slots=16,maxmem=4194304k  \

-smp 2,maxcpus=16,sockets=16,cores=1,threads=1 \

-numa node,nodeid=0,cpus=0,mem=512  \

-numa node,nodeid=1,cpus=1,mem=512  \

-netdev tap,id=tap0 -device virtio-net-pci,id=net0,netdev=tap0 \

-monitor stdio -vnc :1

Check numa in guest:
# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0
node 0 size: 511 MB
node 0 free: 185 MB
node 1 cpus: 1
node 1 size: 511 MB
node 1 free: 38 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10
Comment 3 Artyom 2017-07-31 08:34:32 UTC 
It looks fine, maybe the single difference that I have QEMU packages with ev suffix:
# rpm -qa | grep qemu
libvirt-daemon-driver-qemu-3.2.0-14.el7.x86_64
ipxe-roms-qemu-20170123-1.git4e85b27.el7.noarch
qemu-kvm-ev-2.6.0-28.el7.10.1.x86_64
qemu-img-ev-2.6.0-28.el7.10.1.x86_64
qemu-kvm-common-ev-2.6.0-28.el7.10.1.x86_64

I do not sure if it really relevant.

I sent you my environment details via email, so maybe you will recognize some additional difference.

Sorry not much help from me:(
Comment 4 Eduardo Habkost 2017-07-31 14:00:43 UTC 
(In reply to Artyom from comment #3)
> It looks fine, maybe the single difference that I have QEMU packages with ev
> suffix:
> # rpm -qa | grep qemu
> libvirt-daemon-driver-qemu-3.2.0-14.el7.x86_64
> ipxe-roms-qemu-20170123-1.git4e85b27.el7.noarch
> qemu-kvm-ev-2.6.0-28.el7.10.1.x86_64
> qemu-img-ev-2.6.0-28.el7.10.1.x86_64
> qemu-kvm-common-ev-2.6.0-28.el7.10.1.x86_64
> 
> I do not sure if it really relevant.

It is, as the ACPI tables are generated by QEMU and the guest is reporting broken ACPI tables.  Are you able reproduce using the equivalent qemu-kvm-rhev packages?
Comment 5 Artyom 2017-07-31 14:10:05 UTC 
I believe qemu-kvm-ev coming as part of vdsm dependencies, so it can be problematic for me to change from ev to rhev packages, also when I tried to install it with ev packages already installed I got warning message
Package 10:qemu-kvm-rhev-2.6.0-28.el7_3.9.x86_64 is obsoleted by 10:qemu-kvm-ev-2.6.0-28.el7.10.1.x86_64 which is already installed, so if it obsoleted why do we need to check it?
Comment 6 Eduardo Habkost 2017-07-31 14:22:00 UTC 
(In reply to Artyom from comment #5)
> I believe qemu-kvm-ev coming as part of vdsm dependencies, so it can be
> problematic for me to change from ev to rhev packages, also when I tried to
> install it with ev packages already installed I got warning message
> Package 10:qemu-kvm-rhev-2.6.0-28.el7_3.9.x86_64 is obsoleted by
> 10:qemu-kvm-ev-2.6.0-28.el7.10.1.x86_64 which is already installed, so if it
> obsoleted why do we need to check it?

Because this is a RHEL bug report, and qemu-kvm-ev is a CentOS package.  :)
Comment 7 Artyom 2017-07-31 15:17:53 UTC 
Oh thanks for the explanation, it always a lot of problems with packages when we work upstream.
Ok I downloaded rhev packages:
# rpm -qa | grep qemu
qemu-kvm-common-rhev-2.6.0-28.el7_3.10.x86_64
ipxe-roms-qemu-20170123-1.git4e85b27.el7.noarch
qemu-kvm-tools-rhev-2.6.0-28.el7_3.10.x86_64
qemu-img-rhev-2.6.0-28.el7_3.10.x86_64
libvirt-daemon-driver-qemu-3.2.0-14.el7.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.10.x86_64

but still, I can see only one NUMA node and I can see ACPI message
[    0.000000] Early table checksum verification disabled
[    0.000000] ACPI BIOS Error (bug): A valid RSDP was not found (20130517/tbxfroot-243)
Comment 8 Eduardo Habkost 2017-08-18 23:00:56 UTC 
I'm not being able to reproduce it, either.  Can I get access to the system where the problem is happening?
Comment 9 Artyom 2017-08-26 20:30:18 UTC 
I can not reproduce it on the oVirt the last build, so I will close the bug and reopen it in the case if I will see the problem again.
Note You need to log in before you can comment on or make changes to this bug.