RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2208946 - Wrong numa tuning mode applies on numa memory when "interleave" or "preferred" mode mixed with "restrictive" mode
Summary: Wrong numa tuning mode applies on numa memory when "interleave" or "preferred...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: libvirt
Version: 9.3
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: liang cong
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-05-22 07:27 UTC by liang cong
Modified: 2023-11-07 09:42 UTC (History)
5 users (show)

Fixed In Version: libvirt-9.4.0-1.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-11-07 08:31:41 UTC
Type: Bug
Target Upstream Version: 9.4.0
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-157809 0 None None None 2023-05-22 07:29:20 UTC
Red Hat Product Errata RHSA-2023:6409 0 None None None 2023-11-07 08:32:01 UTC

Description liang cong 2023-05-22 07:27:27 UTC 
Description of problem:Wrong numa tuning mode applies on numa memory when "interleave" or "preferred" mode mixed with "restrictive" mode



Version-Release number of selected component (if applicable):
# rpm -q libvirt qemu-kvm
libvirt-9.3.0-2.el9.x86_64
qemu-kvm-8.0.0-3.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
Scenario 1:
1.1 Start a guest with below setting:
<numatune>
          <memory mode="interleave" nodeset="0"/>
          <memnode cellid="0" mode="restrictive" nodeset="1"/>
  </numatune>
...
<numa>       
    <cell id='0' cpus='0' memory='1048576' unit='KiB'/>       
    <cell id='1' cpus='1' memory='1024000' unit='KiB'/>     
</numa>

1.2 Check guest numa cell 0 memory allocation on host
# grep -B1 1048576 /proc/`pidof qemu-kvm`/smaps
7fecf3e00000-7fed33e00000 rw-p 00000000 00:00 0
Size:            1048576 kB

 grep 7fecf3e00000 /proc/`pidof qemu-kvm`/numa_maps
7fecf3e00000 interleave:0 anon=54798 dirty=54798 N0=6657 N1=48141 kernelpagesize_kB=4

1.3 check qemu cmd line and no specific mode applies on guest numa cell 0 memory
...-overcommit mem-lock=off -smp 2,sockets=2,cores=1,threads=1 -object {"qom-type":"memory-backend-ram","id":"ram-node0","size":1073741824} -numa node,nodeid=0,cpus=0,memdev=ram-node0 -object {"qom-type":"memory-backend-ram","id":"ram-node1","size":1048576000,"host-nodes":[0],"policy":"interleave"} -numa node,nodeid=1,cpus=1,memdev=ram-node1...

Actual results:
We could see that the guest numa cell 0 memory is applied with interleave mode which is different from what we defined.

Scenario 2:
2.1 Start a guest with below setting:
<numatune>
          <memory mode="preferred" nodeset="0"/>
          <memnode cellid="0" mode="restrictive" nodeset="1"/>
  </numatune>
...
<numa>       
    <cell id='0' cpus='0' memory='1048576' unit='KiB'/>       
    <cell id='1' cpus='1' memory='1024000' unit='KiB'/>     
</numa>

2.2 Check guest numa cell 0 memory allocation on host
# grep -B1 1048576 /proc/`pidof qemu-kvm`/smaps
7f26f7e00000-7f2737e00000 rw-p 00000000 00:00 0
Size:            1048576 kB

# grep 7f26f7e00000 /proc/`pidof qemu-kvm`/numa_maps
7f26f7e00000 prefer (many):0 anon=37902 dirty=37902 active=32782 N0=1537 N1=36365 kernelpagesize_kB=4

2.3 check qemu cmd line and no specific mode applies on guest numa cell 0 memory
...-overcommit mem-lock=off -smp 2,sockets=2,cores=1,threads=1 -object {"qom-type":"memory-backend-ram","id":"ram-node0","size":1073741824} -numa node,nodeid=0,cpus=0,memdev=ram-node0 -object {"qom-type":"memory-backend-ram","id":"ram-node1","size":1048576000,"host-nodes":[0],"policy":"preferred"} -numa node,nodeid=1,cpus=1,memdev=ram-node1 ...

Actual results:
We could see that the guest numa cell 0 memory is applied with preferred mode which is different from what we defined.


Expected results:
The guest numa cell 0 memory should apply default numa mode

Additional info:
1. If I start a guest with below setting:
<numatune>
          <memory mode="strict" nodeset="0"/>
          <memnode cellid="0" mode="restrictive" nodeset="1"/>
  </numatune>
...
<numa>       
    <cell id='0' cpus='0' memory='1048576' unit='KiB'/>       
    <cell id='1' cpus='1' memory='1024000' unit='KiB'/>     
</numa>

Then check guest numa cell 0 memory allocation on host, the mode is default:
# grep -B1 1048576 /proc/`pidof qemu-kvm`/smaps
7fd173e00000-7fd1b3e00000 rw-p 00000000 00:00 0
Size:            1048576 kB
# grep 7fd173e00000 /proc/`pidof qemu-kvm`/numa_maps
7fd173e00000 default anon=77326 dirty=77326 active=73230 N0=6657 N1=70669 kernelpagesize_kB=4

2. If restrictive mode could not work with other modes we could add error prompts or restrictions when mixing them, just like if setting:
<numatune>
   <memory mode="restrictive" nodeset="0"/>
   <memnode cellid="0" mode="strict" nodeset="1"/>  <!-- or other
 mode like "preferred", "interleave" -->
</numatune>

 Error msg prompts " 'restrictive' mode is required in memnode element when mode is 'restrictive' in memory element"
Comment 1 Michal Privoznik 2023-05-22 09:54:47 UTC 
Patches posted upstream:

https://listman.redhat.com/archives/libvir-list/2023-May/239950.html
Comment 2 Michal Privoznik 2023-05-22 12:03:57 UTC 
Merged upstream as:

17c8a173b6 numa_conf: Deny other memory modes than 'restrictive' if a memnode is 'restrictive'
f6ba9fc12a numa_conf: Move memnode mode validation into virDomainNumaDefValidate()
a152d856c3 virDomainNumatuneNodeSpecified: Fix const correctness

v9.3.0-105-g17c8a173b6
Comment 3 liang cong 2023-05-23 09:53:41 UTC 
Preverified on upstream build libvirt v9.3.0-110-g3b6d69237f 

Test steps:
scenario 1:
1.1 Define a guest with below setting:
<numatune>
          <memory mode="interleave" nodeset="0"/>     <!-- also preferred or strict mode -->
          <memnode cellid="0" mode="restrictive" nodeset="1"/>
  </numatune>
...
<numa>       
    <cell id='0' cpus='0' memory='1048576' unit='KiB'/>       
    <cell id='1' cpus='1' memory='1024000' unit='KiB'/>     
</numa>

1.2 Error msg prompts
# virsh define vm1.xml
error: Failed to define domain from vm1.xml
error: XML error: 'restrictive' mode is required in memory element when mode is 'restrictive' in memnode element

scenario 2:
2.1 Define a guest with below setting:
<numatune>
          <memnode cellid="0" mode="restrictive" nodeset="1"/>
  </numatune>
...
<numa>       
    <cell id='0' cpus='0' memory='1048576' unit='KiB'/>       
    <cell id='1' cpus='1' memory='1024000' unit='KiB'/>     
</numa>

2.2 Error msg prompts
# virsh define vm1.xml
error: Failed to define domain from vm1.xml
error: XML error: 'restrictive' mode is required in memory element when mode is 'restrictive' in memnode element

Hi michal, please help to check scenario 2, error msg prompts when only restrictive memnode element defined. Could you help to check if it works as design?
Comment 4 Michal Privoznik 2023-05-25 11:57:20 UTC 
(In reply to liang cong from comment #3)

> Hi michal, please help to check scenario 2, error msg prompts when only
> restrictive memnode element defined. Could you help to check if it works as
> design?

Yes, because the default is 'strict'. Therefore, users need to explicitly define 'restrictive'.
Comment 5 liang cong 2023-07-25 07:07:18 UTC 
Verified on build:
# rpm -q libvirt qemu-kvm
libvirt-9.5.0-3.el9.x86_64
qemu-kvm-8.0.0-9.el9.x86_64

Test steps:
scenario 1:
1.1 Define a guest with below setting:
<numatune>
          <memory mode="interleave" nodeset="0"/>     <!-- also preferred or strict mode -->
          <memnode cellid="0" mode="restrictive" nodeset="1"/>
  </numatune>
...
<numa>       
    <cell id='0' cpus='0' memory='1048576' unit='KiB'/>       
    <cell id='1' cpus='1' memory='1024000' unit='KiB'/>     
</numa>

1.2 Error msg prompts
# virsh define vm1.xml
error: Failed to define domain from vm1.xml
error: XML error: 'restrictive' mode is required in memory element when mode is 'restrictive' in memnode element


scenario 2:
2.1 Define a guest with below setting:
<numatune>
          <memnode cellid="0" mode="restrictive" nodeset="1"/>
  </numatune>
...
<numa>       
    <cell id='0' cpus='0' memory='1048576' unit='KiB'/>       
    <cell id='1' cpus='1' memory='1024000' unit='KiB'/>     
</numa>

2.2 Error msg prompts
# virsh define vm1.xml
error: Failed to define domain from vm1.xml
error: XML error: 'restrictive' mode is required in memory element when mode is 'restrictive' in memnode element

scenario 3:
3.1 Define a guest with below setting:
<numatune>
          <memory mode="restrictive" nodeset="0-1"/>     
          <memnode cellid="0" mode="strict" nodeset="1"/>   <!-- also preferred or interleave mode -->
  </numatune>
...
<numa>       
    <cell id='0' cpus='0' memory='1048576' unit='KiB'/>       
    <cell id='1' cpus='1' memory='1024000' unit='KiB'/>     
</numa>

3.2 Error msg prompts
# virsh define vm1.xml
error: Failed to define domain from vm1.xml
error: XML error: 'restrictive' mode is required in memnode element when mode is 'restrictive' in memory element
Comment 9 liang cong 2023-08-07 03:23:32 UTC 
mark it verified for comment 5
Comment 11 errata-xmlrpc 2023-11-07 08:31:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: libvirt security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6409
Note You need to log in before you can comment on or make changes to this bug.