Bug 1534418

Patch posted upstream:

https://www.redhat.com/archives/libvir-list/2018-May/msg01408.html

To POST:

commit fa6bdf6afa878b8d7c5ed71664ee72be8967cdc5
Author:     Michal Privoznik <mprivozn>
AuthorDate: Fri May 18 12:54:46 2018 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Wed May 23 09:00:20 2018 +0200

    qemu: Deny hugepages for non-existent NUMA nodes
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1534418
    
    Just like ec982f6d929f3c23 denies hugepages for non-existent
    guest NUMA nodes in case there are some nodes configured.
    Unfortunately, when there are none, qemuBuildNumaArgStr() is not
    called and thus we have to have check in qemuBuildMemPathStr()
    too.
    
    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

v4.3.0-256-gfa6bdf6afa

Verify on packages:
libvirt-4.4.0-2.el7.x86_64
qemu-kvm-rhev-2.12.0-3.el7.x86_64
kernel: 3.10.0-902.el7.x86_64

Steps:
1. Define and try to start a guest with huge page with nodeset='1',
while no 'numa' settings in guest cpu element, get error.

  <memoryBacking>
    <hugepages>
      <page size='2147483648' unit='KiB' nodeset='1'/>
    </hugepages>
  </memoryBacking>
  <vcpu placement='static' current='6'>10</vcpu>
...
  <cpu mode='host-model' check='partial'>
    <model fallback='allow'/>
  </cpu>
...

# virsh start r7
error: Failed to start domain r7
error: hugepages: node 1 not found

2. Define and start a guest with huge page nodeset and numa settings successfully.
  <memoryBacking>
    <hugepages>
      <page size='2048' unit='KiB' nodeset='1'/>
    </hugepages>
  </memoryBacking>
...
  <cpu mode='host-model'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' cpus='4-7' memory='512000' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
...

# virsh start r7
Domain r7 started

# ps -ef|grep hugepage
...
 -object memory-backend-ram,id=ram-node0,size=524288000 -numa node,nodeid=0,cpus=0-3,memdev=ram-node0 -object memory-backend-file,id=ram-node1,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/39-r7,share=yes,size=524288000 -numa node,nodeid=1,cpus=4-7,memdev=ram-node1
...

3. Destroy the guest, and modify the hugepage nodeset to '2', 
start the the guest, get error.
# virsh start r7
error: Failed to start domain r7
error: hugepages: node 2 not found

4. Add dimm device with node set to '2', try to start the guest, get error, however, the error message need improvement.

# virsh dumpxml r7|grep dimm -A 8
    <memory model='dimm'>
      <source>
        <nodemask>1</nodemask>
        <pagesize unit='KiB'>2048</pagesize>
      </source>
      <target>
        <size unit='KiB'>256000</size>
        <node>2</node>
      </target>
      <address type='dimm' slot='0'/>
    </memory>
  </devices>
</domain>

# virsh start r7
error: Failed to start domain r7
error: unsupported configuration: can't add memory backend for guest node '2' as the guest has only '2' NUMA nodes configured

5. Delete the numa setting and try to start the guest, get error.
# virsh start r7
error: Failed to start domain r7
error: unsupported configuration: At least one numa node has to be configured when enabling memory hotplug

6. Add the numa setting back and edit the dimm device with <node>2</node>, start the guest successfully.

# virsh start r7
Domain r7 started

# ps -ef|grep qemu| grep dimm
...
-object memory-backend-ram,id=ram-node0,size=524288000 -numa node,nodeid=0,cpus=0-3,memdev=ram-node0 -object memory-backend-file,id=ram-node1,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/53-r7,share=yes,size=524288000 -numa node,nodeid=1,cpus=4-7,memdev=ram-node1 -object memory-backend-file,id=memdimm0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/53-r7,share=yes,size=262144000,host-nodes=1,policy=bind -device pc-dimm,node=1,memdev=memdimm0,id=dimm0,slot=0
...

(In reply to chhu from comment #6)
> 4. Add dimm device with node set to '2', try to start the guest, get error,
> however, the error message need improvement.
> 
> # virsh dumpxml r7|grep dimm -A 8
>     <memory model='dimm'>
>       <source>
>         <nodemask>1</nodemask>
>         <pagesize unit='KiB'>2048</pagesize>
>       </source>
>       <target>
>         <size unit='KiB'>256000</size>
>         <node>2</node>
>       </target>
>       <address type='dimm' slot='0'/>
>     </memory>
>   </devices>
> </domain>
> 
> # virsh start r7
> error: Failed to start domain r7
> error: unsupported configuration: can't add memory backend for guest node
> '2' as the guest has only '2' NUMA nodes configured
> 

Hi, Michal 

Would you like to improve above error message in this bug? Thank you!


Regards,
chhu

(In reply to chhu from comment #7)
> (In reply to chhu from comment #6)

> > # virsh start r7
> > error: Failed to start domain r7
> > error: unsupported configuration: can't add memory backend for guest node
> > '2' as the guest has only '2' NUMA nodes configured
> > 
> 
> Hi, Michal 
> 
> Would you like to improve above error message in this bug? Thank you!

I don't quite see what is wrong with it. Is it the confusion that numa nodes are numbered from 0 and thus if you have two numa nodes (#0 and #1) there is no node #2? This is perfectly okay. Anybody who is setting NUMA is expected to know this. If they don't then they should not set up NUMA in the first place.

> > > # virsh start r7
> > > error: Failed to start domain r7
> > > error: unsupported configuration: can't add memory backend for guest node
> > > '2' as the guest has only '2' NUMA nodes configured
> > > 
> > 
> > Hi, Michal 
> > 
> > Would you like to improve above error message in this bug? Thank you!
> 
> I don't quite see what is wrong with it. Is it the confusion that numa nodes
> are numbered from 0 and thus if you have two numa nodes (#0 and #1) there is
> no node #2? This is perfectly okay. Anybody who is setting NUMA is expected
> to know this. If they don't then they should not set up NUMA in the first
> place.

Ok, thank you!

Regards,
chhu

According to comment 5,6,8, set the bug status to VERIFIED.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113