Bug 1783355

Summary: Libvirt uses deprecated numa command line
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: Peter Krempa <pkrempa>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Jing Qi <jinqi>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.1CC: dyuan, imammedo, jdenemar, jinzhao, jsuchane, lhuang, lmen, mprivozn, xuzhang, yalzhang, yuhuang
Target Milestone: rcKeywords: Upstream
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-6.4.0-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-17 17:46:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Peter Krempa 2019-12-13 15:54:35 UTC
Description of problem:
libvirt's NUMA configuration may result in a command line which qemu considers deprecated:

  <cpu>
    <topology sockets='2' cores='4' threads='2'/>
    <numa>
      <cell cpus='0-7' memory='109550' unit='KiB'/>
      <cell cpus='8-15' memory='109550' unit='KiB'/>
    </numa>
  </cpu>

-numa node,nodeid=0,cpus=0-7,mem=107 \
-numa node,nodeid=1,cpus=8-15,mem=107 \

qemu then reports:
2019-12-10T09:47:19.571513Z qemu-kvm: -numa node,nodeid=0,cpus=0-7,mem=107: warning: Parameter -numa node,mem is deprecated,     use -numa node,memdev instead

Version-Release number of selected component (if applicable):
libvirt version: 5.6.0, package: 6.module+el8.1.0+4244+9aa4e6bb
qemu version: 4.1.0qemu-kvm-4.1.0-14.module+el8.1.0+4548+ed1300f4

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Michal Privoznik 2020-01-16 13:04:58 UTC
Design discussion started here:

https://www.redhat.com/archives/libvir-list/2020-January/msg00709.html

Comment 2 Igor Mammedov 2020-03-26 11:40:00 UTC
(In reply to Michal Privoznik from comment #1)
> Design discussion started here:
> 
> https://www.redhat.com/archives/libvir-list/2020-January/msg00709.html

Was it fixed upstream already?

Comment 3 Michal Privoznik 2020-03-26 15:44:54 UTC
Unfortunately, no.

Comment 4 Michal Privoznik 2020-05-12 14:53:47 UTC
Patches posted upstream:

https://www.redhat.com/archives/libvir-list/2020-May/msg00490.html

Comment 6 Michal Privoznik 2020-05-18 08:39:34 UTC
Merged upstream:

aad31f46fb qemuBuildNumaArgStr: Use modern -numa memdev= if old -numa mem= is unsupported
0246a4f0a7 qemuBuildNumaArgStr: Switch order of if() and for()
bba05b01ce qemu: Track numa-mem-supported machine attribute

v6.3.0-103-gaad31f46fb

Comment 10 Jing Qi 2020-08-17 07:30:27 UTC
Verified with
libvirt-6.6.0-2.module+el8.3.0+7567+dc41c0a9.x86_64 &
qemu-kvm-5.1.0-2.module+el8.3.0+7652+b30e6901.x86_64

Start a domain with numa part xml:
 <cpu mode='custom' match='exact' check='none'>
    <model fallback='forbid'>qemu64</model>
    <topology sockets='2' dies='1' cores='4' threads='2'/>
    <numa>
      <cell id='0' cpus='0-7' memory='109550' unit='KiB'/>
      <cell id='1' cpus='8-15' memory='109550' unit='KiB'/>
    </numa>
  </cpu>

# virsh start vm1

Check the qemu command line:

-object memory-backend-ram,id=ram-node0,size=112197632
-numa node,nodeid=0,cpus=0-7,memdev=ram-node0 
-object memory-backend-ram,id=ram-node1,size=112197632
-numa node,nodeid=1,cpus=8-15,memdev=ram-node1

Comment 13 errata-xmlrpc 2020-11-17 17:46:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5137

Comment 14 Yumei Huang 2021-07-01 03:18:44 UTC
Hi, 

Seems libvirt is using the old cmd line again. 

E.g.

  <cpu mode='host-model' check='partial'>
    <numa>
      <cell id='0' cpus='0-3' memory='1048576' unit='KiB'/>
    </numa>
  </cpu>

The qemu cli generated is '-numa node,nodeid=0,cpus=0-3,mem=1024'.


Packages:
libvirt-client-7.4.0-1.module+el8.5.0+11218+83343022.x86_64
qemu-kvm-6.0.0-21.module+el8.5.0+11555+e0ab0d09

Comment 15 Jing Qi 2021-07-01 04:56:13 UTC
I can't reproduce it in the same version .
Is the libvirt upgraded from some lower version in your environment?

Comment 16 Yumei Huang 2021-07-01 05:57:22 UTC
(In reply to Jing Qi from comment #15)
> I can't reproduce it in the same version .
> Is the libvirt upgraded from some lower version in your environment?

No, it's fresh installed. 

BTW, I noticed the old cmd line is used in early versions as the warning message in https://bugzilla.redhat.com/show_bug.cgi?id=1938956#c0.

Comment 17 Michal Privoznik 2021-07-01 09:16:06 UTC
The use of old/new cmd line is dependent on machine type. I mean, when libvirt queries QEMU capabilities, it looks for 'numa-mem-supported' attribute, e.g. like this:

{"hotpluggable-cpus": true, "name": "pc-q35-5.2", "numa-mem-supported": false, "default-cpu-type": "qemu64-x86_64-cpu", "cpu-max": 288, "deprecated": false, "default-ram-id": "pc.ram"}

and only if numa-mem-supported is true then the old -numa mem=X is used (IIRC there were some problems with guest ABI, hence machine type dependency). I think pc-q35-5.1 and pc-i440fx-5.1 were the first machine types that started reporting numa-mem-supported:false.

Comment 18 Yumei Huang 2021-07-01 09:45:28 UTC
(In reply to Michal Privoznik from comment #17)
> The use of old/new cmd line is dependent on machine type. I mean, when
> libvirt queries QEMU capabilities, it looks for 'numa-mem-supported'
> attribute, e.g. like this:
> 
> {"hotpluggable-cpus": true, "name": "pc-q35-5.2", "numa-mem-supported":
> false, "default-cpu-type": "qemu64-x86_64-cpu", "cpu-max": 288,
> "deprecated": false, "default-ram-id": "pc.ram"}
> 
> and only if numa-mem-supported is true then the old -numa mem=X is used
> (IIRC there were some problems with guest ABI, hence machine type
> dependency). I think pc-q35-5.1 and pc-i440fx-5.1 were the first machine
> types that started reporting numa-mem-supported:false.


Yes, it's using the new cmd line with q35 machine types. Thanks Michal for the explanation!