Bug 1662586

+++ This bug was initially created as a clone of Bug #1217144 +++

Reproduce this bug in RHEL-8, detailed steps can be see in 
https://bugzilla.redhat.com/show_bug.cgi?id=1217144#c8

Description of problem:
=======================

When trying to edit the <cpu> field in the domain xml using virsh edit, I get the following error:

[root@rhos-compute-node-10 ~]# virsh edit rhel71
error: XML error: Missing 'memory' attribute in NUMA cell
Failed. Try again? [y,n,f,?]:

I edited the <cpu> field like this:

  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0,1'/>
      <cell id='1' cpus='2,3'/>
    </numa>
  </cpu>

However, this should be allowable on this libvirt version:

[root@rhos-compute-node-10 ~]# rpm -qa | grep libvirt
libvirt-daemon-config-network-1.2.8-16.el7_1.2.x86_64
libvirt-client-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-storage-1.2.8-16.el7_1.2.x86_64
libvirt-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-nodedev-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-kvm-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-config-nwfilter-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-nwfilter-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-interface-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-qemu-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-lxc-1.2.8-16.el7_1.2.x86_64
libvirt-python-1.2.8-7.el7_1.1.x86_64
libvirt-daemon-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-network-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-secret-1.2.8-16.el7_1.2.x86_64


Version-Release number of selected component:
=============================================

1.2.8-16_1.2

How reproducible
================

Always


Steps to Reproduce:
===================
1. Create a VM
2. virsh edit instance-name
3. In the editor, make the <cpu> field look like this (assuming 4 vcpus and 2 NUMA nodes):

  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0,1'/>
      <cell id='1' cpus='2,3'/>
    </numa>
  </cpu>


Actual results
==============

error: XML error: Missing 'memory' attribute in NUMA cell
Failed. Try again? [y,n,f,?]:

Selecting 'f' will not force, and the <cpu> looks like this still:

  <cpu mode='host-passthrough'>
  </cpu>

Expected results
================

The domain xml should be written out with numa cells not having a 'memory' attribute specified.


Additional info
===============

--- Additional comment from RHEL Product and Program Management on 2015-04-29 16:07:32 UTC ---

Since this bug report was entered in bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

--- Additional comment from Peter Krempa on 2015-04-30 07:25:35 UTC ---

When you are going to enable a NUMA topology you need to specify the memory for each NUMA cell since that can't be inferred from the total memory size in a sane way.

As of the 'f' key in "virsh edit". The force option tries to define the XML again. Since it's invalid it will fail since the parser can't be forced to accept an invalid XML.

Closing as NOTABUG, the memory attribute for the <cell> element is mandatory.

--- Additional comment from Daniel Berrange on 2015-04-30 09:27:59 UTC ---

The XML given was slightly misleading as to the problem - what Sean was actually trying todo was to create 2 numa nodes, one of which has all the RAM and the other which has no RAM.

eg

    <numa>
      <cell id='0' cpus='0,1' memory='131072' unit='KiB'/>
      <cell id='1' cpus='2,3'/>
    </numa>

Of course nothing is actually preventing use of memory='0', so this can in fact be achieved.

    <numa>
      <cell id='0' cpus='0,1' memory='131072' unit='KiB'/>
      <cell id='1' cpus='2,3' memory='0' unit='KiB'/>
    </numa>

--- Additional comment from Daniel Berrange on 2015-04-30 15:18:40 UTC ---

I'm actually re-opening this as I think we should address it.

In the capabilities XML, if you have a node we no memory, libvirt leaves out the memory='0' attribute.  Given this, I think that in the guest XML, if a NUMA node has no memory= attribute, it should be treated the same as if it had memory='0'.

There is a corresponding problem with CPUs. it is impossible to leave out the 'cpus=' attribute in the domain XML, and you also cannot set it to the empty string, as libvirt complains it cannot parse the bitmap.

Essentially, we should be free to create NUMA nodes which have zero CPUs, or zero memory, and libvirt is preventing that entirely for CPUs and unnecessarily mandating memory= attribute.

--- Additional comment from Peter Krempa on 2015-06-26 11:18:07 UTC ---

For feature completeness this will require a specific vCPU hotplug API which unfortunately slipped to the next release, thus I'll move this too.

--- Additional comment from Libor Miksik on 2015-11-04 16:18:28 UTC ---

Since the release flag was set to ? after the pm_ack flag was set to + (was likely set for the previous release), the pm_ack flag has been reset to ? by the bugbot (pm-rhel). This action ensures the proper review by Product Management.

--- Additional comment from Peter Krempa on 2017-02-24 15:13:18 UTC ---

I've posted a patch [1] that adds syntax sugar to allow specifying 0 memory for a numa node by omitting 'memory' attribute. Unfortunately while qemu accepts the command line (with or without the patch) the resulting NUMA topology from guest's point of view is broken. I'll clone this bug later so that that issue can be fixed.

Until that is fixed it doesn't really make much sense to push that patch.

CPU-less NUMA nodes are now possible:
Libvirt now allows to hotplug or hotunplug individual vcpus. Together with the ability to configure them in the XML ( http://libvirt.org/formatdomain.html#elementsCPUAllocation ) this should provide features requested by this BZ:

commit bd8f10c62aa4d0fa09bf74bc5f337a3a28b0d694
Author: Peter Krempa <pkrempa>
Date:   Fri Feb 10 17:14:22 2017 +0100

    test: qemuhotplugtest: Add testing of individual vcpu hotplug
    
    Test that the vcpu entity selection code works properly

commit f557b3351e0b6dee07b434ef65c81887c243d406
Author: Peter Krempa <pkrempa>
Date:   Tue Jun 21 17:17:41 2016 +0200

    qemu: Implement individual vcpu hotplug API
    
    Add code that validates user's selection of cores and then uses the
    existing code to plug in the vCPU.

commit f0326d6dd921a9b30437c0bc1f4aa98ec5a31272
Author: Peter Krempa <pkrempa>
Date:   Tue Jun 21 10:55:30 2016 +0200

    virsh: Implement command for virDomainSetVcpu called setvcpu
    
    Add a simple virsh command handler which makes use of the new API.

commit 8f657259bb9c8310b1f232249174f06f30089755
Author: Peter Krempa <pkrempa>
Date:   Tue Jun 21 10:44:51 2016 +0200

    lib: Add API for specific vCPU hot(un)plug
    
    Similarly to domainSetGuestVcpus this commit adds API which allows to
    modify state of individual vcpus rather than just setting the count.
    
    This allows to enable CPUs in specific guest NUMA nodes to achieve any
    necessary configuration.

[1] https://www.redhat.com/archives/libvir-list/2017-February/msg01087.html

--- Additional comment from jiyan on 2018-12-30 09:54:10 UTC ---

############# Reproduce this issue on the following version (RHEL-7.1): 
Version:
kernel-3.10.0-229.el7.x86_64
libvirt-1.2.8-16.el7.x86_64
qemu-kvm-1.5.3-86.el7.x86_64

Steps:
1. Edit VM's numa-cell element without configuring vcpus
# virsh edit test1
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' memory='512000' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
error: XML error: Missing 'cpus' attribute in NUMA cell
Failed. Try again? [y,n,f,?]:

2. Edit VM's numa-cell element without configuring memory
# virsh edit test1
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' cpus='4-7'/>
    </numa>
  </cpu>
error: XML error: Missing 'memory' attribute in NUMA cell
Failed. Try again? [y,n,f,?]:


############# Still can reproduce this issue on the following version (latest RHEL-7): 
Version:
libvirt-4.5.0-10.el7_6.3.x86_64
qemu-kvm-rhev-2.12.0-19.el7_6.2.x86_64
kernel-3.10.0-957.5.1.el7.x86_64

Steps:
1. Edit VM's numa-cell element without configuring vcpus
# virsh edit test1
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' memory='512000' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element cpu in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]: 
error: XML error: Missing 'cpus' attribute in NUMA cell

2. Edit VM's numa-cell element without configuring memory
# virsh edit test1
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' cpus='4-7'/>
    </numa>
  </cpu>
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element cpu in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]: 
error: XML error: missing element or attribute './@memory'

############# Still can reproduce this issue on the following version (RHEL-8 slow train): 
Version:
libvirt-4.5.0-16.module+el8+2586+bf759444.x86_64
qemu-kvm-2.12.0-50.module+el8+2596+0a642e54.x86_64
kernel-4.18.0-57.el8.x86_64

Steps:
1. Edit VM's numa-cell element without configuring vcpus
# virsh edit test1
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' memory='512000' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element cpu in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]: 
error: XML error: Missing 'cpus' attribute in NUMA cell


2. Edit VM's numa-cell element without configuring memory
# virsh edit test1
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' cpus='4-7'/>
    </numa>
  </cpu>
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element cpu in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]: 
error: XML error: missing element or attribute './@memory'

############# Still can reproduce this issue on the following version (RHEL-8 fast train): 
Version:
libvirt-4.10.0-1.module+el8+2317+367e35b5.x86_64
kernel-4.18.0-57.el8.x86_64
qemu-kvm-3.1.0-1.module+el8+2538+1516be75.x86_64

Steps:
1. Edit VM's numa-cell element without configuring vcpus
# virsh edit test1
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' memory='512000' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element cpu in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]: 
error: XML error: Missing 'cpus' attribute in NUMA cell

2. Edit VM's numa-cell element without configuring memory
# virsh edit test1
  <cpu mode='host-passthrough'>
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' discard='yes'/>
      <cell id='1' cpus='4-7'/>
    </numa>
  </cpu>
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element cpu in interleave
Element domain failed to validate content

Failed. Try again? [y,n,f,?]: 
error: XML error: missing element or attribute './@memory'