Bug 1741658

Summary:	libvirt doesn't copy all query-hotpluggable-cpus properties for CPU hotplug
Product:	Red Hat Enterprise Linux Advanced Virtualization	Reporter:	Eduardo Habkost <ehabkost>
Component:	libvirt	Assignee:	Peter Krempa <pkrempa>
Status:	CLOSED ERRATA	QA Contact:	jiyan <jiyan>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	8.1	CC:	imammedo, jdenemar, jiyan, jsuchane, jtomko, lhuang, lmen, mrezanin, pkrempa, xuzhang
Target Milestone:	rc	Flags:	knoel: mirror+
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	libvirt-5.6.0-3.el8	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-11-06 07:18:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Eduardo Habkost 2019-08-15 18:19:36 UTC

This bug was initially created as a copy of Bug #1741451

I am copying this bug because: 
The plan is to change QEMU to keep it compatible with existing libvirt, but making libvirt flexible and support additional CPU properties is also necessary.  See https://bugzilla.redhat.com/show_bug.cgi?id=1741451#c3


Description of problem:
Failed to hot-plug vcpus

Version-Release number of selected component (if applicable):
qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1.x86_64
libvirt-5.6.0-1.virtcov.el8.x86_64
kernel-4.18.0-129.el8.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare a running VM with the following vcpus conf
# virsh domstate test 
running

# virsh vcpucount test 
maximum      config         4
maximum      live           4
current      config         2
current      live           2

2. Hot-plug vcpu and check stap info
# virsh setvcpus test 3 --live
error: internal error: unable to execute QEMU command 'device_add': Invalid CPU die-id: 4294967295 must be in range 0:3

# stap /usr/share/doc/libvirt-docs/examples/systemtap/qemu-monitor.stp 
  0.000 begin
  2.943 > 0x7f62ec025900 {"execute":"device_add","arguments":{"driver":"Skylake-Server-IBRS-x86_64-cpu","id":"vcpu2","socket-id":2,"core-id":0,"thread-id":0},"id":"libvirt-280"}
  2.948 < 0x7f62ec025900 {"id": "libvirt-280", "error": {"class": "GenericError", "desc": "Invalid CPU die-id: 4294967295 must be in range 0:3"}}

Actual results:
As step-2 shows

Expected results:
Hot-plugging should succeed

Additional info:
Can not reproduce this issue on qemu-kvm-4.0.0-6.module+el8.1.0+3736+a2aefea3.x86_64

Comment 1 Jiri Denemark 2019-08-16 07:29:26 UTC

*** Bug 1741807 has been marked as a duplicate of this bug. ***

Comment 2 Peter Krempa 2019-08-29 13:49:26 UTC

Patches posted upstream:

https://www.redhat.com/archives/libvir-list/2019-August/msg01322.html

Comment 3 Peter Krempa 2019-08-29 14:48:59 UTC

Fixed upstream:

commit c87c42f0eb695b0d0860da81b4fdd2e47ea6d7c7 (HEAD -> master, origin/master, origin/HEAD)
Author: Peter Krempa <pkrempa>
Date:   Thu Aug 29 15:15:23 2019 +0200

    qemu: command: Use all vCPU properties when creating args for vCPU hotplug
    
    As qemu documents we should use everything in the 'props' sub-object of
    the data returned by query-hotpluggable-cpus. Until now we only used
    everything we recognized, but that may break in cases when qemu
    introduces new fields.
    
    This change requires a fix to the test data as some fields were
    reordered.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1741658
    
    Signed-off-by: Peter Krempa <pkrempa>

commit a0b61591f237d13024ecffa9699aa4e0831bc23a
Author: Peter Krempa <pkrempa>
Date:   Thu Aug 29 14:47:10 2019 +0200

    qemu: Extract and store vCPU properties as qemu returned them
    
    In addition to the data that libvirt needs and extracts internally,
    copy and store the whole 'props' JSON sub-object of the data returned by
    query-hotpluggable-cpus for future use.
    
    Signed-off-by: Peter Krempa <pkrempa>

v5.7.0-rc1-9-gc87c42f0eb

Comment 6 jiyan 2019-09-02 04:53:58 UTC

Reproduced this bug on libvirt-5.6.0-2.module+el8.1.0+4015+63576633.x86_64.

Version:
kernel-4.18.0-141.el8.x86_64
qemu-kvm-4.1.0-5.module+el8.1.0+4076+b5e41ebc.x86_64
libvirt-5.6.0-2.module+el8.1.0+4015+63576633.x86_64

Stesp:
1. Prepare a running VM with the following vcpu conf
# virsh domstate test
running

# virsh vcpucount test
maximum      config         8
maximum      live           8
current      config         4
current      live           4

2. Open one terminal for qemu-monitor.stp and hot-plug vcpu to VM in another terminal
# stap /usr/share/doc/libvirt-docs/examples/systemtap/qemu-monitor.stp

# virsh setvcpus test 5

3. Check the output of qemu-monitor.stp  (** No "die-id" Property passed to "device_add" **)
  0.000 begin
 21.648 > 0x7ff68c025510 {"execute":"device_add","arguments":{"driver":"Skylake-Client-IBRS-x86_64-cpu","id":"vcpu4","socket-id":4,"core-id":0,"thread-id":0},"id":"libvirt-276"}
 21.650 < 0x7ff68c025510 {"return": {}, "id": "libvirt-276"}
 21.650 > 0x7ff68c025510 {"execute":"query-hotpluggable-cpus","id":"libvirt-277"}
 21.650 < 0x7ff68c025510 {"return": [{"props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 7}, "vcpus-count": 1, "type": "Skylake-Client-IBRS-x86_64-cpu"}, {"props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 6}, "vcpus-count": 1, "type": "Skylake-Client-IBRS-x86_64-cpu"}, {"props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 5}, "vcpus-count": 1, "type": "Skylake-Client-IBRS-x86_64-cpu"}, {"props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 4}, "vcpus-count": 
 21.650 > 0x7ff68c025510 {"execute":"query-cpus-fast","id":"libvirt-278"}
 21.651 < 0x7ff68c025510 {"return": [{"arch": "x86", "thread-id": 16418, "props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 0}, "qom-path": "/machine/unattached/device[0]", "cpu-index": 0, "target": "x86_64"}, {"arch": "x86", "thread-id": 16419, "props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 1}, "qom-path": "/machine/unattached/device[2]", "cpu-index": 1, "target": "x86_64"}, {"arch": "x86", "thread-id": 16420, "props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 2}, 
 21.656 ! 0x7ff68c025510 {"timestamp": {"seconds": 1567399504, "microseconds": 968169}, "event": "ACPI_DEVICE_OST", "data": {"info": {"device": "vcpu4", "source": 1, "status": 0, "slot": "4", "slot-type": "CPU"}}}

4. Update libvirt and restart libvirtd
# yum update libvirt* -y

# rpm -qa libvirt 
libvirt-5.6.0-3.module+el8.1.0+4110+a6d45c3d.x86_64

# systemctl restart libvirtd

5. Repeat step-2 and step-3
# stap /usr/share/doc/libvirt-docs/examples/systemtap/qemu-monitor.stp

# virsh setvcpus test 6

# stap /usr/share/doc/libvirt-docs/examples/systemtap/qemu-monitor.stp  (** "die-id" Property was passed to "device_add" **)
  0.000 begin
  5.782 > 0x7f16b4001640 {"execute":"device_add","arguments":{"driver":"Skylake-Client-IBRS-x86_64-cpu","id":"vcpu5","core-id":0,"thread-id":0,   **** "die-id":0, ****  "socket-id":5},"id":"libvirt-15"}
  5.783 < 0x7f16b4001640 {"return": {}, "id": "libvirt-15"}
  5.784 > 0x7f16b4001640 {"execute":"query-hotpluggable-cpus","id":"libvirt-16"}
  5.784 < 0x7f16b4001640 {"return": [{"props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 7}, "vcpus-count": 1, "type": "Skylake-Client-IBRS-x86_64-cpu"}, {"props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 6}, "vcpus-count": 1, "type": "Skylake-Client-IBRS-x86_64-cpu"}, {"props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 5}, "vcpus-count": 1, "qom-path": "/machine/peripheral/vcpu5", "type": "Skylake-Client-IBRS-x86_64-cpu"}, {"props": {"core-id": 0, "thread-id": 0, "die
  5.784 > 0x7f16b4001640 {"execute":"query-cpus-fast","id":"libvirt-17"}
  5.784 < 0x7f16b4001640 {"return": [{"arch": "x86", "thread-id": 16418, "props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 0}, "qom-path": "/machine/unattached/device[0]", "cpu-index": 0, "target": "x86_64"}, {"arch": "x86", "thread-id": 16419, "props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 1}, "qom-path": "/machine/unattached/device[2]", "cpu-index": 1, "target": "x86_64"}, {"arch": "x86", "thread-id": 16420, "props": {"core-id": 0, "thread-id": 0, "die-id": 0, "socket-id": 2}, 
  5.790 ! 0x7f16b4001640 {"timestamp": {"seconds": 1567399636, "microseconds": 679222}, "event": "ACPI_DEVICE_OST", "data": {"info": {"device": "vcpu5", "source": 1, "status": 0, "slot": "5", "slot-type": "CPU"}}}

# virsh vcpucount test
maximum      config         8
maximum      live           8
current      config         4
current      live           6

The test result is as expected, move this bug to be verified.

Comment 8 errata-xmlrpc 2019-11-06 07:18:55 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3723