Bug 1959676 - guest status is paused after loadvm on rhel8.5.0
Summary: guest status is paused after loadvm on rhel8.5.0
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.5
Hardware: x86_64
OS: Linux
low
low
Target Milestone: rc
: 8.5
Assignee: Kevin Wolf
QA Contact: Tingting Mao
URL:
Whiteboard:
Depends On:
Blocks: 1957194
TreeView+ depends on / blocked
 
Reported: 2021-05-12 05:03 UTC by Yanhui Ma
Modified: 2021-11-16 08:38 UTC (History)
11 users (show)

Fixed In Version: qemu-kvm-6.0.0-26.module+el8.5.0+12044+525f0ebc
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-16 07:53:01 UTC
Type: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2021:4684 0 None None None 2021-11-16 07:54:12 UTC

Description Yanhui Ma 2021-05-12 05:03:24 UTC
Description of problem:

After loadvm, the guest status is paused (restore-vm), not running, need to cont manually. But on rhel8.4.0, after loadvm, the guest is running.

Version-Release number of selected component (if applicable):
qemu-kvm-6.0.0-16.module+el8.5.0+10848+2dccc46d.x86_64

How reproducible:
100%

Steps to Reproduce:
1. boot rhel8.5.0 guest
2.# nc -U /tmp/monitor-qmpmonitor1-20210510-054143-uJATAGJ0
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 6}, "package": "qemu-kvm-6.0.0-16.module+el8.5.0+10848+2dccc46d"}, "capabilities": ["oob"]}}
{"execute": "qmp_capabilities", "id": "kDiYPtqp"}
{"return": {}, "id": "kDiYPtqp"}
{"execute": "human-monitor-command", "arguments": {"command-line": "savevm sn1"}, "id": "JcqY2Shq"}
{"timestamp": {"seconds": 1620724916, "microseconds": 71650}, "event": "STOP"}
{"timestamp": {"seconds": 1620725089, "microseconds": 880209}, "event": "RESUME"}
{"return": "", "id": "JcqY2Shq"}
{"execute": "human-monitor-command", "arguments": {"command-line": "loadvm sn1"}, "id": "y5joUWoc"}
{"timestamp": {"seconds": 1620725125, "microseconds": 630937}, "event": "STOP"}
{"return": "", "id": "y5joUWoc"}
3.

Actual results:

After loadvm, the guest status is paused (restore-vm), not running, need to cont manually.
Expected results:

Just like behavior on rhel8.4.0, guest is running after loadvm.
Additional info:

Comment 1 Xueqiang Wei 2021-05-12 05:23:17 UTC
It should have been fixed by Kevin.

[PATCH] hmp: Fix loadvm to resume the VM on success instead of failure

Comment 2 John Ferlan 2021-05-14 14:23:23 UTC
(In reply to Xueqiang Wei from comment #1)
> It should have been fixed by Kevin.
> 
> [PATCH] hmp: Fix loadvm to resume the VM on success instead of failure

https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg03082.html

Kevin - since this is your patch, assigned to you for the downstream backport for RHEL-AV 8.5.0

Comment 3 Xueqiang Wei 2021-06-07 17:27:48 UTC
Yanhui,

Introduce snapshot-{save, load, delete} QMP commands since qemu 6.0. And according to https://bugzilla.redhat.com/show_bug.cgi?id=1621944#c28, I suggest you add the new qmp commands to your rhel8.5 test plan.

By the way,  guest status is running after load vm with snapshot-load.



The usage like:

1. boot a guest

    -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0 \
    -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/kvm_autotest_root/images/rhel840-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device scsi-hd,id=image1,drive=drive_image1,write-cache=on \

2. connect qmp monitor and test the three commands

# nc -U /tmp/avocado_dnfurc83/monitor-qmpmonitor1-20210527-054807-9krOJDgy
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 6}, "package": "qemu-kvm-6.0.0-17.module+el8.5.0+11173+c9fce0bb"}, "capabilities": ["oob"]}}

{ 'execute' : 'qmp_capabilities' }
{"return": {}}

{"execute": "snapshot-save", "arguments": {"job-id": "snapsave0", "tag": "my-snap", "vmstate": "drive_image1", "devices": ["drive_image1"]}}
{"timestamp": {"seconds": 1623082154, "microseconds": 351551}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "snapsave0"}}
{"timestamp": {"seconds": 1623082154, "microseconds": 351629}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "snapsave0"}}
{"return": {}}
{"timestamp": {"seconds": 1623082154, "microseconds": 369877}, "event": "STOP"}
{"timestamp": {"seconds": 1623082186, "microseconds": 773379}, "event": "RESUME"}
{"timestamp": {"seconds": 1623082186, "microseconds": 846654}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "snapsave0"}}
{"timestamp": {"seconds": 1623082186, "microseconds": 846734}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "snapsave0"}}
{"timestamp": {"seconds": 1623082186, "microseconds": 846766}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "snapsave0"}}

{"execute": "query-jobs"}
{"return": [{"current-progress": 1, "status": "concluded", "total-progress": 1, "type": "snapshot-save", "id": "snapsave0"}]}


{"execute": "snapshot-load", "arguments": {"job-id": "snapload0", "tag": "my-snap", "vmstate": "drive_image1", "devices": ["drive_image1"]}}
{"timestamp": {"seconds": 1623082467, "microseconds": 522967}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "snapload0"}}
{"timestamp": {"seconds": 1623082467, "microseconds": 523047}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "snapload0"}}
{"return": {}}
{"timestamp": {"seconds": 1623082467, "microseconds": 551096}, "event": "STOP"}
{"timestamp": {"seconds": 1623082481, "microseconds": 920940}, "event": "RESUME"}
{"timestamp": {"seconds": 1623082481, "microseconds": 923878}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "snapload0"}}
{"timestamp": {"seconds": 1623082481, "microseconds": 923962}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "snapload0"}}
{"timestamp": {"seconds": 1623082481, "microseconds": 924001}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "snapload0"}}

{"execute": "query-jobs"}
{"return": [{"current-progress": 1, "status": "concluded", "total-progress": 1, "type": "snapshot-load", "id": "snapload0"}, {"current-progress": 1, "status": "concluded", "total-progress": 1, "type": "snapshot-save", "id": "snapsave0"}]}

{"execute": "query-status"}
{"return": {"status": "running", "singlestep": false, "running": true}}


{"execute": "snapshot-delete", "arguments": {"job-id": "snapdelete0", "tag": "my-snap", "devices": ["drive_image1"]}}  
{"timestamp": {"seconds": 1623082730, "microseconds": 324380}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "snapdelete0"}}
{"timestamp": {"seconds": 1623082730, "microseconds": 324468}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "snapdelete0"}}
{"return": {}}
{"timestamp": {"seconds": 1623082730, "microseconds": 397695}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "snapdelete0"}}
{"timestamp": {"seconds": 1623082730, "microseconds": 397749}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "snapdelete0"}}
{"timestamp": {"seconds": 1623082730, "microseconds": 397786}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "snapdelete0"}}

{"execute": "query-jobs"}
{"return": [{"current-progress": 1, "status": "concluded", "total-progress": 1, "type": "snapshot-delete", "id": "snapdelete0"}, {"current-progress": 1, "status": "concluded", "total-progress": 1, "type": "snapshot-load", "id": "snapload0"}, {"current-progress": 1, "status": "concluded", "total-progress": 1, "type": "snapshot-save", "id": "snapsave0"}]}

Comment 4 Yanhui Ma 2021-06-11 02:51:17 UTC
Hi Kevin,(In reply to Xueqiang Wei from comment #3)
> Yanhui,
> 
> Introduce snapshot-{save, load, delete} QMP commands since qemu 6.0. And
> according to https://bugzilla.redhat.com/show_bug.cgi?id=1621944#c28, I
> suggest you add the new qmp commands to your rhel8.5 test plan.
> 

Thanks for your info, Xueqiang.

Hi Kevin, 

Could you also help confirm that? Should we use new QMP commands on RHEL8.5?
If so, we will modify cases.

Thanks!  
> By the way,  guest status is running after load vm with snapshot-load.
> 
> 
> 
> The usage like:
> 
> 1. boot a guest
> 
>     -device
> pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,
> chassis=3 \
>     -device
> virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0 \
>     -blockdev
> node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,
> aio=threads,filename=/home/kvm_autotest_root/images/rhel840-64-virtio-scsi.
> qcow2,cache.direct=on,cache.no-flush=off \
>     -blockdev
> node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-
> flush=off,file=file_image1 \
>     -device scsi-hd,id=image1,drive=drive_image1,write-cache=on \
> 
> 2. connect qmp monitor and test the three commands
> 
> # nc -U /tmp/avocado_dnfurc83/monitor-qmpmonitor1-20210527-054807-9krOJDgy
> {"QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 6},
> "package": "qemu-kvm-6.0.0-17.module+el8.5.0+11173+c9fce0bb"},
> "capabilities": ["oob"]}}
> 
> { 'execute' : 'qmp_capabilities' }
> {"return": {}}
> 
> {"execute": "snapshot-save", "arguments": {"job-id": "snapsave0", "tag":
> "my-snap", "vmstate": "drive_image1", "devices": ["drive_image1"]}}
> {"timestamp": {"seconds": 1623082154, "microseconds": 351551}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "snapsave0"}}
> {"timestamp": {"seconds": 1623082154, "microseconds": 351629}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "snapsave0"}}
> {"return": {}}
> {"timestamp": {"seconds": 1623082154, "microseconds": 369877}, "event":
> "STOP"}
> {"timestamp": {"seconds": 1623082186, "microseconds": 773379}, "event":
> "RESUME"}
> {"timestamp": {"seconds": 1623082186, "microseconds": 846654}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "snapsave0"}}
> {"timestamp": {"seconds": 1623082186, "microseconds": 846734}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "snapsave0"}}
> {"timestamp": {"seconds": 1623082186, "microseconds": 846766}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "snapsave0"}}
> 
> {"execute": "query-jobs"}
> {"return": [{"current-progress": 1, "status": "concluded", "total-progress":
> 1, "type": "snapshot-save", "id": "snapsave0"}]}
> 
> 
> {"execute": "snapshot-load", "arguments": {"job-id": "snapload0", "tag":
> "my-snap", "vmstate": "drive_image1", "devices": ["drive_image1"]}}
> {"timestamp": {"seconds": 1623082467, "microseconds": 522967}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "snapload0"}}
> {"timestamp": {"seconds": 1623082467, "microseconds": 523047}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "snapload0"}}
> {"return": {}}
> {"timestamp": {"seconds": 1623082467, "microseconds": 551096}, "event":
> "STOP"}
> {"timestamp": {"seconds": 1623082481, "microseconds": 920940}, "event":
> "RESUME"}
> {"timestamp": {"seconds": 1623082481, "microseconds": 923878}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "snapload0"}}
> {"timestamp": {"seconds": 1623082481, "microseconds": 923962}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "snapload0"}}
> {"timestamp": {"seconds": 1623082481, "microseconds": 924001}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "snapload0"}}
> 
> {"execute": "query-jobs"}
> {"return": [{"current-progress": 1, "status": "concluded", "total-progress":
> 1, "type": "snapshot-load", "id": "snapload0"}, {"current-progress": 1,
> "status": "concluded", "total-progress": 1, "type": "snapshot-save", "id":
> "snapsave0"}]}
> 
> {"execute": "query-status"}
> {"return": {"status": "running", "singlestep": false, "running": true}}
> 
> 
> {"execute": "snapshot-delete", "arguments": {"job-id": "snapdelete0", "tag":
> "my-snap", "devices": ["drive_image1"]}}  
> {"timestamp": {"seconds": 1623082730, "microseconds": 324380}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "snapdelete0"}}
> {"timestamp": {"seconds": 1623082730, "microseconds": 324468}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "snapdelete0"}}
> {"return": {}}
> {"timestamp": {"seconds": 1623082730, "microseconds": 397695}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "snapdelete0"}}
> {"timestamp": {"seconds": 1623082730, "microseconds": 397749}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "snapdelete0"}}
> {"timestamp": {"seconds": 1623082730, "microseconds": 397786}, "event":
> "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "snapdelete0"}}
> 
> {"execute": "query-jobs"}
> {"return": [{"current-progress": 1, "status": "concluded", "total-progress":
> 1, "type": "snapshot-delete", "id": "snapdelete0"}, {"current-progress": 1,
> "status": "concluded", "total-progress": 1, "type": "snapshot-load", "id":
> "snapload0"}, {"current-progress": 1, "status": "concluded",
> "total-progress": 1, "type": "snapshot-save", "id": "snapsave0"}]}

Comment 5 Kevin Wolf 2021-06-11 16:12:00 UTC
(In reply to Yanhui Ma from comment #4)
> Could you also help confirm that? Should we use new QMP commands on RHEL8.5?
> If so, we will modify cases.

I think as long as libvirt still uses the HMP commands, this is what we should be testing.

Maybe Daniel can tell us whether libvirt is likely to move to the new QMP commands in the RHEL 8 lifetime or whether this is only relevant for RHEL 9.

Comment 6 Daniel Berrangé 2021-06-11 16:27:04 UTC
(In reply to Kevin Wolf from comment #5)
> (In reply to Yanhui Ma from comment #4)
> > Could you also help confirm that? Should we use new QMP commands on RHEL8.5?
> > If so, we will modify cases.
> 
> I think as long as libvirt still uses the HMP commands, this is what we
> should be testing.
> 
> Maybe Daniel can tell us whether libvirt is likely to move to the new QMP
> commands in the RHEL 8 lifetime or whether this is only relevant for RHEL 9.

No plans to change it in RHEL-8.

It is still TBD whether RHEL-9 QEMU will even have internal snapshot turned on, but if it does, then we'll aim to make libvirt use the new APIs in 9.

Comment 9 Yanan Fu 2021-07-30 01:38:58 UTC
QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.

Comment 12 Tingting Mao 2021-08-02 09:05:24 UTC
Verified this bug as below.


Tested with:
qemu-kvm-6.0.0-26.module+el8.5.0+12044+525f0ebc
kernel-modules-4.18.0-325.el8.x86_64


Steps:
1. Boot guest 
/usr/libexec/qemu-kvm \
        -name 'guest' \
        -machine q35 \
        -nodefaults \
        -device VGA,bus=pcie.0,addr=0x2 \
        -object iothread,id=iothread0 \
        -device virtio-scsi-pci,id=scsi1,bus=pcie.0,addr=0x6,iothread=iothread0 \
        -blockdev driver=file,cache.direct=on,cache.no-flush=off,filename=./rhel850-64-virtio-scsi.qcow2,node-name=my_file \
        -blockdev driver=qcow2,node-name=my,file=my_file \
        -device scsi-hd,drive=my \
        -vnc :0 \
        -monitor stdio \
        -m 8192 \
        -smp 8 \
        -device virtio-net-pci,mac=9a:b5:b6:b1:b2:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pcie.0,addr=0x9  \
        -netdev tap,id=idxgXAlm \
        -chardev socket,server=on,path=/var/tmp/monitor-qmpmonitor1-20210721-024113-AsZ7KYro,id=qmp_id_qmpmonitor1,wait=off  \
        -mon chardev=qmp_id_qmpmonitor1,mode=control \
        -device pcie-root-port,id=pcie.0-root-port-8,slot=8,chassis=8,addr=0x8,bus=pcie.0 \


2. Save and load internal snapshot via QMP
# nc -U monitor-qmpmonitor1-20210721-024113-AsZ7KYro
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 6}, "package": "qemu-kvm-6.0.0-26.module+el8.5.0+12044+525f0ebc"}, "capabilities": ["oob"]}}
{"execute": "qmp_capabilities", "id": "kDiYPtqp"}
{"return": {}, "id": "kDiYPtqp"}
{"execute": "human-monitor-command", "arguments": {"command-line": "savevm sn1"}, "id": "JcqY2Shq"}
{"timestamp": {"seconds": 1627894565, "microseconds": 581536}, "event": "STOP"}
{"timestamp": {"seconds": 1627894568, "microseconds": 705247}, "event": "RESUME"}
{"return": "", "id": "JcqY2Shq"}
{"execute": "human-monitor-command", "arguments": {"command-line": "loadvm sn1"}, "id": "y5joUWoc"}
{"timestamp": {"seconds": 1627894583, "microseconds": 852503}, "event": "STOP"}
{"timestamp": {"seconds": 1627894586, "microseconds": 873033}, "event": "RESUME"}  -------------------------------> Resumed now.
{"return": "", "id": "y5joUWoc"}



Results:
loadvm via QMP works now.

Comment 14 errata-xmlrpc 2021-11-16 07:53:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4684


Note You need to log in before you can comment on or make changes to this bug.