Bug 1991671 - vmstate differs between -audiodev and QEMU_AUDIO_DRV when no sound frontends devs present.
Summary: vmstate differs between -audiodev and QEMU_AUDIO_DRV when no sound frontends ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.5
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 8.5
Assignee: Dr. David Alan Gilbert
QA Contact: Guo, Zhiyi
URL:
Whiteboard:
Depends On:
Blocks: 1957194 1977891 1992197
TreeView+ depends on / blocked
 
Reported: 2021-08-09 16:29 UTC by Daniel Berrangé
Modified: 2021-11-16 08:59 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-6.0.0-28.module+el8.5.0+12271+fffa967b
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1992197 (view as bug list)
Environment:
Last Closed: 2021-11-16 07:55:01 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-92935 0 None None None 2021-08-09 16:30:29 UTC
Red Hat Product Errata RHBA-2021:4684 0 None None None 2021-11-16 07:55:59 UTC

Description Daniel Berrangé 2021-08-09 16:29:35 UTC
Description of problem:

Consider the two ways of configuring the QEMU audio backend

$ QEMU_AUDIO_DRV=none ./qemu-system-x86_64 -cdrom ~/Downloads/Fedora-Workstation-Live-x86_64-32-1.6.iso -S -monitor stdio

and

$ qemu-system-x86_64 -cdrom ~/Downloads/Fedora-Workstation-Live-x86_64-32-1.6.iso -S -monitor stdio -audiodev none,id=a

These both result in the exact same guest ABI, since nothing about audio backend affects the guest. They have the exact same backend behaviour too.

When live migrating, however, their vmstate differs

The config using -audiodev contains an extra "audio" section in its vmstate.

This appears to be because -audiodev is processed unconditionally at QEMU startup, while $QEMU_AUDIO_DRV is processed lazily when the first sound device is added to the VM (whether at cold boot, or hotplug at runtime).

Due to the lazy init for $QEMU_AUDIO_DRV, the 'audio' section doesn't get added to vmstate.

Amuzingly/frustratingly the "audio" vmstate section is entirely empty, so it serves no functional purpose during live migration, except to break compatibility in this way.


This has unfortunately broken live migration backwards compatibility since libvirt switched to using -audiodev instead of $QEMU_AUDIO_DRV, since libvirt will configure an audio backend regardless of whether any sound frontend device exists.

Version-Release number of selected component (if applicable):
qemu-kvm-6.0.0-26.module+el8.5.0+12044+525f0ebc.x86_64

How reproducible:
Always

Steps to Reproduce:
1. $ QEMU_AUDIO_DRV=none ./qemu-system-x86_64 -cdrom ~/Downloads/Fedora-Workstation-Live-x86_64-32-1.6.iso -S -monitor stdio
2. migrate 'exec:dd of=env.img'
3. $ qemu-system-x86_64 -cdrom ~/Downloads/Fedora-Workstation-Live-x86_64-32-1.6.iso -S -monitor stdio -audiodev none,id=a
4. migrate 'exec:dd of=args.img'
5. scripts/analyze-migration.py -f env.img
6. scripts/analyze-migration.py -f args.img

Actual results:
Extra "audio" section in args.img

Expected results:
vmstate is identical for both ways of configuring audio backend.

Additional info:

Comment 1 Dr. David Alan Gilbert 2021-08-09 16:34:37 UTC
I think we might be able to fix this just by making a .needed on the audio vmstate to say never to send it.
Old qemu's should ignore the absence of one.

Comment 2 Dr. David Alan Gilbert 2021-08-09 17:40:18 UTC
Posted upstream as:

audio: Never send migration section

now, which downstream qemu's do you want that in?

Comment 3 Daniel Berrangé 2021-08-10 12:42:21 UTC
(In reply to Dr. David Alan Gilbert from comment #2)
> Posted upstream as:
> 
> audio: Never send migration section
> 
> now, which downstream qemu's do you want that in?

RHEL-AV 8.5 and RHEL-9.0  will both have libvirt versions which trigger this bug, so QEMU in those two streams.  Base RHEL 8.5 will still have older libvirt so doesn't need this in base QEMU.

Comment 4 Dr. David Alan Gilbert 2021-08-10 17:52:50 UTC
OK, created 1992197 as a 9 clone; although 9 final will pick it up anyway through rebase, so we might not have to do much.

Comment 5 Dr. David Alan Gilbert 2021-08-10 18:33:31 UTC
Fix went in upstream as da77adbaf619c4d170cb

Comment 7 Danilo de Paula 2021-08-11 13:53:04 UTC
QA ACK, please?

Comment 8 Guo, Zhiyi 2021-08-12 01:18:31 UTC
(In reply to Danilo Cesar Lemes de Paula from comment #7)
> QA ACK, please?

Done!

Comment 13 Yanan Fu 2021-08-19 06:35:22 UTC
QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.

Comment 14 Guo, Zhiyi 2021-08-19 08:01:21 UTC
Check this bug from libvirt layer:

pkgs on rhel8.4 host:
qemu-kvm-5.2.0-16.module+el8.4.0+10806+b7d97207.x86_64
libvirt-7.0.0-14.1.module+el8.4.0+11095+d46acebf.x86_64

pkgs on rhel8.5 host:
qemu-kvm-6.0.0-28.module+el8.5.0+12271+fffa967b.x86_64
libvirt-7.6.0-2.module+el8.5.0+12219+a5ea13d2.x86_64

Steps:
Boot a VM with qemu cli generated by libvirt on rhel 8.4 host:
# cat /var/log/libvirt/qemu/rhel84.log
2021-05-24-02:28:14, ), qemu version: 5.2.0qemu-kvm-5.2.0-16.module+el8.4.0+10806+b7d97207, kernel: 4.18.0-305.el8.x86_64, hostname: smicro-x12s-01.rhts.eng.pek2.re
dhat.com
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \
HOME=/var/lib/libvirt/qemu/domain-1-rhel84 \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-rhel84/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-rhel84/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-rhel84/.config \
QEMU_AUDIO_DRV=none \
/usr/libexec/qemu-kvm \
-name guest=rhel84,debug-threads=on \
...
-machine pc-q35-rhel8.4.0,accel=kvm,usb=off,dump-guest-core=off,memory-backend=pc.ram \
-cpu Skylake-Client,ss=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,mpx=on,umip=on,md-clear=on,xsaves=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,hle=off,rtm=off,a
vx512f=off,avx512dq=off,clwb=off,avx512cd=off,avx512bw=off,avx512vl=off,avx512vnni=off,avx512-bf16=off,taa-no=off \
...
-vnc 0.0.0.0:0 \
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 \
...

Then migrate this VM to rhel 8.5 host, check the qemu cli generated, it becomes:
2021-08-19 07:42:33.358+0000: starting up libvirt version: 7.6.0, package: 2.module+el8.5.0+12219+a5ea13d2 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 202
1-08-12-05:59:49, ), qemu version: 6.0.0qemu-kvm-6.0.0-28.module+el8.5.0+12271+fffa967b, kernel: 4.18.0-330.el8.x86_64, hostname: smicro-x12s-02.rhts.eng.pek2.redha
t.com
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \
HOME=/var/lib/libvirt/qemu/domain-1-rhel84 \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-rhel84/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-rhel84/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-rhel84/.config \
/usr/libexec/qemu-kvm \
-name guest=rhel84,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-rhel84/master-key.aes"}' \
-machine pc-q35-rhel8.4.0,accel=kvm,usb=off,dump-guest-core=off,memory-backend=pc.ram \
-cpu Skylake-Client,ss=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,mpx=on,umip=on,md-clear=on,xsaves=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,hle=off,rtm=off,avx512f=off,avx512dq=off,clwb=off,avx512cd=off,avx512bw=off,avx512vl=off,avx512vnni=off,avx512-bf16=off,taa-no=off \
...
-audiodev id=audio1,driver=none \
-vnc 0.0.0.0:0,audiodev=audio1 \
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 \
...

Then migrate this VM back, now the qemu cli has changed back to original with QEMU_AUDIO_DRV=none

Comment 15 Guo, Zhiyi 2021-08-19 08:20:39 UTC
Mark verified per comment 14 as ping-pong migration between 8.4 & 8.5 doesn't have audiodev/QEMU_AUDIO_DRV on the fix build qemu-kvm-6.0.0-28.module+el8.5.0+12271+fffa967b.x86_64.

Comment 17 errata-xmlrpc 2021-11-16 07:55:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4684


Note You need to log in before you can comment on or make changes to this bug.