Bug 1977891 - libvirt can't be migrated back due to new audio element in domain definition
Summary: libvirt can't be migrated back due to new audio element in domain definition
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.5
Hardware: All
OS: Unspecified
high
urgent
Target Milestone: rc
: 8.5
Assignee: Daniel Berrangé
QA Contact: Lili Zhu
URL:
Whiteboard:
: 1981431 (view as bug list)
Depends On: 1991671 1992197
Blocks: 1969483
TreeView+ depends on / blocked
 
Reported: 2021-06-30 16:21 UTC by smitterl
Modified: 2022-07-21 17:00 UTC (History)
19 users (show)

Fixed In Version: libvirt-7.6.0-1.module+el8.5.0+12097+2c77910b
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-16 07:54:28 UTC
Type: Bug
Target Upstream Version: 7.6.0
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2021:4684 0 None None None 2021-11-16 07:55:15 UTC

Internal Links: 2109350

Description smitterl 2021-06-30 16:21:14 UTC
Description of problem:
virsh define adds an audio element to the domain definition. Therefore, the domain can't be migrated back.


Version-Release number of selected component (if applicable):
source host: libvirt-daemon-7.0.0-14.1.module+el8.4.0+11095+d46acebf
target host: libvirt-7.4.0-1.module+el8.5.0+11218+83343022

How reproducible:
100%

Steps to Reproduce:
vm.xml is a valid domain definition, name 'vm', not containing any element <audio>
1. On source host: virsh define vm.xml
2. Migrate machine to target host
3. On target host: virsh dumpxml vm|grep audio

Actual results:
    <audio id='1' type='none'/>

4. Try migrate back
error: unsupported configuration: unknown audio type 'none'


Expected results:
Migration back succeeds.

Additional info:
The corresponding qemu version: qemu-kvm-6.0.0-21.module+el8.5.0+11555+e0ab0d09
The domain can be started successfully on libvirt-7.4.0-1 with that audio element.

Comment 5 Daniel Berrangé 2021-06-30 16:34:41 UTC
(In reply to smitterl from comment #0)
> Description of problem:
> virsh define adds an invalid audio element to the domain definition.
> Therefore, the domain can't start or be used after migration.
> 
> 
> Version-Release number of selected component (if applicable):
> libvirt-7.4.0-1.module+el8.5.0+11218+83343022.x86_64
> libvirt-7.4.0-1.module+el8.5.0+11218+83343022.s390x
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> vm.xml is a valid domain definition, name 'vm', not containing any element
> <audio>
> 1. virsh define vm.xml
> 2. virsh dumpxml vm|grep audio
> 
> Actual results:
>     <audio id='1' type='none'/>
> 
> 
> 
> Expected results:
> empty
> 
> Additional info:
> 1. The domain can't start afterwards as element is invalid

What do you mean by "invalid" here ?  Can you give the actual error message you receive.

A <audio type='none'/> is valid in all my own tests


> 2. When I migrate a machine to another host with
> libvirt-7.4.0-1.module+el8.5.0+11218+83343022.s390x, the running destination
> vm contains that audio element too. The domain can't be migrated back.

Can you give the error you get from migration ?

Comment 6 smitterl 2021-06-30 16:50:00 UTC
(In reply to Daniel Berrangé from comment #5)
> (In reply to smitterl from comment #0)
> > Description of problem:
> > virsh define adds an invalid audio element to the domain definition.
> > Therefore, the domain can't start or be used after migration.
> > 
> > 
> > Version-Release number of selected component (if applicable):
> > libvirt-7.4.0-1.module+el8.5.0+11218+83343022.x86_64
> > libvirt-7.4.0-1.module+el8.5.0+11218+83343022.s390x
> > 
> > How reproducible:
> > 100%
> > 
> > Steps to Reproduce:
> > vm.xml is a valid domain definition, name 'vm', not containing any element
> > <audio>
> > 1. virsh define vm.xml
> > 2. virsh dumpxml vm|grep audio
> > 
> > Actual results:
> >     <audio id='1' type='none'/>
> > 
> > 
> > 
> > Expected results:
> > empty
> > 
> > Additional info:
> > 1. The domain can't start afterwards as element is invalid
> 
> What do you mean by "invalid" here ?  Can you give the actual error message
> you receive.
> 
> A <audio type='none'/> is valid in all my own tests
> 
> 
> > 2. When I migrate a machine to another host with
> > libvirt-7.4.0-1.module+el8.5.0+11218+83343022.s390x, the running destination
> > vm contains that audio element too. The domain can't be migrated back.
> 
> Can you give the error you get from migration ?

I updated the description and title. I got confused between the different hosts and versions I worked with, sorry. As you said offline, migrating back from AV 8.5 to AV 8.4 doesn't work.

Comment 7 Han Han 2021-07-01 02:07:38 UTC
Look like this issue https://gitlab.com/libvirt/libvirt/-/issues/179

Comment 8 Jaroslav Suchanek 2021-07-01 14:09:50 UTC
Dan - would you please take care of this one. Thanks.

Comment 9 smitterl 2021-07-06 11:53:15 UTC
Additional information: the audio element cannot be detached from the running machine. I get an error
`# virsh detach-device avocado-vt-remote-vm1 audio.xml --live
error: Failed to detach device from audio.xml
error: Operation not supported: live detach of device 'audio' is not supported
`
Therefore, I can't think of a workaround to fix the vm setup before migrating back but I might be missing something.

Comment 10 Daniel Berrangé 2021-07-06 12:08:38 UTC
The 'virsh migrate' command allows you to provide a replacement XML document to use for the guest config on the target host. It is probably possible to take the current guest XML 'virsh dumpxml --migratable guestname', remove the <audio> element(s) and then pass this document to 'virsh migrate'.

Comment 11 smitterl 2021-07-06 16:50:18 UTC
(In reply to Daniel Berrangé from comment #10)
> The 'virsh migrate' command allows you to provide a replacement XML document
> to use for the guest config on the target host. It is probably possible to
> take the current guest XML 'virsh dumpxml --migratable guestname', remove
> the <audio> element(s) and then pass this document to 'virsh migrate'.

IIUC, you are referring to the --xml argument. I tried this but it didn't work either:

# cat migratable.xml |grep audio
>>empty
# virsh dumpxml avocado-vt-vm1|grep audio
    <audio id='1' type='none'/>
# virsh migrate avocado-vt-vm1 --live --verbose --postcopy qemu+ssh://<my-dest-ip>/system --xml migratable.xml 
error: unsupported configuration: unknown audio type 'none'

Comment 12 smitterl 2021-07-06 16:51:11 UTC
(--persistent-xml migratable --persistent raise the same error)

Comment 16 Daniel Berrangé 2021-07-08 10:05:47 UTC
Upstream patch proposed https://listman.redhat.com/archives/libvir-list/2021-July/msg00158.html

Comment 18 Peter Krempa 2021-07-12 14:31:28 UTC
*** Bug 1981431 has been marked as a duplicate of this bug. ***

Comment 21 Lili Zhu 2021-08-09 12:39:52 UTC
Pkgs on source host:
libvirt-daemon-7.6.0-1.module+el8.5.0+12097+2c77910b.x86_64
qemu-kvm-6.0.0-26.module+el8.5.0+12044+525f0ebc.x86_64

Pkgs on target host:
libvirt-7.0.0-14.3.module+el8.4.0+11878+84e54169.x86_64
qemu-kvm-5.2.0-16.module+el8.4.0+11923+e8b883e4.4.x86_64

1. define a guest with the vnc graphics, start the guest, check guest xml
# virsh dumpxml avocado-vt-vm1
...
 <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <audio id='1' type='none'/>    <=== default audio device for vnc graphics has been added
...

2. check the migratable xml
# virsh dumpxml avocado-vt-vm1 --migratable |grep "<audio"
(no output)

3. migrate the guest to target host
# virsh migrate avocado-vt-vm1 qemu+ssh://$hostname/system --verbose --live --p2p
Migration: [100 %]error: internal error: qemu unexpectedly closed the monitor: 2021-08-09T12:29:54.186850Z qemu-kvm: Unknown savevm section or instance 'audio' 0. Make sure that your current VM setup matches your saved VM setup, including any hotplugged devices
2021-08-09T12:29:54.187115Z qemu-kvm: load of migration failed: Invalid argument

Hi, Daniel
Please help to check the above testing result. Thanks

Comment 22 Daniel Berrangé 2021-08-09 16:34:12 UTC
(In reply to Lili Zhu from comment #21)
> 3. migrate the guest to target host
> # virsh migrate avocado-vt-vm1 qemu+ssh://$hostname/system --verbose --live
> --p2p
> Migration: [100 %]error: internal error: qemu unexpectedly closed the
> monitor: 2021-08-09T12:29:54.186850Z qemu-kvm: Unknown savevm section or
> instance 'audio' 0. Make sure that your current VM setup matches your saved
> VM setup, including any hotplugged devices
> 2021-08-09T12:29:54.187115Z qemu-kvm: load of migration failed: Invalid
> argument
> 
> Hi, Daniel
> Please help to check the above testing result. Thanks

This is an unexpected problem in QEMU which causes vmstate changes when QEMU has no sound frontend device present. I filed bug 1991671 to track this problem.

We'll keep this libvirt bug open until we see the QEMU one solved

Comment 23 Lili Zhu 2021-08-21 09:46:06 UTC
Pkgs on source host:
libvirt-7.6.0-2.module+el8.5.0+12219+a5ea13d2.x86_64
qemu-kvm-6.0.0-28.module+el8.5.0+12271+fffa967b.x86_64

Pkgs on target host:
libvirt-7.0.0-14.1.module+el8.4.0+11095+d46acebf.x86_64
qemu-kvm-5.2.0-16.module+el8.4.0+10806+b7d97207.x86_64

1. define a guest with the vnc graphics, start the guest, check guest xml
# virsh dumpxml avocado-vt-vm1
...
 <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <audio id='1' type='none'/>    <=== default audio device for vnc graphics has been added
...

2. check the migratable xml
# virsh dumpxml avocado-vt-vm1 --migratable |grep "<audio"
(no output)

3. migrate the guest to target host
# virsh migrate avocado-vt-vm1 qemu+ssh://$hostname/system --verbose --live --p2p
Migration: [100 %]

4.

Comment 24 Lili Zhu 2021-08-21 09:47:24 UTC
As the testing result matches with the expected result, mark the bug as verified.

Comment 26 errata-xmlrpc 2021-11-16 07:54:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4684


Note You need to log in before you can comment on or make changes to this bug.