Bug 2130089

Summary: libvirt did not do storage image pre-creation on the target host when doing storage migration
Product: Red Hat Enterprise Linux 9 Reporter: yafu <yafu>
Component: libvirtAssignee: Peter Krempa <pkrempa>
libvirt sub component: Live Migration QA Contact: Fangge Jin <fjin>
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: unspecified CC: jdenemar, lcheng, lmen, pkrempa, virt-maint, xuzhang, yalzhang
Version: 9.2Keywords: Automation, AutomationTriaged, Regression, Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-8.8.0-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-09 07:27:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version: 8.8.0
Embargoed:

Description yafu 2022-09-27 07:22:16 UTC
Description of problem:
libvirt did not do storage image pre-creation on the target host when doing storage migration

Version-Release number of selected component (if applicable):
libvirt-8.7.0-1.el9.x86_64
qemu-kvm-7.1.0-1.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Prepare a pool with the same path both on source and target host:
#virsh pool-list
 Name       State    Autostart
--------------------------------
 default    active   yes

#virsh pool-dumpxml default
<pool type='dir'>
  <name>default</name>
  <uuid>c98f3a9d-07a3-4447-bb0d-95d4deed55fc</uuid>
  <capacity unit='bytes'>75125227520</capacity>
  <allocation unit='bytes'>34892738560</allocation>
  <available unit='bytes'>40232488960</available>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images</path>
    <permissions>
      <mode>0711</mode>
      <owner>0</owner>
      <group>0</group>
      <label>system_u:object_r:virt_image_t:s0</label>
    </permissions>
  </target>
</pool>


2.Start a guest with the images in the pool dir:
#virsh start avocado-vt-vm1

# virsh domblklist avocado-vt-vm1
 Target   Source
--------------------------------------------------------
 vda      /var/lib/libvirt/images/jeos-27-x86_64.qcow2


3.Do storage migration:
# virsh migrate avocado-vt-vm1 qemu+ssh://**.**.com/system --live --verbose --copy-storage-all 
error: Path '/var/lib/libvirt/images/jeos-27-x86_64.qcow2' is not accessible: No such file or directory

Actual results:
Migration failed because libvirt did not pre-create storage image on the target host.

Expected results:
Migration should complete successfully.

Additional info:
1.Can not reproduce with libvirt-8.5.0-6.el9.x86_64 qemu-kvm-7.0.0-12.el9.x86_64.

Comment 1 Peter Krempa 2022-09-27 07:32:59 UTC
This should be the same root cause as https://bugzilla.redhat.com/show_bug.cgi?id=2125111

In that BZ it seems that the storage was pre-created by the user, but another wrong code path was hit.

Should be fixed by:

commit 83ffeae75a0dfe8077d5e98f8a48615c62db0284
Author: Peter Krempa <pkrempa>
Date:   Thu Sep 8 11:55:08 2022 +0200

    qemu: migration: Fix setup of non-shared storage migration in qemuMigrationSrcBeginPhase
    
    In commit 6111b2352242e9 removing pre-blockdev code paths I've
    improperly refactored the setup of non-shared storage migration.
    
    Specifically the code checking that there are disks and setting up the
    NBD data in the migration cookie was originally outside of the loop
    checking the user provided list of specific disks to migrate, but became
    part of the block as it was not un-indented when a higher level block
    was being removed.
    
    The above caused that if non-shared storage migration is requested, but
    the user doesn't provide the list of disks to migrate (thus implying to
    migrate every appropriate disk) the code doesn't actually setup the
    migration and then later on falls back to the old-style migration which
    no longer works with blockdev.
    
    Move the check that there's anything to migrate out of the
    'nmigrate_disks' block.
    
    Fixes: 6111b2352242e93c6d2c29f9549d596ed1056ce5
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2125111
    Resolves: https://gitlab.com/libvirt/libvirt/-/issues/373
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

v8.7.0-93-g83ffeae75a

I'll keep this one open for now to see whether it's properly fixed too.

Comment 2 Peter Krempa 2022-09-27 07:36:11 UTC
You can verify that the migration code still works properly by adding '--migrate-disks vda'. This works around the broken setup code and the rest of the migration should work fine.

Comment 4 Fangge Jin 2022-10-11 07:13:22 UTC
Pre-verified on libvirt-8.8.0-1.el9.x86_64

Comment 8 Fangge Jin 2022-11-08 09:41:48 UTC
Verified with libvirt-8.9.0-2.el9.x86_64

Comment 10 errata-xmlrpc 2023-05-09 07:27:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (libvirt bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2171