Bug 1069784

Summary: block commit/pull support for disks using libgfapi volumes
Product: Red Hat Enterprise Linux 7 Reporter: Peter Krempa <pkrempa>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: dyuan, mzhan, rbalakri, shyu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.2.7-1.el7 Doc Type: Bug Fix
Doc Text:
Cause: libvirt's internal structures didn't store enough information to allow block operations (commit, pull) on networked storage and libvirt also lacked a way to represent network storage paths to these APIs Consequence: A user wasn't able to start block operations if a disk was backed by a backing chain containing networked storage. Fix: libvirt now provides a way to choose networked storage images in the backing chain and allows to start the block operations on them. Result: users are now able to use block commit and block pull with disks backed by gluster, rbd and such.
 Story Points: ---
Clone Of:
: 1069791 (view as bug list) Environment:
Last Closed: 2015-03-05 07:30:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1045196, 1069791    

Description Peter Krempa 2014-02-25 16:29:20 UTC 
Description of problem:
Currently there's no way to represent gluster storage paths to use with APIs doing block commit or block pull. Layered management applications need this support to delete and shorten snapshot image chains.

Version-Release number of selected component (if applicable):
libvirt-1.1.1-24.el7

See also:
https://bugzilla.redhat.com/show_bug.cgi?id=1069407
Comment 1 Peter Krempa 2014-02-25 16:30:50 UTC 
Relevant upstream discussion:

http://www.redhat.com/archives/libvir-list/2014-February/msg01226.html
Comment 5 Peter Krempa 2014-07-08 11:34:32 UTC 
Fixed upstream:

commit c6bf2f0ffc5d51f03eeb83103c323ff7949e8b59
Author: Peter Krempa <pkrempa>
Date:   Mon Apr 28 15:39:19 2014 +0200

    qemu: Add support for networked disks for block pull/block rebase
    
    Now that we are able to select images from the backing chain via indexed
    access we should also convert possible network sources to
    qemu-compatible strings before passing them to qemu.

commit 60244b56afb3438080b55ac838766af3a0037e03
Author: Peter Krempa <pkrempa>
Date:   Mon Apr 28 15:39:19 2014 +0200

    qemu: Add support for networked disks for block commit
    
    Now that we are able to select images from the backing chain via indexed
    access we should also convert possible network sources to
    qemu-compatible strings before passing them to qemu.

commit 37183e5db83fd476be552fd889edc74d82927981
Author: Peter Krempa <pkrempa>
Date:   Fri May 16 15:40:06 2014 +0200

    lib: Introduce flag VIR_DOMAIN_BLOCK_REBASE_RELATIVE
    
    Introduce flag for the block rebase API to allow the rebase operation to
    leave the chain relatively addressed. Also adds a virsh switch to enable
    this behavior.

commit bda44ca2ca5bf06721df07b09df4e102d929bd7c
Author: Peter Krempa <pkrempa>
Date:   Tue May 13 17:59:32 2014 +0200

    lib: Introduce flag VIR_DOMAIN_BLOCK_COMMIT_RELATIVE
    
    Introduce flag for the block commit API to allow the commit operation to
    leave the chain relatively addressed. Also adds a virsh switch to enable
    this behavior.

v1.2.6-67-gc6bf2f0
Comment 7 Shanzhi Yu 2014-12-31 06:31:18 UTC 
Verify this bug with libvirt-1.2.8-11.el7.x86_64
Steps as below

Scenario I Block commit

1. Both source file and new snapshot file based on glusterfs
1.1 Prepare a running domain with source file based on glusterfs

# virsh dumpxml rh7 |grep disk -A 6
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>

1.2 Create three snapshot, new snapshot file based on glusterfs

# cat s1.xml 
<domainsnapshot>
<name>SNAP</name>
<disks>
<disk name='vda' type='network'>
<driver type='qcow2'/>
<source protocol='gluster' name='gluster-vol1/r7-raw.SNAP'>
<host name='10.66.5.38'/>
</source>
</disk>
</disks>
</domainsnapshot>

#  for i in s1 s2 s3;do sed -e s/SNAP/$i/g s1.xml >$i-snap.xml;virsh snapshot-create rh7 $i-snap.xml --disk-only;done
Domain snapshot s1 created from 's1-snap.xml'
Domain snapshot s2 created from 's2-snap.xml'
Domain snapshot s3 created from 's3-snap.xml'

# virsh dumpxml rh7|grep disk -A 24
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore type='network' index='1'>
        <format type='qcow2'/>
        <source protocol='gluster' name='gluster-vol1/r7-raw.s2'>
          <host name='10.66.5.38'/>
        </source>
        <backingStore type='network' index='2'>
          <format type='qcow2'/>
          <source protocol='gluster' name='gluster-vol1/r7-raw.s1'>
            <host name='10.66.5.38'/>
          </source>
          <backingStore type='network' index='3'>
            <format type='raw'/>
            <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
              <host name='10.66.5.38'/>
            </source>
            <backingStore/>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

1.3 Do blockcommit on inactive layer

# virsh blockcommit rh7 vda --base vda[2] --top vda[1] --verbose --wait 
Block Commit: [100 %]
Now in synchronized phase

# virsh dumpxml rh7|grep disk -A 18
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore type='network' index='1'>
        <format type='qcow2'/>
        <source protocol='gluster' name='gluster-vol1/r7-raw.s1'>
          <host name='10.66.5.38'/>
        </source>
        <backingStore type='network' index='2'>
          <format type='raw'/>
          <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
            <host name='10.66.5.38'/>
          </source>
          <backingStore/>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

1.4 Do blockcommit on active layer
# virsh blockcommit rh7 vda --active --pivot --verbose --wait 
Block Commit: [100 %]
Successfully pivoted

# virsh dumpxml rh7|grep disk -A 6
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>

2. Source file based on glusterfs while new snapshot file based on local

2.1 Prepare a running domain with source file based on glusterfs

# virsh dumpxml rh7|grep disk -A 6 
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>

2.2 Create three external disk snapshot with new snapshot file based on local
# for i in 1 2 3;do virsh snapshot-create-as rh7 s$i --disk-only --diskspec vda,snapshot=external,file=/var/lib/libvirt/images/rh7-raw.s$i --atomic;done
 
# virsh dumpxml rh7|grep disk -A 18
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/rh7-raw.s3'/>
      <backingStore type='file' index='1'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/rh7-raw.s2'/>
        <backingStore type='file' index='2'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/rh7-raw.s1'/>
          <backingStore type='network' index='3'>
            <format type='raw'/>
            <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
              <host name='10.66.5.38'/>
            </source>
            <backingStore/>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

2.3 Do blockcommit on inactive layer
# virsh blockcommit rh7 vda --base vda[2] --top vda[1] --verbose --wait Block Commit: [100 %]
Now in synchronized phase

# virsh dumpxml rh7|grep disk -A 16
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/rh7-raw.s3'/>
      <backingStore type='file' index='1'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/rh7-raw.s1'/>
        <backingStore type='network' index='2'>
          <format type='raw'/>
          <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
            <host name='10.66.5.38'/>
          </source>
          <backingStore/>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

2.4 Do blockcommit on active layer
# virsh blockcommit rh7 vda --active --pivot --verbose --wait Block Commit: [100 %]
Successfully pivoted

# virsh dumpxml rh7|grep disk -A 6
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>

3. Source file based on local while new snapshot file based on glusterfs
3.1 Prepare a running domain with source file based on local

# virsh dumpxml rhel7|grep disk -A 6
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/r7-qcow2.img'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>

3.2 Create three external disk snapshot with new snapshot file baesd on glusterfs

# for i in s1 s2 s3;do sed -e s/SNAP/$i/g snap.xml >$i-snap.xml;virsh snapshot-create rhel7 $i-snap.xml --disk-only;done
Domain snapshot s1 created from 's1-snap.xml'
Domain snapshot s2 created from 's2-snap.xml'
Domain snapshot s3 created from 's3-snap.xml'

# cat snap.xml 
<domainsnapshot>
    <name>SNAP</name>
<disks>
<disk name='vda' type='network'>
<driver type='qcow2'/>
<source protocol='gluster' name='gluster-vol1/r7-qcow2.SNAP'>
<host name='10.66.5.38'/>
</source>
</disk>
</disks>
</domainsnapshot>

# virsh dumpxml rhel7|grep disk -A 24
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-qcow2.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore type='network' index='1'>
        <format type='qcow2'/>
        <source protocol='gluster' name='gluster-vol1/r7-qcow2.s2'>
          <host name='10.66.5.38'/>
        </source>
        <backingStore type='network' index='2'>
          <format type='qcow2'/>
          <source protocol='gluster' name='gluster-vol1/r7-qcow2.s1'>
            <host name='10.66.5.38'/>
          </source>
          <backingStore type='file' index='3'>
            <format type='qcow2'/>
            <source file='/var/lib/libvirt/images/r7-qcow2.img'/>
            <backingStore/>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

3.3 Do blockcommit on inactive layer
# virsh blockcommit rhel7 vda --base vda[2] --top vda[1] --verbose --wait 
Block Commit: [100 %]
Now in synchronized phase

# virsh dumpxml rhel7|grep disk -A 16
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-qcow2.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore type='network' index='1'>
        <format type='qcow2'/>
        <source protocol='gluster' name='gluster-vol1/r7-qcow2.s1'>
          <host name='10.66.5.38'/>
        </source>
        <backingStore type='file' index='2'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/r7-qcow2.img'/>
          <backingStore/>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

3.4 Do blockcommit on active layer
# virsh blockcommit rhel7 vda --active --pivot --verbose --wait 
Block Commit: [100 %]
Successfully pivoted

# virsh dumpxml rhel7|grep disk -A 6
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/r7-qcow2.img'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>


Scenario II Block pull

1. Both source file and new snapshot file based on glusterfs

NB:Step 1.1 1.2 is same like step 1.1 and 1.2 in Scenario I 

1.1 

1.2

# virsh dumpxml rh7|grep disk -A 24
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore type='network' index='1'>
        <format type='qcow2'/>
        <source protocol='gluster' name='gluster-vol1/r7-raw.s2'>
          <host name='10.66.5.38'/>
        </source>
        <backingStore type='network' index='2'>
          <format type='qcow2'/>
          <source protocol='gluster' name='gluster-vol1/r7-raw.s1'>
            <host name='10.66.5.38'/>
          </source>
          <backingStore type='network' index='3'>
            <format type='raw'/>
            <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
              <host name='10.66.5.38'/>
            </source>
            <backingStore/>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

1.3 Do blockpull

# virsh blockpull rh7 vda --base vda[2] --verbose --wait 
Block Pull: [100 %]
Pull complete

# virsh dumpxml rh7|grep disk -A 24
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore type='network' index='1'>
        <format type='qcow2'/>
        <source protocol='gluster' name='gluster-vol1/r7-raw.s1'>
          <host name='10.66.5.38'/>
        </source>
        <backingStore type='network' index='2'>
          <format type='raw'/>
          <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
            <host name='10.66.5.38'/>
          </source>
          <backingStore/>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

# virsh blockpull rh7 vda  --verbose --wait 
Block Pull: [100 %]
Pull complete

# virsh dumpxml rh7|grep disk -A 6
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-raw.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>

2. Source file based on glusterfs while new snapshot file based on local
NB:
Step 2.1 2.2 is same like step 2.1 and 2.2 in Scenario I

2.1
2.2
# virsh dumpxml rh7 |grep disk -A 24
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/rh7-raw.s3'/>
      <backingStore type='file' index='1'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/rh7-raw.s2'/>
        <backingStore type='file' index='2'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/rh7-raw.s1'/>
          <backingStore type='network' index='3'>
            <format type='raw'/>
            <source protocol='gluster' name='gluster-vol1/r7-raw.img'>
              <host name='10.66.5.38'/>
            </source>
            <backingStore/>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>


2.3 Do blockpull

# virsh blockpull rh7 vda --verbose --wait 
Block Pull: [100 %]
Pull complete
# virsh dumpxml rhel7|grep disk -A 6
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/r7-qcow2.img'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>


3. Source file based on local while new snapshot file based on glusterfs
NB:
Step 3.1 3.2 is same like step 3.1 and 3.2 in Scenario I

3.1
3.2
# virsh dumpxml rhel7|grep disk -A 22
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-qcow2.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore type='network' index='1'>
        <format type='qcow2'/>
        <source protocol='gluster' name='gluster-vol1/r7-qcow2.s2'>
          <host name='10.66.5.38'/>
        </source>
        <backingStore type='network' index='2'>
          <format type='qcow2'/>
          <source protocol='gluster' name='gluster-vol1/r7-qcow2.s1'>
            <host name='10.66.5.38'/>
          </source>
          <backingStore type='file' index='3'>
            <format type='qcow2'/>
            <source file='/var/lib/libvirt/images/r7-qcow2.img'/>
            <backingStore/>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>

3.3 Do blockpull 

# virsh blockpull rhel7 vda --verbose --wait 
Block Pull: [100 %]
Pull complete

# virsh dumpxml rhel7|grep disk -A 6
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/r7-qcow2.s3'>
        <host name='10.66.5.38'/>
      </source>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>


Given above test, I'd like change this bug to VERIFIED status
Comment 9 errata-xmlrpc 2015-03-05 07:30:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html