Bug 1072653

Summary: vol-upload should change the volume target format type after uploading a different format file to it
Product: Red Hat Enterprise Linux 7 Reporter: chhu
Component: libvirtAssignee: John Ferlan <jferlan>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: low Docs Contact:
Priority: unspecified    
Version: 7.0CC: ajia, dyuan, eblake, mzhan, pzhang, rbalakri, shyu, xuzhang, yanyang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.8-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 07:31:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1176182    

Description chhu 2014-03-05 01:56:00 UTC
Description:
After running vol-upload to upload a raw image file to a volume, whose target format type is qcow2, the volume's target format type is not changed, without doing pool-refresh manually, run vol-clone/vol-create-from with this volume failed with error.

Version-Release number of selected component (if applicable):
libvirt-1.1.1-25.el7.x86_64
qemu-kvm-rhev-1.5.3-50.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. create two images for the default pool
# qemu-img create -f qcow2 test-qcow2.img 4G
Formatting 'test-qcow2.img', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off
# qemu-img create  test.img 2G
Formatting 'test.img', fmt=raw size=2147483648

2. check the new created images are as volumes in default pool
# virsh pool-refresh default
Pool default refreshed

# virsh vol-list default|grep test  
test-qcow2.img       /var/lib/libvirt/images/test-qcow2.img  
test.img             /var/lib/libvirt/images/test.img

# virsh vol-info test-qcow2.img default
Name:           test-qcow2.img
Type:           file
Capacity:       4.00 GiB
Allocation:     196.00 KiB

# virsh vol-info test.img default
Name:           test.img
Type:           file
Capacity:       2.00 GiB
Allocation:     0.00 B

# virsh vol-dumpxml test-qcow2.img default| grep format
    <format type='qcow2'/>
# virsh vol-dumpxml test.img default| grep format
    <format type='raw'/>

3. run vol-upload to upload the raw image file test.img to the test-qcow2.img volume, the vol-upload successed without any output message, the volume allocation is changed, but the volume target format type is not changed.

# virsh vol-upload test-qcow2.img test.img default
# virsh vol-info test-qcow2.img default
Name:           test-qcow2.img
Type:           file
Capacity:       4.00 GiB
Allocation:     2.00 GiB
# virsh vol-dumpxml test-qcow2.img default
<volume type='file'>
  <name>test-qcow2.img</name>
  <key>/var/lib/libvirt/images/test-qcow2.img</key>
  <source>
  </source>
  <capacity unit='bytes'>4294967296</capacity>
  <allocation unit='bytes'>2147680256</allocation>
  <target>
    <path>/var/lib/libvirt/images/test-qcow2.img</path>
    <format type='qcow2'/
.......

4. run vol-clone/ vol-create-from with the changed volume: test-qcow2.img. Met the error: "Image is not in qcow2 format"

# virsh vol-clone test-qcow2.img test-qcow2-upclone.img default
error: Failed to clone vol from test-qcow2.img
error: internal error: Child process (/usr/bin/qemu-img convert -f qcow2 -O qcow2 -o
compat=1.1 /var/lib/libvirt/images/test-qcow2.img /var/lib/libvirt/images/test-qcow2-upclone.img)
 unexpected exit status 1: qemu-img: Could not open '/var/lib/libvirt/images/test-qcow2.img':
Image is not in qcow2 format
qemu-img: Could not open '/var/lib/libvirt/images/test-qcow2.img'


Expected results:
In step3: the vol-upload return with output messages, upload successed, or upload a different format file to a volume is forbidden.
The volume target format is changed if the vol-upload is successed, the same as the volume allocation.

Actual results:
In step3: the vol-upload return without output messages, the volume allocation is changed, the volume target format is not changed.

Comment 1 Yang Yang 2014-03-12 07:36:50 UTC
The target image format and capacity are changed as expected after refreshing pool. 

However, no any messages are output after vol-upload returns.
I hit the similar issue when i run vol-download.

# virsh vol-download qcow3-vol /var/lib/libvirt/images/raw default

No any messages are printed after runing vol-download.

Comment 3 John Ferlan 2014-07-24 20:45:01 UTC
Not sure volume upload between different formats was 

Also of note (assuming you are in the directory for the 'default' pool):

$ qemu-img create -f qcow2 test-qcow2-4g.img 4G
$ qemu-img info /home/vm-images/test-qcow2-4g.img
image: /home/vm-images/test-qcow2-4g.img
file format: qcow2
virtual size: 4.0G (4294967296 bytes)
disk size: 196K
cluster_size: 65536
$ qemu-img create -f qcow2 test-qcow2-2g.img 2G
$ qemu-img info test-qcow2-2g.img
image: test-qcow2-2g.img
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 196K
cluster_size: 65536
$ virsh vol-refresh default
$ virsh vol-upload test-qcow2-4g.img test-qcow2-2g.img default
$ qemu-img info test-qcow2-4g.img
image: test-qcow2-4g.img
file format: qcow2
virtual size: 2.0G (2147483648 bytes)
disk size: 196K
cluster_size: 65536
$ virsh vol-info test-qcow2-4g.img default
Name:           test-qcow2-4g.img
Type:           file
Capacity:       4.00 GiB
Allocation:     196.00 KiB
$


Not so sure uploading anything into a qcow2 file will do what is expected - although there is the guestfish tool which may be able to handle the job.

Of course that's the issue here - should we disallow - that's something I'll continue to investigate... It might be possible to use the convert/resize options as well in order to do what's desired; however, whether it's "worth the effort" or there's "enough" space in the pool for the conversion is an issue that has to be considered.

Comment 4 John Ferlan 2014-07-28 13:33:51 UTC
Posted a possible solution upstream:

http://www.redhat.com/archives/libvir-list/2014-July/msg01395.html

Comment 5 John Ferlan 2014-08-04 14:47:38 UTC
Pushed upstream:

commit 4a85bf3e2fa703fdc14e8c49d5017ef04832a1d7
Author: John Ferlan <jferlan>
Date:   Mon Jul 28 08:39:09 2014 -0400

    storage: Refresh storage pool after upload
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1072653
    
    Upon successful upload of a volume, the target volume and storage pool
    were not updated to reflect any changes as a result of the upload. Make
    use of the existing stream close callback mechanism to force a backend
    pool refresh to occur in a separate thread once the stream closes. The
    separate thread should avoid potential deadlocks if the refresh needed
    to wait on some event from the event loop which is used to perform
    the stream callback.


$ git describe
v1.2.7-5-g4a85bf3

Comment 7 Eric Blake 2014-12-22 17:29:53 UTC
The patch for this bug caused CVE-2014-8135.

Comment 8 Pei Zhang 2014-12-30 02:56:19 UTC
Verify Version :
libvirt-1.2.8-11.el7.x86_64
qemu-kvm-rhev-2.1.2-17.el7.x86_64
kernel-3.10.0-220.el7.x86_64

Verify steps :

1.create two different types volumes in default pool

# qemu-img create -f qcow2 qcow2.img 4G
Formatting 'qcow2.img', fmt=qcow2 size=4294967296 encryption=off cluster_size=65536 lazy_refcounts=off 
# qemu-img create -f raw raw.img 2G
Formatting 'raw.img', fmt=raw size=2147483648 

# qemu-img info qcow2.img 
image: qcow2.img
file format: qcow2
virtual size: 4.0G (4294967296 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
# qemu-img info raw.img 
image: raw.img
file format: raw
virtual size: 2.0G (2147483648 bytes)
disk size: 0

1.1 check vol info
# virsh vol-list default --details
 Name         Path                                 Type  Capacity  Allocation
------------------------------------------------------------------------------
 qcow2.img    /var/lib/libvirt/images/qcow2.img    file  4.00 GiB  196.00 KiB
 raw.img      /var/lib/libvirt/images/raw.img      file  2.00 GiB      0.00 B


# virsh vol-dumpxml qcow2.img --pool default
<volume type='file'>
 ......
  <capacity unit='bytes'>4294967296</capacity>
  <allocation unit='bytes'>200704</allocation>
  <target>
    <path>/var/lib/libvirt/images/qcow2.img</path>
    <format type='qcow2'/>   <===qcow2 
    ......
</volume>
# virsh vol-dumpxml raw.img --pool default
<volume type='file'>
  ......
  <capacity unit='bytes'>2147483648</capacity>
  <allocation unit='bytes'>0</allocation>
  <target>
    <path>/var/lib/libvirt/images/raw.img</path>
    <format type='raw'/>  <===raw 
  ......
</volume>

2.try to upload volume of qcow2 format to raw

# virsh vol-upload --vol raw.img qcow2.img --pool default

3.after upload successfully , check raw.img volume info

# qemu-img info raw.img 
image: raw.img
file format: qcow2 <=== qcow2 type 
virtual size: 4.0G (4294967296 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false

4.check volume info 

# virsh vol-list default --details
 Name         Path                                 Type  Capacity  Allocation
------------------------------------------------------------------------------
 qcow2.img    /var/lib/libvirt/images/qcow2.img    file  4.00 GiB  196.00 KiB
 raw.img      /var/lib/libvirt/images/raw.img      file  4.00 GiB  196.00 KiB

# virsh vol-dumpxml raw.img --pool default
<volume type='file'>
  ......
  <capacity unit='bytes'>4294967296</capacity>
  <allocation unit='bytes'>200704</allocation>
  <target>
    <path>/var/lib/libvirt/images/raw.img</path>
    <format type='qcow2'/>  <===it is qcow2 
    ......
</volume>

5. try to operate uploaded volume .do vol-clone successfully

# virsh vol-clone raw.img raw.img.clone default
Vol raw.img.clone cloned from raw.img

# virsh vol-info raw.img.clone  default
Name:           raw.img.clone
Type:           file
Capacity:       4.00 GiB
Allocation:     196.00 KiB

try to upload raw to qcow2

1.prepare two volumes

# virsh vol-list default --details
 Name         Path                                 Type  Capacity  Allocation
------------------------------------------------------------------------------
 qcow2.img    /var/lib/libvirt/images/qcow2.img    file  1.00 GiB  196.00 KiB
 raw.img      /var/lib/libvirt/images/raw.img      file  1.00 GiB      0.00 B

2.upload volume of raw format to qcow2. upload successfully.
# virsh vol-upload --vol qcow2.img raw.img  default

3.check volume info

# qemu-img info qcow2.img 
image: qcow2.img
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G

# virsh vol-list default --details
 Name         Path                                 Type  Capacity  Allocation
------------------------------------------------------------------------------
 qcow2.img    /var/lib/libvirt/images/qcow2.img    file  1.00 GiB    1.00 GiB 
 raw.img      /var/lib/libvirt/images/raw.img      file  1.00 GiB      0.00 B

# qemu-img info qcow2.img 
image: qcow2.img
file format: raw  <===format is raw 
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G

4.operate uploaded volume. do vol-clone successfully.
# virsh vol-clone qcow2.img qcow2.img.clone --pool default
Vol qcow2.img.clone cloned from qcow2.img

# virsh vol-dumpxml qcow2.img.clone default
<volume type='file'>
  <name>qcow2.img.clone</name>
  <key>/var/lib/libvirt/images/qcow2.img.clone</key>
  <source>
  </source>
  <capacity unit='bytes'>1073741824</capacity>
  <allocation unit='bytes'>1073741824</allocation>
  <target>
    <path>/var/lib/libvirt/images/qcow2.img.clone</path>
    <format type='raw'/>  
......
</volume>

additional info: 
Try to upload a domain image file . Guest can start and can do some operations in guest after upload successfully. 
Try to refresh pool during vol-upload , upload successfully . 

Volume target format type was changed after uploading a different format file . and the volume works well .
Move to verified.

Comment 10 errata-xmlrpc 2015-03-05 07:31:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html