Bug 1519144

For iSCSI image, "qemu-img convert" doesn't report error when converting image from qcow2 to raw but specifying the same filename for src and destination.

Its result is like the following:
# qemu-img info  iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2
image: json:{"driver": "raw", "file": {"lun": "2", "portal": "10.16.67.19", "driver": "iscsi", "transport": "tcp", "target": "iqn.2017-04.com.yilzhang:t1"}}
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: unavailable

# qemu-img create -f qcow2  iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2 1G
Formatting 'iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16

# qemu-img info  iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2
image: json:{"driver": "qcow2", "file": {"lun": "2", "portal": "10.16.67.19", "driver": "iscsi", "transport": "tcp", "target": "iqn.2017-04.com.yilzhang:t1"}}
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: unavailable
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

# qemu-img convert -p -f qcow2 -O raw   iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2   iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2
qemu-img: iSCSI Failure: SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_FIELD_IN_CDB(0x2400)
    (100.00/100%)
# echo $?
0

# qemu-img info  iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2
image: json:{"driver": "raw", "file": {"lun": "2", "portal": "10.16.67.19", "driver": "iscsi", "transport": "tcp", "target": "iqn.2017-04.com.yilzhang:t1"}}
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: unavailable

Hi,

How important is this exactly?  It would be very hard to properly implement, probably mostly as a byproduct of BZ 1513543.

The reason is that image/file creation infrastructure is detached from the normal I/O and image opening infrastructure in qemu, and therefore, we can tell when you try to open an image that another (or the same) qemu instance has opened already, but we cannot tell when you try to create an image file that's currently in use.

So there are two parts of this bug.

One is the following: You have some existing image, you want to convert it, oops, you specified the same image twice and now your target is overwritten (with nothing).  Sad, but there are many ways you can lose images if you aren't careful, so I wouldn't consider this really a bug.


The second can be phrased as follows, too:

$ qemu-img create -f qcow2 foo.qcow2 1M
Formatting 'foo.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ qemu-io -c 'write -P 42 0 1M' foo.qcow2
wrote 1048576/1048576 bytes at offset 0
1 MiB, 1 ops; 0.0159 sec (62.641 MiB/sec and 62.6409 ops/sec)
$ qemu-system-x86_64 -blockdev \
    node-name=foo,driver=qcow2,file.driver=file,file.filename=foo.qcow2
[...]

[on another shell]
$ qemu-img create -f qcow2 foo.qcow2 1M
Formatting 'foo.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16
qemu-img: foo.qcow2: Failed to get "write" lock
Is another process using the image?
$ ./qemu-io -Ur -c 'read -P 42 0 1M' foo.qcow2
read failed: Input/output error

(because the image is a raw image of size 0 now)


So despite the locks, you can still overwrite an image file while it is in use by some other process, by means of creating a new image on top of it.

That is indeed an issue, but I don't consider it a huge one either.  If you really want to corrupt an image while it is in use, you can always do so using non-qemu tools.  And if you're concerned about accidentally overwriting existing images...  Well, the locks will not prevent that when you're the only writer (see part one).

Finally let me note that it's probably impossible to do the file lock check and file creation atomically.  I can't think of a way for there not to be a small window where somebody could open the file and we'd overwrite it anyway.


Summing up, I'll see whether I can find a simple (albeit a bit hacky) solution that works now -- but it will only work on local files -- and if I cannot, then maybe we can see to something after BZ 1513543.


(PS: I do not think we can ever do something for protocols which do not support file locking or for which qemu hasn't implemented it, i.e. currently anything but file-posix.  And comparing filenames does not seem like an option to me.)

Sent an upstream series: http://lists.nongnu.org/archive/html/qemu-block/2018-04/msg00440.html

Fix included in qemu-kvm-rhev-2.12.0-7.el7

Verified the bug like below, the result is expected. So set the bug as verified.

Tested packages:
qemu-kvm-rhev-2.12.0-7.el7
kernel-3.10.0-918.el7

Steps:
1. Check info of base.qcow2 file
# qemu-img info base.qcow2 
image: base.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 1.8G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
2. Boot a VM from base.qcow2 file
/usr/libexec/qemu-kvm \
        -name 'guest-rhel7.5' \
        -machine pc \
        -nodefaults \
        -vga qxl \
        -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x8 \
        -drive id=drive_image1,if=none,snapshot=off,werror=stop,rerror=stop,aio=threads,cache=unsafe,format=qcow2,file=base.qcow2 \
        -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \
        -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=unsafe,media=cdrom,file=$2 \
        -device ide-cd,id=cd1,drive=drive_cd1,bus=ide.0,unit=0 \
        -vnc :0 \
        -monitor stdio \
        -m 8192 \
        -smp 8 \
        -device virtio-net-pci,mac=9a:b5:b6:b1:b5:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pci.0,addr=0x9  \
        -netdev tap,id=idxgXAlm \
# pwd
/home/test/test_for_qcow2
3. Create one new image named base.qcow2 in director "/home/test/test_for_qcow2"
# qemu-img create -f qcow2 base.qcow2 10G
Formatting 'base.qcow2', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16
qemu-img: base.qcow2: Failed to get "write" lock
Is another process using the image?
4. Check info of base.qcow2
# qemu-img info base.qcow2 
qemu-img: Could not open 'base.qcow2': Failed to get shared "write" lock
Is another process using the image?
# qemu-img info base.qcow2 -U
image: base.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 1.8G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
5. Check the VM
It works well.

Packages tested:
qemu-kvm-rhev-2.12.0-7.el7
kernel-3.10.0-919.el7.x86_64

Test steps:
Scenario 1:
1. Create qcow2 image
# qemu-img create -f qcow2 test.qcow2 1G
# qemu-img info test.qcow2 
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

2. Convert the image to original image
# qemu-img convert -f qcow2 -O raw test.qcow2 test.qcow2 
qemu-img: test.qcow2: error while converting raw: Failed to get "write" lock
Is another process using the image?
# qemu-img info test.qcow2 
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Scenario 2:
Run iotest 153
# rpm -ivhf qemu-kvm-rhev-2.12.0-7.el7.src.rpm
# rpmbuild -bp /root/rpmbuild/SPECS/qemu-kvm.spec --nodeps
# cd /root/rpmbuild/BUILD/qemu-2.12.0/
# ./configure
# export QEMU_PROG=/usr/libexec/qemu-kvm
# export QEMU_IMG_PROG=/usr/bin/qemu-img
# export QEMU_IO_PROG=/usr/bin/qemu-io
# export QEMU_NBD_PROG=/usr/bin/qemu-nbd
# cd tests/qemu-iotests
# ./check -qcow2 153
QEMU          -- "/usr/libexec/qemu-kvm" -nodefaults -machine accel=qtest
QEMU_IMG      -- "/usr/bin/qemu-img" 
QEMU_IO       -- "/usr/bin/qemu-io"  --cache writeback -f qcow2
QEMU_NBD      -- "/usr/bin/qemu-nbd" 
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- Linux/x86_64 hp-dl385g7-09 3.10.0-919.el7.x86_64
TEST_DIR      -- /root/rpmbuild/BUILD/qemu-2.12.0/tests/qemu-iotests/scratch
SOCKET_SCM_HELPER -- 

153        
Passed all 1 tests

Reproduced the issue with qemu-kvm-rhev-2.10.0-21.el7
1. Create qcow2 image
# qemu-img create -f qcow2 test.qcow2 1G
# qemu-img info test.qcow2 
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

2. Convert the image to original image
# qemu-img convert -f qcow2 -O raw test.qcow2 test.qcow2 
qemu-img: Could not open 'test.qcow2': Failed to get "write" lock
Is another process using the image?
# qemu-img info test.qcow2 
image: test.qcow2
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: 0

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3443