Bug 1519144 - qemu-img: image locking doesn't cover image creation
Summary: qemu-img: image locking doesn't cover image creation
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.5
Hardware: All
OS: Linux
low
low
Target Milestone: rc
: ---
Assignee: Max Reitz
QA Contact: Tingting Mao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-30 09:25 UTC by yilzhang
Modified: 2018-11-01 11:01 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-rhev-2.12.0-7.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-01 11:01:10 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description yilzhang 2017-11-30 09:25:45 UTC
Description of problem:
Convert image from qcow2 to raw, but specifying the same filename for src and destination, qemu-img will report error; But after that, "qemu-img info" displays wrong format

Version-Release number of selected component (if applicable):
Host kernel:   3.10.0-797.el7.ppc64le
qemu-kvm-rhev: qemu-kvm-rhev-2.10.0-8.el7

How reproducible: 100%

Steps to Reproduce:
1. Create one image with qcow2 format
# qemu-img create -f qcow2   imageA  1G

2. Convert this image from qcow2 to raw, but specifying the same filename for src and destination
#  qemu-img convert -f qcow2 -O raw  imageA   imageA
qemu-img: Could not open 'imageA': Failed to get "write" lock
Is another process using the image?

3. Get information of the image
#  qemu-img info imageA


Actual results:
image: imageA
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: 0


Expected results:
Step3, the image's format should not be changed, that is, should still be "qcow2"

Additional info:
PPC and x86 both have this issue
PPC:
Host kernel:   3.10.0-797.el7.ppc64le
qemu-kvm-rhev: qemu-kvm-rhev-2.10.0-8.el7

x86:
Host kernel:   3.10.0-799.el7.x86_64
qemu-kvm-rhev: qemu-kvm-rhev-2.10.0-9.el7

Comment 2 yilzhang 2017-11-30 09:42:55 UTC
For iSCSI image, "qemu-img convert" doesn't report error when converting image from qcow2 to raw but specifying the same filename for src and destination.

Its result is like the following:
# qemu-img info  iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2
image: json:{"driver": "raw", "file": {"lun": "2", "portal": "10.16.67.19", "driver": "iscsi", "transport": "tcp", "target": "iqn.2017-04.com.yilzhang:t1"}}
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: unavailable

# qemu-img create -f qcow2  iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2 1G
Formatting 'iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16

# qemu-img info  iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2
image: json:{"driver": "qcow2", "file": {"lun": "2", "portal": "10.16.67.19", "driver": "iscsi", "transport": "tcp", "target": "iqn.2017-04.com.yilzhang:t1"}}
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: unavailable
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

# qemu-img convert -p -f qcow2 -O raw   iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2   iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2
qemu-img: iSCSI Failure: SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_FIELD_IN_CDB(0x2400)
    (100.00/100%)
# echo $?
0

# qemu-img info  iscsi://10.16.67.19/iqn.2017-04.com.yilzhang:t1/2
image: json:{"driver": "raw", "file": {"lun": "2", "portal": "10.16.67.19", "driver": "iscsi", "transport": "tcp", "target": "iqn.2017-04.com.yilzhang:t1"}}
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: unavailable

Comment 3 Max Reitz 2017-12-01 15:00:53 UTC
Hi,

How important is this exactly?  It would be very hard to properly implement, probably mostly as a byproduct of BZ 1513543.

The reason is that image/file creation infrastructure is detached from the normal I/O and image opening infrastructure in qemu, and therefore, we can tell when you try to open an image that another (or the same) qemu instance has opened already, but we cannot tell when you try to create an image file that's currently in use.

So there are two parts of this bug.

One is the following: You have some existing image, you want to convert it, oops, you specified the same image twice and now your target is overwritten (with nothing).  Sad, but there are many ways you can lose images if you aren't careful, so I wouldn't consider this really a bug.


The second can be phrased as follows, too:

$ qemu-img create -f qcow2 foo.qcow2 1M
Formatting 'foo.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ qemu-io -c 'write -P 42 0 1M' foo.qcow2
wrote 1048576/1048576 bytes at offset 0
1 MiB, 1 ops; 0.0159 sec (62.641 MiB/sec and 62.6409 ops/sec)
$ qemu-system-x86_64 -blockdev \
    node-name=foo,driver=qcow2,file.driver=file,file.filename=foo.qcow2
[...]

[on another shell]
$ qemu-img create -f qcow2 foo.qcow2 1M
Formatting 'foo.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16
qemu-img: foo.qcow2: Failed to get "write" lock
Is another process using the image?
$ ./qemu-io -Ur -c 'read -P 42 0 1M' foo.qcow2
read failed: Input/output error

(because the image is a raw image of size 0 now)


So despite the locks, you can still overwrite an image file while it is in use by some other process, by means of creating a new image on top of it.

That is indeed an issue, but I don't consider it a huge one either.  If you really want to corrupt an image while it is in use, you can always do so using non-qemu tools.  And if you're concerned about accidentally overwriting existing images...  Well, the locks will not prevent that when you're the only writer (see part one).

Finally let me note that it's probably impossible to do the file lock check and file creation atomically.  I can't think of a way for there not to be a small window where somebody could open the file and we'd overwrite it anyway.


Summing up, I'll see whether I can find a simple (albeit a bit hacky) solution that works now -- but it will only work on local files -- and if I cannot, then maybe we can see to something after BZ 1513543.


(PS: I do not think we can ever do something for protocols which do not support file locking or for which qemu hasn't implemented it, i.e. currently anything but file-posix.  And comparing filenames does not seem like an option to me.)

Comment 4 Max Reitz 2018-04-20 22:37:32 UTC
Sent an upstream series: http://lists.nongnu.org/archive/html/qemu-block/2018-04/msg00440.html

Comment 6 Miroslav Rezanina 2018-07-04 08:14:48 UTC
Fix included in qemu-kvm-rhev-2.12.0-7.el7

Comment 8 Tingting Mao 2018-07-11 07:50:15 UTC
Verified the bug like below, the result is expected. So set the bug as verified.

Tested packages:
qemu-kvm-rhev-2.12.0-7.el7
kernel-3.10.0-918.el7

Steps:
1. Check info of base.qcow2 file
# qemu-img info base.qcow2 
image: base.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 1.8G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
2. Boot a VM from base.qcow2 file
/usr/libexec/qemu-kvm \
        -name 'guest-rhel7.5' \
        -machine pc \
        -nodefaults \
        -vga qxl \
        -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x8 \
        -drive id=drive_image1,if=none,snapshot=off,werror=stop,rerror=stop,aio=threads,cache=unsafe,format=qcow2,file=base.qcow2 \
        -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \
        -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=unsafe,media=cdrom,file=$2 \
        -device ide-cd,id=cd1,drive=drive_cd1,bus=ide.0,unit=0 \
        -vnc :0 \
        -monitor stdio \
        -m 8192 \
        -smp 8 \
        -device virtio-net-pci,mac=9a:b5:b6:b1:b5:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pci.0,addr=0x9  \
        -netdev tap,id=idxgXAlm \
# pwd
/home/test/test_for_qcow2
3. Create one new image named base.qcow2 in director "/home/test/test_for_qcow2"
# qemu-img create -f qcow2 base.qcow2 10G
Formatting 'base.qcow2', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16
qemu-img: base.qcow2: Failed to get "write" lock
Is another process using the image?
4. Check info of base.qcow2
# qemu-img info base.qcow2 
qemu-img: Could not open 'base.qcow2': Failed to get shared "write" lock
Is another process using the image?
# qemu-img info base.qcow2 -U
image: base.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 1.8G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
5. Check the VM
It works well.

Comment 9 Ping Li 2018-07-11 08:27:58 UTC
Packages tested:
qemu-kvm-rhev-2.12.0-7.el7
kernel-3.10.0-919.el7.x86_64

Test steps:
Scenario 1:
1. Create qcow2 image
# qemu-img create -f qcow2 test.qcow2 1G
# qemu-img info test.qcow2 
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

2. Convert the image to original image
# qemu-img convert -f qcow2 -O raw test.qcow2 test.qcow2 
qemu-img: test.qcow2: error while converting raw: Failed to get "write" lock
Is another process using the image?
# qemu-img info test.qcow2 
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Scenario 2:
Run iotest 153
# rpm -ivhf qemu-kvm-rhev-2.12.0-7.el7.src.rpm
# rpmbuild -bp /root/rpmbuild/SPECS/qemu-kvm.spec --nodeps
# cd /root/rpmbuild/BUILD/qemu-2.12.0/
# ./configure
# export QEMU_PROG=/usr/libexec/qemu-kvm
# export QEMU_IMG_PROG=/usr/bin/qemu-img
# export QEMU_IO_PROG=/usr/bin/qemu-io
# export QEMU_NBD_PROG=/usr/bin/qemu-nbd
# cd tests/qemu-iotests
# ./check -qcow2 153
QEMU          -- "/usr/libexec/qemu-kvm" -nodefaults -machine accel=qtest
QEMU_IMG      -- "/usr/bin/qemu-img" 
QEMU_IO       -- "/usr/bin/qemu-io"  --cache writeback -f qcow2
QEMU_NBD      -- "/usr/bin/qemu-nbd" 
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- Linux/x86_64 hp-dl385g7-09 3.10.0-919.el7.x86_64
TEST_DIR      -- /root/rpmbuild/BUILD/qemu-2.12.0/tests/qemu-iotests/scratch
SOCKET_SCM_HELPER -- 

153        
Passed all 1 tests

Reproduced the issue with qemu-kvm-rhev-2.10.0-21.el7
1. Create qcow2 image
# qemu-img create -f qcow2 test.qcow2 1G
# qemu-img info test.qcow2 
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

2. Convert the image to original image
# qemu-img convert -f qcow2 -O raw test.qcow2 test.qcow2 
qemu-img: Could not open 'test.qcow2': Failed to get "write" lock
Is another process using the image?
# qemu-img info test.qcow2 
image: test.qcow2
file format: raw
virtual size: 1.0G (1073741824 bytes)
disk size: 0

Comment 11 errata-xmlrpc 2018-11-01 11:01:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3443


Note You need to log in before you can comment on or make changes to this bug.