Bug 1417306

Summary: QEMU image file locking (libguestfs)
Product: Red Hat Enterprise Linux 7 Reporter: Ademar Reis <areis>
Component: libguestfsAssignee: Richard W.M. Jones <rjones>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 7.0CC: aliang, cegolf, chayang, coli, famz, hachen, hhan, jherrman, jsuchane, juzhang, leiwang, michen, mtessun, ngu, pingl, ptoscano, qzhang, rjones, virt-maint, wshi, xchen, xuwei, ycui, yoguo
Target Milestone: rcKeywords: FutureFeature
Target Release: 7.4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libguestfs-1.36.6-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Qcow2 image file locking When opening a .qcow2 image file in read-write mode using libguestfs utilities, the QEMU emulator now locks the image, which prevents other processes from modifying the image at the same time and possibly corrupting the image data.
Story Points: ---
Clone Of: 1378241 Environment:
Last Closed: 2018-04-10 09:15:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1378241, 1378242, 1432523, 1472272    
Bug Blocks: 1415250, 1415252, 1469590    

Description Ademar Reis 2017-01-27 22:02:07 UTC
+++ This bug was initially created as a clone of Bug #1378241 +++

QEMU image locking, which should prevent multiple runs of QEMU or qemu-img when a VM is running.

Upstream series (v7): https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg01306.html

Comment 2 juzhang 2017-02-03 01:34:11 UTC
Hi Lei,

Could you review this bz and mark qa_ack+?

Best Regards,
Junyi

Comment 6 Richard W.M. Jones 2017-09-14 17:04:55 UTC
This is upstream in quite a long set of commits:

410593e48b1b9c471a0fffcc7bf140d7424acb31
0eb1380ddc9621d678661e831fad502a2830891f
ea94f39e465faf9a83ba761e8a24a037b9ea9f86
a22eecbdb1057531195faace657510da4a569829
bf7d627305418284682cc6020189215a5015dbad
3d2b84231fca14b7e2beeeeab0e852f44dba0c21
9fe592808ccfd9ed184b88ca9c6cad2e1798dee3

Comment 7 Richard W.M. Jones 2017-09-14 17:05:35 UTC
I'm going to backport these to 1.36 so hopefully no extra
work will be required for RHEL 7.5.

Comment 9 YongkuiGuo 2017-10-13 07:48:40 UTC
Hi rjones,

I tried to verify this bug with packages of libguestfs-1.36.7-1.el7.x86_64 and qemu-xxx-rhev-2.10.0-1.el7.x86_64. After running 'guestfish -v -x -a RHEL-Server-7.3-64-hvm.raw' command, i cannot see 'locking=off' info which you added in patch according to comment 5. How should I verify this bug? thx

Comment 10 Richard W.M. Jones 2017-10-13 07:52:08 UTC
You should see it if you set the backend to direct and use --ro:

LIBGUESTFS_BACKEND=direct guestfish -v -x --ro -a rhel.raw -i

However for normal operations through libvirt, we are still waiting
on libvirt changes so that won't work.

Comment 11 YongkuiGuo 2017-10-13 09:28:27 UTC
(In reply to Richard W.M. Jones from comment #10)
> You should see it if you set the backend to direct and use --ro:
> 
> LIBGUESTFS_BACKEND=direct guestfish -v -x --ro -a rhel.raw -i
> 
> However for normal operations through libvirt, we are still waiting
> on libvirt changes so that won't work.

I set the backend to direct but omitted the '--ro' option. Now the guestfish result contains 'backing.file.locking=off' info. 

But I am a bit confused. I downgraded the libguestfs version to 1.36.3. The version of qemu is still 2.10.0. Again run 'LIBGUESTFS_BACKEND=direct guestfish -v -x --ro -a RHEL-Server-7.3-64-hvm.raw -i', then continued to execute '!qemu-img info /home/RHEL-Server-7.3-64-hvm.raw' successfully in guestfish shell. Is it normal that the qemu-img command still can read info from raw file? I saw a line 'Since qemu 2.10, qemu locks all drives even when they are opened readonly.' in your patch.

Comment 12 Richard W.M. Jones 2017-10-13 09:35:48 UTC
(In reply to YongkuiGuo from comment #11)
> But I am a bit confused. I downgraded the libguestfs version to 1.36.3. The
> version of qemu is still 2.10.0. Again run 'LIBGUESTFS_BACKEND=direct
> guestfish -v -x --ro -a RHEL-Server-7.3-64-hvm.raw -i', then continued to
> execute '!qemu-img info /home/RHEL-Server-7.3-64-hvm.raw' successfully in
> guestfish shell. Is it normal that the qemu-img command still can read info
> from raw file? I saw a line 'Since qemu 2.10, qemu locks all drives even
> when they are opened readonly.' in your patch.

It looks as if ‘qemu-img info’ does not acquire a lock if the
format is raw.  This makes sense because there's no reason to
acquire a lock simply to read the size of a raw file.

However what's a bit more surprising is that ‘qemu-img info’ also
does not acquire a lock if the format is qcow2.  I would have expected
that, so that could be a bug (in qemu-img).

Comment 13 Richard W.M. Jones 2017-10-13 09:45:10 UTC
OK I talked to Fam about this, and in fact this is expected behaviour.

When using --ro, libguestfs creates an overlay on top of the image
(image is the backing file).  In this case it is expected that
qemu-img info will still be able to work on the backing file.

Comment 14 YongkuiGuo 2017-10-16 09:46:23 UTC
Verified with packages:
libguestfs-1.36.7-1.el7.x86_64
qemu-kvm-rhev-2.10.0-1.el7.x86_64
qemu-img-rhev-2.10.0-1.el7.x86_64

Steps:

1. 
# LIBGUESTFS_BACKEND=direct guestfish -v -x --ro -a RHEL-Server-7.3-64-hvm.raw -i
--------------------------------------------------------------
...
[28259ms] /usr/libexec/qemu-kvm \
    -global virtio-blk-pci.scsi=off \
    -nodefconfig \
    -enable-fips \
    -nodefaults \
    -display none \
    -machine accel=kvm:tcg \
    -cpu host \
    -m 500 \
    -no-reboot \
    -rtc driftfix=slew \
    -no-hpet \
    -global kvm-pit.lost_tick_policy=discard \
    -kernel /var/tmp/.guestfs-0/appliance.d/kernel \
    -initrd /var/tmp/.guestfs-0/appliance.d/initrd \
    -object rng-random,filename=/dev/urandom,id=rng0 \
    -device virtio-rng-pci,rng=rng0 \
    -device virtio-scsi-pci,id=scsi \
    -drive file.file.filename=/tmp/libguestfsvtLE4Y/overlay1.qcow2,cache=unsafe,file.driver=qcow2,file.backing.file.locking=off,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \
...

--------------------------------------------------------------
The output contains 'locking=off'. So verified this bug.

Comment 20 errata-xmlrpc 2018-04-10 09:15:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0677