Bug 1664318

Summary: Upgrade libguestfs package to 1.39.14
Product: [Community] Virtualization Tools Reporter: mathieu.tarral
Component: libguestfsAssignee: Richard W.M. Jones <rjones>
Status: MODIFIED --- QA Contact:
Severity: medium Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: ptoscano, tburke
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Fixed In Version: libguestfs-1.39.11-3.fc29 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description mathieu.tarral 2019-01-08 12:43:08 UTC
Description of problem:
the libguestfs version available in the Fedora 29 repositories
has a bug which makes it impossible to start a QEMU image created by libguestfs.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. run libguestfs to create a read-only clone a QEMU image
2. start the image
3. QEMU fails -> Failed to get "write" lock

Actual results:
QEMU fails with the message:
Failed to get "write" lock

libvirt: QEMU Driver error : internal error: process exited while connecting to monitor: 2019-01-08T10:43:31.779093Z qemu-system-x86_64: -drive file=/tmp/osw-instances-1mw3fvuk/de81c7ff-b795-47f7-aa21-54d27c33fbbb/de81c7ff-b795-47f7-aa21-54d27c33fbbb,format=qcow2,if=none,id=drive-virtio-disk0: Failed to get "write" lock
Is another process using the image?

Expected results:
The VM should have started normally.

Additional info:
I found this bug report on launchpad:

stating that the bug should be fixed in libguestfs 1.40, with a link to the commit on Github:

Please tell me if it can be done, or what is blocking you ATM.


Comment 1 Richard W.M. Jones 2019-01-08 13:42:20 UTC
You'll probably find a few more of these because qemu broke a load of stuff
recently.  1.40 should be released soon but in the meantime I pushed a build
with this patch.

Comment 2 Richard W.M. Jones 2019-01-08 15:39:43 UTC
Fix is here, I don't know why Bodhi didn't update this bug though.

Comment 3 mathieu.tarral 2019-01-08 16:13:31 UTC
I downloaded your source package, but for some reason, DNF won't install it:

$ sudo dnf install libguestfs-1.39.11-3.fc29.src.rpm
Dernière vérification de l’expiration des métadonnées effectuée il y a 0:29:47 le mar. 08 janv. 2019 16:41:59 CET.
Erreur : Un paquet source rpm ne sera pas installé (libguestfs-1:1.39.11-3.fc29.src).

Error: a source package will not be installed

Did i do something wrong ?

Also I tried to login on Bodhi to drop a comment, but the login page give me a invalid ID error.

Thanks !

Comment 4 mathieu.tarral 2019-01-08 16:20:32 UTC
Ok, DNF is not supposed to deal with source package.

I'm not familiar with updating a package outside of the traditional "sudo dnf upgrade"
How should I do this ? download all the binary packages one by one ?
or wait for the package to be in the repositories

I'd like to test your update and give you a confirmation that it works.

Comment 5 Richard W.M. Jones 2019-01-08 17:26:18 UTC
Go to the build on the Builds tab:


Run a command such as:

  koji download-build libguestfs-1.39.11-3.fc29

and dnf install the packages you need.

Comment 6 mathieu.tarral 2019-01-09 09:59:14 UTC
It turns out that the patch didn't solve my problem.
But I think i need to explain how my project works for you to better understand
what's going on.

I'm building a sort of VM inspector.
When i start inspecting a VM, a new VM (let's call it VM_COPY) will be created with a hard disk using the inspected VM disk as baking file (to avoid modifying it)
Then i start a libguestfs instance (which creates a temporary VM too, with its own hard disks and baking files from the inspected VM)

  -> /home/wenzel/kvm/ubuntu16.04.qcow2
  -> /tmp/osw-instances/VM_COPY/{uuid}.qcow2
      baking file: /home/wenzel/kvm/ubuntu16.04.qcow2
  -> /tmp/libguestfsxxxxx/overlay1.qcow2
     baking file: /tmp/osw-instances/VM_COPY/{uuid}.qcow2
  -> /tmp/libguestfsxxxxx/overlay2.qcow2
     baking file: /var/tmp/.guestfs-1000/appliance.d/root

And when i'm done with libguestfs (without calling g.shutdown()), the next step of my inspection is to start the VM_COPY domain.
And it's at this point that I have the message about the lock from QEMU.

As long as libguestfs is running, even if I specified read-only, i cannot start my VM_COPY.

Maybe it can help you figure out the problem ?

It's the first time I have this issue.
I always used my project this way and It was working before.

When did QEMU introduced this "locked" feature ?

Thanks Richard !

Comment 7 Richard W.M. Jones 2019-01-09 10:18:05 UTC
> And when i'm done with libguestfs (without calling g.shutdown()), the next
> step of my inspection is to start the VM_COPY domain.
> And it's at this point that I have the message about the lock from QEMU.

This is really a question about the new qemu locking feature added in 2.10.
But why not close the libguestfs connection before starting the VM?

Also there is an "lslocks" command which will show you the processes which
are holding locks on a file.  That should be able to precisely answer your
question about what exactly is holding the lock.

Comment 8 mathieu.tarral 2019-01-09 12:30:04 UTC
> But why not close the libguestfs connection before starting the VM?

Because I have a plugin based architecture and some plugins might require this libguestfs instance
after the VM_COPY has been shutdown, at the end of the analysis.

But that's a detail.

I look at `lsocks` output while having libguestfs running, and the only I can see is 3 locks related to libvirt:
53:virtlogd        26648  POSIX    5B WRITE 0          0          0 /run/user/1000/libvirt/virtlogd.pid
59:libvirtd         9212  POSIX    4B WRITE 0          0          0 /run/user/1000/libvirt/libvirtd.pid
75:libvirtd         1259  POSIX    4B WRITE 0          0          0 /run/libvirtd.pid

The rest is not helpful because the path is incomplete and the process is (undefined)

I can stick with shutting down libguestfs before starting my VM_COPY, that's not a problem for now.
I was just curious how this could be fixed in the long term.

Comment 9 Richard W.M. Jones 2019-01-09 12:42:31 UTC
It's qemu (or qemu-img) which is holding locks.  Neither libguestfs nor libvirt should be
holding locks on VM images as far as I'm aware (definitely not libguestfs, not certain
about libvirt).