Red Hat Bugzilla – Bug 865464
libvirtd uses environment's TMPDIR unless you specify one using <qemu:env>
Last modified: 2016-04-26 16:05:58 EDT
Description of problem:
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.
Type: 'help' for help on commands
'man' to read the manual
'quit' to quit the shell
><fs> add-drive /dev/null
libguestfs: error: could not create appliance through libvirt: internal error process exited while connecting to monitor: 2012-10-11 13:56:56.893+0000: 12185: debug : virFileClose:72 : Closed fd 31
2012-10-11 13:56:56.893+0000: 12185: debug : virFileClose:72 : Closed fd 66
2012-10-11 13:56:56.906+0000: 12185: debug : virFileClose:72 : Closed fd 3
qemu-kvm: -drive file=/var/tmp/.guestfs-1000/root.11693,if=none,id=drive-scsi0-0-1-0,format=raw,cache=unsafe: could not open disk image /var/tmp/.guestfs-1000/root.11693: No such file or directory
However the file does exist:
$ ll /var/tmp/.guestfs-1000 -Z
-rwxr-xr-x. rjones rjones unconfined_u:object_r:user_tmp_t:s0 checksum
-rw-r--r--. rjones rjones system_u:object_r:virt_content_t:s0 initrd
-rw-r--r--. rjones rjones system_u:object_r:virt_content_t:s0 initrd.11693
-rw-r--r--. rjones rjones system_u:object_r:virt_content_t:s0 kernel
-rw-r--r--. rjones rjones system_u:object_r:virt_content_t:s0 kernel.11693
-rw-r--r--. rjones rjones system_u:object_r:svirt_image_t:s0 root
-rw-r--r--. rjones rjones system_u:object_r:svirt_image_t:s0 root.11693
Version-Release number of selected component (if applicable):
SELinux is enforcing.
Steps to Reproduce:
1. See above.
It works when you run the same (guestfish) command as root.
I don't think this is SELinux. Why does it say "No such
file or directory" (vs. "Permission denied"). That doesn't
sound like an SELinux bug, but something to do with libguestfs
not creating the file before qemu starts.
Similar to bug 864871?
Also, setting SELinux to permissive does NOT make this
bug go away, so it does sound like it's nothing to do with
FWIW this was "fixed" by running:
(as non-root) to get rid of the per-user libvirtd.
Who knows ... libvirt was not updated recently.
My theory is that the old libvirtd was probably started by some
test job, so it had TMPDIR set to a directory that no longer
exists. When qemu tries to create snapshots (even though we are
setting <qemu:env>) for some reason it's still seeing TMPDIR set
to the non-existent directory.
Here is a reproducer:
TMPDIR=/tmp/foo guestfish -a /dev/null run # successful
rm -rf /tmp/foo
guestfish -a /dev/null run # fails with error above
Reassigning to libvirt, because the per-user libvirtd should
not be sensitive to $TMPDIR that happened to be used in one
OK, I get it.
We only set <qemu:env> TMPDIR *if* TMPDIR is actually set in
the libguestfs environment. So in fact we were not setting
it in this case.
In any case, it's still a bug in libvirt that we need to set
TMPDIR at all. It should ignore TMPDIR from the caller's
The libguestfs bug is fixed in
However this is still a libvirt bug. All non-root callers must specify
<qemu:env> TMPDIR, because if they don't they risk getting a
random TMPDIR from another process, and if that random directory
has been deleted then they won't be able to start any transient
I don't think this is a libvirt bug really, although the interaction here is certainly undesirable. If the user has set a custom TMPDIR env for their login session, applications (including libvirtd) should be honouring that variable.
The problem arising here is that libguestfs is setting a customized TMPDIR of its own. We can't distinguish between a user specified TMPDIR that we should honour, vs a libguestfs transient TMPDIR that we should ignore.
The only solution I see here is to stop using auto-start of libvirtd from libguestfs, and have libguestfs launch libvirtd explicitly (if not already running of course) with TMPDIR cleared (or the user's TMPDIR intact).
This can be done using a URI parameter qemu:///session?autostart=0 or by setting LIBVIRT_AUTOSTART=0 env variable. We probably want the env var, so that this works when the user provides a custom URI. It could then try to open the connection and if this fails, then spawn libvirtd & retry.
Alternatively could libguestfs avoid setting TMPDIR at all, and simply pass its desired directory as a parameter to mkstemp() and similar calls, and set TMPDIR only for child apps it might spawn ?
I think using autostart=0 is a good idea. I didn't know that
However I still think this is a libvirt bug. The problem is
that environment variables from one process leak into another
process. It's unexpected that setting TMPDIR in one process
should affect all future processes where TMPDIR may not be
set at all.
(In reply to Daniel Berrange from comment #9)
> I don't think this is a libvirt bug really, although the interaction here is
> certainly undesirable. If the user has set a custom TMPDIR env for their
> login session, applications (including libvirtd) should be honouring that
> However I still think this is a libvirt bug. The problem is
> that environment variables from one process leak into another
> process. It's unexpected that setting TMPDIR in one process
> should affect all future processes where TMPDIR may not be
> set at all.
It sounds like the only way libvirt could 'fix' this is to completely ignore the environments TMPDIR, which sounds unexpected to me. I don't really see this as a libvirt bug.
Rich, is this still relevant?
Actually this seems to just be the same as bug 856619
*** This bug has been marked as a duplicate of bug 856619 ***
Yes this is still a bug and yes I agree it's a particular
case of the more general design flaw / bug 856619.