Bug 865464 - libvirtd uses environment's TMPDIR unless you specify one using <qemu:env>
libvirtd uses environment's TMPDIR unless you specify one using <qemu:env>
Status: CLOSED DUPLICATE of bug 856619
Product: Virtualization Tools
Classification: Community
Component: libvirt (Show other bugs)
unspecified
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Libvirt Maintainers
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-11 10:00 EDT by Richard W.M. Jones
Modified: 2016-04-26 16:05 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-04-26 15:52:11 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Richard W.M. Jones 2012-10-11 10:00:27 EDT
Description of problem:

$ guestfish 

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> add-drive /dev/null
><fs> run
libguestfs: error: could not create appliance through libvirt: internal error process exited while connecting to monitor: 2012-10-11 13:56:56.893+0000: 12185: debug : virFileClose:72 : Closed fd 31
2012-10-11 13:56:56.893+0000: 12185: debug : virFileClose:72 : Closed fd 66
2012-10-11 13:56:56.906+0000: 12185: debug : virFileClose:72 : Closed fd 3
qemu-kvm: -drive file=/var/tmp/.guestfs-1000/root.11693,if=none,id=drive-scsi0-0-1-0,format=raw,cache=unsafe: could not open disk image /var/tmp/.guestfs-1000/root.11693: No such file or directory
 [code=1 domain=10]

However the file does exist:

$ ll /var/tmp/.guestfs-1000 -Z
-rwxr-xr-x. rjones rjones unconfined_u:object_r:user_tmp_t:s0 checksum
-rw-r--r--. rjones rjones system_u:object_r:virt_content_t:s0 initrd
-rw-r--r--. rjones rjones system_u:object_r:virt_content_t:s0 initrd.11693
-rw-r--r--. rjones rjones system_u:object_r:virt_content_t:s0 kernel
-rw-r--r--. rjones rjones system_u:object_r:virt_content_t:s0 kernel.11693
-rw-r--r--. rjones rjones system_u:object_r:svirt_image_t:s0 root
-rw-r--r--. rjones rjones system_u:object_r:svirt_image_t:s0 root.11693

Version-Release number of selected component (if applicable):

libguestfs-1.19.48-1.fc18.x86_64
libvirt-0.10.2-3.fc19.x86_64

SELinux is enforcing.

How reproducible:

100%

Steps to Reproduce:
1. See above.
Comment 1 Richard W.M. Jones 2012-10-11 10:01:09 EDT
It works when you run the same (guestfish) command as root.
Comment 2 Richard W.M. Jones 2012-10-11 10:04:55 EDT
I don't think this is SELinux.  Why does it say "No such
file or directory" (vs. "Permission denied").  That doesn't
sound like an SELinux bug, but something to do with libguestfs
not creating the file before qemu starts.

Similar to bug 864871?
Comment 3 Richard W.M. Jones 2012-10-11 10:06:12 EDT
Also, setting SELinux to permissive does NOT make this
bug go away, so it does sound like it's nothing to do with
SELinux.
Comment 4 Richard W.M. Jones 2012-10-11 10:19:57 EDT
FWIW this was "fixed" by running:
killall libvirtd
(as non-root) to get rid of the per-user libvirtd.

Who knows ...  libvirt was not updated recently.
Comment 5 Richard W.M. Jones 2012-10-11 10:26:58 EDT
My theory is that the old libvirtd was probably started by some
test job, so it had TMPDIR set to a directory that no longer
exists.  When qemu tries to create snapshots (even though we are
setting <qemu:env>) for some reason it's still seeing TMPDIR set
to the non-existent directory.

Here is a reproducer:

killall libvirtd
mkdir /tmp/foo
TMPDIR=/tmp/foo guestfish -a /dev/null run   # successful
rm -rf /tmp/foo
guestfish -a /dev/null run   # fails with error above

Reassigning to libvirt, because the per-user libvirtd should
not be sensitive to $TMPDIR that happened to be used in one
caller.
Comment 6 Richard W.M. Jones 2012-10-11 10:28:10 EDT
libvirt-0.10.2-3.fc19.x86_64
Comment 7 Richard W.M. Jones 2012-10-11 10:47:34 EDT
OK, I get it.

We only set <qemu:env> TMPDIR *if* TMPDIR is actually set in
the libguestfs environment.  So in fact we were not setting
it in this case.

In any case, it's still a bug in libvirt that we need to set
TMPDIR at all.  It should ignore TMPDIR from the caller's
environment.
Comment 8 Richard W.M. Jones 2012-10-11 10:53:21 EDT
The libguestfs bug is fixed in
commit 67e9572286f98b830bef64cfcae5a072bf3a1882.

However this is still a libvirt bug.  All non-root callers must specify
<qemu:env> TMPDIR, because if they don't they risk getting a
random TMPDIR from another process, and if that random directory
has been deleted then they won't be able to start any transient
qemu processes.
Comment 9 Daniel Berrange 2012-10-11 10:55:44 EDT
I don't think this is a libvirt bug really, although the interaction here is certainly undesirable. If the user has set a custom TMPDIR env for their login session, applications (including libvirtd) should be honouring that variable.

The problem arising here is that libguestfs is setting a customized TMPDIR of its own. We can't distinguish between a user specified TMPDIR that we should honour, vs a libguestfs transient TMPDIR that we should ignore.

The only solution I see here is to stop using auto-start of libvirtd from libguestfs, and have libguestfs launch libvirtd explicitly (if not already running of course) with TMPDIR cleared (or the user's TMPDIR intact).

This can be done using a URI parameter qemu:///session?autostart=0 or by setting LIBVIRT_AUTOSTART=0 env variable. We probably want the env var, so that this works when the user provides a custom URI. It could then try to open the connection and if this fails, then spawn libvirtd & retry.

Alternatively could libguestfs avoid setting TMPDIR at all, and simply pass its desired directory as a parameter to mkstemp() and similar calls, and set TMPDIR only for child apps it might spawn ?
Comment 10 Richard W.M. Jones 2012-10-11 11:00:35 EDT
I think using autostart=0 is a good idea.  I didn't know that
option existed.

However I still think this is a libvirt bug.  The problem is
that environment variables from one process leak into another
process.  It's unexpected that setting TMPDIR in one process
should affect all future processes where TMPDIR may not be
set at all.
Comment 11 Cole Robinson 2016-04-26 15:47:24 EDT
Dan says:

(In reply to Daniel Berrange from comment #9)
> I don't think this is a libvirt bug really, although the interaction here is
> certainly undesirable. If the user has set a custom TMPDIR env for their
> login session, applications (including libvirtd) should be honouring that
> variable.
> 

Rich says:
> However I still think this is a libvirt bug.  The problem is
> that environment variables from one process leak into another
> process.  It's unexpected that setting TMPDIR in one process
> should affect all future processes where TMPDIR may not be
> set at all.



It sounds like the only way libvirt could 'fix' this is to completely ignore the environments TMPDIR, which sounds unexpected to me. I don't really see this as a libvirt bug.

Rich, is this still relevant?
Comment 12 Cole Robinson 2016-04-26 15:52:11 EDT
Actually this seems to just be the same as bug 856619

*** This bug has been marked as a duplicate of bug 856619 ***
Comment 13 Richard W.M. Jones 2016-04-26 16:05:58 EDT
Yes this is still a bug and yes I agree it's a particular
case of the more general design flaw / bug 856619.

Note You need to log in before you can comment on or make changes to this bug.