Bug 921292

Summary: qemu: could not open disk image /tmp/.../snapshot1: Permission denied
Product: [Community] Virtualization Tools Reporter: Richard W.M. Jones <rjones>
Component: libguestfsAssignee: Richard W.M. Jones <rjones>
Status: CLOSED UPSTREAM QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: dyasny, imcleod, mbooth, mzatko
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1101528 (view as bug list) Environment:
Last Closed: 2013-03-18 18:02:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1101528    

Description Richard W.M. Jones 2013-03-13 22:22:51 UTC
Description of problem:

qemu cannot open a file with permissions:
libguestfs: -rw-r--r--. root root unconfined_u:object_r:user_tmp_t:s0 snapshot1

See full log at end.

(Reported by Maros Zatko)

Version-Release number of selected component (if applicable):

libguestfs 1.20.4

How reproducible:

100%

Steps to Reproduce:
1. The reporter was running ImageFactory as root.
  
Additional info:

libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/tmp"
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -Z /tmp/.guestfs-0
libguestfs: drwx------. root root system_u:object_r:initrc_tmp_t:s0 .
libguestfs: drwxrwxrwt. root root system_u:object_r:tmp_t:s0       ..
libguestfs: -rwx------. root root system_u:object_r:initrc_tmp_t:s0 checksum
libguestfs: -rw-r--r--. root root system_u:object_r:virt_content_t:s0 initrd
libguestfs: -rw-r--r--. root root system_u:object_r:virt_content_t:s0 initrd.30672
libguestfs: -rw-r--r--. root root system_u:object_r:virt_content_t:s0 kernel
libguestfs: -rw-r--r--. root root system_u:object_r:virt_content_t:s0 kernel.30672
libguestfs: -rw-r--r--. qemu qemu system_u:object_r:virt_content_t:s0 root
libguestfs: -rw-r--r--. qemu qemu system_u:object_r:virt_content_t:s0 root.30672
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -Z /tmp/libguestfsVjjUey
libguestfs: drwxr-xr-x. root root unconfined_u:object_r:user_tmp_t:s0 .
libguestfs: drwxrwxrwt. root root system_u:object_r:tmp_t:s0       ..
libguestfs: srwxrwxr-x. root qemu unconfined_u:object_r:user_tmp_t:s0 console.sock
libguestfs: srwxrwxr-x. root qemu unconfined_u:object_r:user_tmp_t:s0 guestfsd.sock
libguestfs: -rw-r--r--. root root unconfined_u:object_r:user_tmp_t:s0 snapshot1
libguestfs: [03916ms] launch libvirt guest
libguestfs: trace: launch = -1 (error)
2013-03-13 22:39:03,433 ERROR imgfac.Builder.Builder thread(5e1d0771) Message: Exception encountered in _build_image_from_template thread
2013-03-13 22:39:03,433 ERROR imgfac.Builder.Builder thread(5e1d0771) Message: could not create appliance through libvirt: internal error process exited while connecting to monitor: qemu-kvm: -drive file=/tmp/libguestfsVjjUey/snapshot1,if=none,id=drive-scsi0-0-1-0,format=qcow2,cache=unsafe: could not open disk image /tmp/libguestfsVjjUey/snapshot1: Permission denied
 [code=1 domain=10]
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/imgfac/Builder.py", line 135, in _build_image_from_template
    self.os_plugin.create_base_image(self, template, parameters)
  File "/usr/lib/python2.7/site-packages/imagefactory_plugins/FedoraOS/FedoraOS.py", line 298, in create_base_image
    self.output_descriptor = self.guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.7/site-packages/oz/RedHat.py", line 1174, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, "yes")
  File "/usr/lib/python2.7/site-packages/oz/RedHat.py", line 1139, in _internal_customize
    self._collect_setup(modified_xml)
  File "/usr/lib/python2.7/site-packages/oz/RedHat.py", line 425, in _collect_setup
    g_handle = self._guestfs_handle_setup(libvirt_xml)
  File "/usr/lib/python2.7/site-packages/oz/Guest.py", line 930, in _guestfs_handle_setup
    g.launch()
  File "/bin/imagefactory", line 53, in launch
    _GuestFS.launch(self)
  File "/usr/lib/python2.7/site-packages/guestfs.py", line 272, in launch
    return libguestfsmod.launch (self._o)
RuntimeError: could not create appliance through libvirt: internal error process exited while connecting to monitor: qemu-kvm: -drive file=/tmp/libguestfsVjjUey/snapshot1,if=none,id=drive-scsi0-0-1-0,format=qcow2,cache=unsafe: could not open disk image /tmp/libguestfsVjjUey/snapshot1: Permission denied
 [code=1 domain=10]
libguestfs: trace: close
libguestfs: closing guestfs handle 0x7f3278a34cf0 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsVjjUey

============ Final Image Details ============
UUID: 7c386243-33b0-4d1a-808d-a7b72c199fc8
Type: base_image
Status: FAILED
Status Details: {'error': 'could not create appliance through libvirt: internal error process exited while connecting to monitor: qemu-kvm: -drive file=/tmp/libguestfsVjjUey/snapshot1,if=none,id=drive-scsi0-0-1-0,format=qcow2,cache=unsafe: could not open disk image /tmp/libguestfsVjjUey/snapshot1: Permission denied\n [code=1 domain=10]', 'activity': 'Base Image build failed with exception.'}

Comment 1 Richard W.M. Jones 2013-03-13 22:25:13 UTC
To the original reporter:

- What version of libvirt is installed?

- Do the following commands *when run as root* fail?

  truncate /tmp/test1.img -s1G
  guestfish -v -x --ro -a /tmp/test1.img run

Comment 2 Richard W.M. Jones 2013-03-13 23:03:42 UTC
I can't reproduce this so far.  I have:

libguestfs-1.20.4-1.fc18.x86_64
libvirt-daemon-0.10.2.3-1.fc18.x86_64
qemu-system-x86-1.2.2-6.fc18.x86_64

tmp-on-tmpfs enabled

Comment 3 Maros Zatko 2013-03-14 14:16:17 UTC
It seems that commands above don't fail
- http://fpaste.org/2283/

libguestfs-1.20.4-1.fc18.x86_64
python-libguestfs-1.20.4-1.fc18.x86_64
libguestfs-tools-c-1.20.4-1.fc18.x86_64
libvirt-daemon-0.10.2.3-1.fc18.x86_64
qemu-system-x86-1.2.2-6.fc18.x86_64

tmpfs on /tmp type tmpfs (rw,seclabel)

Comment 4 Richard W.M. Jones 2013-03-15 22:36:15 UTC
I worked out why this error was printed.

First thing to say, the error message from qemu is incredibly
misleading.  When it says "could not open disk image <name>"
it does NOT (necessarily) mean that it couldn't open the file
<name>.  It could also mean that it failed to open any backing
disks that <name> has (but it won't print the name of the backing
disk, it'll print the name of the "top" file).

This qemu annoyance gets me every time ...

Anyway, in this case snapshot1 is backed by /tmp/.guestfs-0/root,
and it is this backing file ("root") which is causing the problem.

Recall that qemu is running as user 'qemu.qemu'.  /tmp/.guestfs-0
has the following permissions:

> libguestfs: command: run: ls
> libguestfs: command: run: \ -a
> libguestfs: command: run: \ -l
> libguestfs: command: run: \ -Z /tmp/.guestfs-0
> libguestfs: drwx------. root root system_u:object_r:initrc_tmp_t:s0 .
              ^^^^^^^^^^^
[...]
> libguestfs: -rw-r--r--. qemu qemu system_u:object_r:virt_content_t:s0 root

Since the directory is not readable by non-root, qemu fails to
open the file and would give the error message shown in the
summary.

I checked the code and it turns out that we don't set the mode
on /tmp/.guestfs-$UID when we first create it.  We leave that up
to the umask, so if the first person who happens to run libguestfs
and create that directory has an especially restrictive umask,
it could end up with almost any permissions.

So this is a bug in libguestfs.

You should be able to reproduce this bug at will by doing:

  chmod 0700 /tmp/.guestfs-0

BTW I notice that .guestfs-0 is being created in /tmp in this
situation.  Normally I'd expect it to be created in /var/tmp,
but it's possible that OpenStack / ImageFactory is setting $TMPDIR,
or the reporter set this.

Comment 5 Richard W.M. Jones 2013-03-15 22:41:31 UTC
I (will shortly) push an upstream fix for this:
commit af1c53d104180415a8584c48f19fd4ea7df224f5

Comment 6 Richard W.M. Jones 2013-03-18 18:02:31 UTC
Fixed in 1.21.21.