Bug 801486

Possible reproducer:

restart libvirtd while imagefactory is running

Try to restart imagefactory. It doesn't restart.

running with 'LIBGUESTFS_DEBUG=1 imagefactory --rest --debug --foreground' gives this:


2012-03-08 11:17:34,185 INFO root thread(MainThread) Message: Launching in foreground...
new guestfs handle 0x1541050
[00000ms] febootstrap-supermin-helper --verbose -f checksum '/usr/lib64/guestfs/supermin.d' x86_64
supermin helper [00000ms] whitelist = (not specified), host_cpu = x86_64, kernel = (null), initrd = (null), appliance = (null)
supermin helper [00000ms] inputs[0] = /usr/lib64/guestfs/supermin.d
checking modpath /lib/modules/2.6.32-220.el6.x86_64 is a directory
picked vmlinuz-2.6.32-220.el6.x86_64 because modpath /lib/modules/2.6.32-220.el6.x86_64 exists
supermin helper [00000ms] finished creating kernel
supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d
supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d/base.img
supermin helper [00001ms] visiting /usr/lib64/guestfs/supermin.d/daemon.img
supermin helper [00001ms] visiting /usr/lib64/guestfs/supermin.d/hostfiles
supermin helper [00086ms] finished creating appliance
[00092ms] begin testing qemu features
[00124ms] finished testing qemu features
[00125ms] /usr/libexec/qemu-kvm \
    -drive file=/dev/zero,if=virtio \
    -nodefconfig \
    -enable-kvm \
    -nodefaults \
    -nographic \
    -m 500 \
    -no-reboot \
    -device virtio-serial \
    -serial stdio \
    -chardev socket,path=/tmp/libguestfs4WZxBf/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -kernel /tmp/.guestfs-0/kernel.4047 \
    -initrd /tmp/.guestfs-0/initrd.4047 \
    -append 'panic=1 console=ttyS0 udevtimeout=300 noapic acpi=off printk.time=1 cgroup_disable=memory selinux=0 guestfs_verbose=1 TERM=xterm ' \
    -drive file=/tmp/.guestfs-0/root.4047,snapshot=on,if=virtio,cache=unsafe
accept_from_daemon: 0x1541050 g->state = 1
recv_from_daemon: 0x1541050 g->state = 1, size_rtn = 0x7fff0f3989fc, buf_rtn = 0x7fff0f3989c0
qemu: could not load kernel '/tmp/.guestfs-0/kernel.4047': Permission denied
child_cleanup: 0x1541050: child process died
Traceback (most recent call last):
  File "/usr/bin/imagefactory", line 221, in <module>
    sys.exit(Application().main())
  File "/usr/bin/imagefactory", line 152, in main
    self._init_guestfs()
  File "/usr/bin/imagefactory", line 217, in _init_guestfs
    g.launch()
  File "/usr/lib/python2.6/site-packages/guestfs.py", line 152, in launch
    return libguestfsmod.launch (self._o)
RuntimeError: unexpected end of file when reading from daemon.
See earlier debug messages.
Or you can run 'libguestfs-test-tool' and post the complete output into
a bug report or message to the libguestfs mailing list.
closing guestfs handle 0x1541050 (state 0)



[root@qeblade33 tmp]# ls -ldZ .guestfs-0/
drwx------. root root unconfined_u:object_r:tmp_t:s0   .guestfs-0/



[root@qeblade33 tmp]# ls -laZ .guestfs-0/
drwx------. root root unconfined_u:object_r:tmp_t:s0   .
drwxrwxrwt. root root system_u:object_r:tmp_t:s0       ..
-rwx------. root root unconfined_u:object_r:tmp_t:s0   checksum
-rw-r--r--. root root unconfined_u:object_r:tmp_t:s0   initrd
-rw-r--r--. root root unconfined_u:object_r:tmp_t:s0   initrd.4047
lrwxrwxrwx. root root unconfined_u:object_r:tmp_t:s0   kernel -> /boot/vmlinuz-2.6.32-220.el6.x86_64
lrwxrwxrwx. root root unconfined_u:object_r:tmp_t:s0   kernel.4047 -> /boot/vmlinuz-2.6.32-220.el6.x86_64
-rw-r--r--. root root unconfined_u:object_r:tmp_t:s0   root
-rw-r--r--. root root unconfined_u:object_r:tmp_t:s0   root.4047

This miniroot linking process has suddenly become problematic.  Bizarre.  

This directory and the links and files inside of it are created incidentally as part our use of libguestfs.  In theory, the directory should stick around as long as a factory process is running as root, and the links should stick around as long as the PID they are associated with is alive.

I need to punt this one over to Richard Jones for some feedback.

My best wild guess is that this is SELinux related.

Can you attach the audit log from this system?

Created attachment 568693 [details]
Audit log

Notice the kernel is unreadable.  Now the kernel is
the first thing that qemu tries to read, so it could
simply be that.  Or it could be that the kernel is
actually unreadable.  Could you check its permissions
(in /boot)?

Maybe it is an SELinux thing after all.  Here are the AVCs:

type=AVC msg=audit(1331223396.667:51131): avc:  denied  { read } for  pid=3990 comm="qemu-kvm" name="kernel.3980" dev=sdb1 ino=22216730 scontext=unconfined_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:tmp_t:s0 tclass=lnk_file

type=AVC msg=audit(1331223434.277:51134): avc:  denied  { read } for  pid=4039 comm="qemu-kvm" name="kernel.4029" dev=sdb1 ino=22216730 scontext=unconfined_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:tmp_t:s0 tclass=lnk_file

type=AVC msg=audit(1331223454.362:51135): avc:  denied  { read } for  pid=4060 comm="qemu-kvm" name="kernel.4047" dev=sdb1 ino=22216730 scontext=unconfined_u:unconfined_r:qemu_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:tmp_t:s0 tclass=lnk_file

The only selinux / libguestfs bugs I'm aware of are these
two, and neither looks like the problem you've got here:
https://bugzilla.redhat.com/show_bug.cgi?id=729365
https://bugzilla.redhat.com/show_bug.cgi?id=670258

There is one suggestion here you could try:
https://bugzilla.redhat.com/show_bug.cgi?id=670258#c3
# setsebool -P allow_unconfined_qemu_transition 0

moving to 1.1 to recheck then

Moved to on_qa for retest. I don't think we should consider for z.

As we have not seen this again and cannot reproduce I am going to close this.