Bug 890291 - RFE: qemu:///session mode for root
Summary: RFE: qemu:///session mode for root
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: ---
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 8.0
Assignee: Virtualization Maintenance
QA Contact: yafu
URL:
Whiteboard:
: 1657561 (view as bug list)
Depends On: 1045069
Blocks: TRACKER-bugs-affecting-libguestfs 1375157
TreeView+ depends on / blocked
 
Reported: 2012-12-26 08:27 UTC by Mohua Li
Modified: 2020-03-18 10:21 UTC (History)
21 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-18 10:21:56 UTC
Type: Feature Request
Target Upstream Version:


Attachments (Terms of Use)

Description Mohua Li 2012-12-26 08:27:59 UTC
Description of problem:

guestfish fail to start at /root directory, as the sVIRT context is incorrect, 

52bf
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: command: run: qemu-img
libguestfs: command: run: \ create
libguestfs: command: run: \ -f qcow2
libguestfs: command: run: \ -b /var/tmp/.guestfs-0/root.22707
libguestfs: command: run: \ -o backing_fmt=raw
libguestfs: command: run: \ /tmp/libguestfs41L0JX/snapshot1
Formatting '/tmp/libguestfs41L0JX/snapshot1', fmt=qcow2 size=4294967296 backing_file='/var/tmp/.guestfs-
0/root.22707' backing_fmt='raw' encryption=off cluster_size=65536 lazy_refcounts=off 
libguestfs: [00081ms] create libvirt XML
libguestfs: trace: disk_format "/root/test1.img"
libguestfs: command: run: qemu-img
libguestfs: command: run: \ info
libguestfs: command: run: \ /tmp/libguestfs41L0JX/format.2
libguestfs: trace: disk_format = "raw"
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schem
as/domain/qemu/1.0">\n  <name>guestfs-1a1akg8wf84jb0wk</name>\n  <memory unit="MiB">500</memory>\n  <cur
rentMemory unit="MiB">500</currentMemory>\n  <vcpu>1</vcpu>\n  <clock offset="utc"/>\n  <os>\n    <type>
hvm</type>\n    <kernel>/var/tmp/.guestfs-0/kernel.22707</kernel>\n    <initrd>/var/tmp/.guestfs-0/initr
d.22707</initrd>\n    <cmdline>panic=1 console=ttyS0 udevtimeout=600 no_timer_check acpi=off printk.time
=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=screen-256color</cmdline>\n  </o
s>\n  <on_reboot>destroy</on_reboot>\n  <devices>\n    <controller type="scsi" index="0" model="virtio-s
csi"/>\n    <disk device="disk" type="file">\n      <source file="/root/test1.img"/>\n      <target dev=
"sda" bus="scsi"/>\n      <driver name="qemu" type="raw" cache="none"/>\n      <address type="drive" con
troller="0" bus="0" target="0" unit="0"/>\n    </disk>\n    <disk type="file" device="disk">\n      <sou
rce file="/tmp/libguestfs41L0JX/snapshot1"/>\n      <target dev="sdb" bus="scsi"/>\n      <driver name="
qemu" type="qcow2" cache="unsafe"/>\n      <address type="drive" controller="0" bus="0" target="1" unit=
"0"/>\n      <shareable/>\n    </disk>\n    <serial type="unix">\n      <source mode="connect" path="/tm
p/libguestfs41L0JX/console.sock"/>\n      <target port="0"/>\n    </serial>\n    <channel type="unix">\n
      <source mode="connect" path="/tmp/libguestfs41L0JX/guestfsd.sock"/>\n      <target type="virtio" n
ame="org.libguestfs.channel.0"/>\n    </channel>\n  </devices>\n  <qemu:commandline>\n    <qemu:env name
="TMPDIR" value="/var/tmp"/>\n  </qemu:commandline>\n</domain>\n
libguestfs: [00088ms] launch libvirt guest
libguestfs: error: could not create appliance through libvirt: internal error process exited while conne
cting to monitor: qemu-kvm: -drive file=/root/test1.img,if=none,id=drive-scsi0-0-0-0,format=raw,cache=no
ne: could not open disk image /root/test1.img: Permission denied
 [code=1 domain=10]
libguestfs: trace: launch = -1 (error)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x24024a0 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfs41L0JX
[root@intel-i7-12-4 ~]# ll
total 24
-rw-------. 1 root root      2903 Dec 25 10:59 anaconda-ks.cfg
-rw-r--r--. 1 root root     13741 Dec 25 10:59 libguestfs-install.log
-rw-r--r--. 1 root root 104857600 Dec 26 13:48 test1.img
[root@intel-i7-12-4 ~]# ls -Z
-rw-------. root root system_u:object_r:admin_home_t:s0 anaconda-ks.cfg
-rw-r--r--. root root system_u:object_r:admin_home_t:s0 libguestfs-install.log
-rw-r--r--. root root system_u:object_r:admin_home_t:s0 test1.img
52bf
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: command: run: qemu-img
libguestfs: command: run: \ create
libguestfs: command: run: \ -f qcow2
libguestfs: command: run: \ -b /var/tmp/.guestfs-0/root.22707
libguestfs: command: run: \ -o backing_fmt=raw
libguestfs: command: run: \ /tmp/libguestfs41L0JX/snapshot1
Formatting '/tmp/libguestfs41L0JX/snapshot1', fmt=qcow2 size=4294967296 backing_file='/var/tmp/.guestfs-
0/root.22707' backing_fmt='raw' encryption=off cluster_size=65536 lazy_refcounts=off 
libguestfs: [00081ms] create libvirt XML
libguestfs: trace: disk_format "/root/test1.img"
libguestfs: command: run: qemu-img
libguestfs: command: run: \ info
libguestfs: command: run: \ /tmp/libguestfs41L0JX/format.2
libguestfs: trace: disk_format = "raw"
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schem
as/domain/qemu/1.0">\n  <name>guestfs-1a1akg8wf84jb0wk</name>\n  <memory unit="MiB">500</memory>\n  <cur
rentMemory unit="MiB">500</currentMemory>\n  <vcpu>1</vcpu>\n  <clock offset="utc"/>\n  <os>\n    <type>
hvm</type>\n    <kernel>/var/tmp/.guestfs-0/kernel.22707</kernel>\n    <initrd>/var/tmp/.guestfs-0/initr
d.22707</initrd>\n    <cmdline>panic=1 console=ttyS0 udevtimeout=600 no_timer_check acpi=off printk.time
=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=screen-256color</cmdline>\n  </o
s>\n  <on_reboot>destroy</on_reboot>\n  <devices>\n    <controller type="scsi" index="0" model="virtio-s
csi"/>\n    <disk device="disk" type="file">\n      <source file="/root/test1.img"/>\n      <target dev=
"sda" bus="scsi"/>\n      <driver name="qemu" type="raw" cache="none"/>\n      <address type="drive" con
troller="0" bus="0" target="0" unit="0"/>\n    </disk>\n    <disk type="file" device="disk">\n      <sou
rce file="/tmp/libguestfs41L0JX/snapshot1"/>\n      <target dev="sdb" bus="scsi"/>\n      <driver name="
qemu" type="qcow2" cache="unsafe"/>\n      <address type="drive" controller="0" bus="0" target="1" unit=
"0"/>\n      <shareable/>\n    </disk>\n    <serial type="unix">\n      <source mode="connect" path="/tm
p/libguestfs41L0JX/console.sock"/>\n      <target port="0"/>\n    </serial>\n    <channel type="unix">\n
      <source mode="connect" path="/tmp/libguestfs41L0JX/guestfsd.sock"/>\n      <target type="virtio" n
ame="org.libguestfs.channel.0"/>\n    </channel>\n  </devices>\n  <qemu:commandline>\n    <qemu:env name
="TMPDIR" value="/var/tmp"/>\n  </qemu:commandline>\n</domain>\n
libguestfs: [00088ms] launch libvirt guest
libguestfs: error: could not create appliance through libvirt: internal error process exited while conne
cting to monitor: qemu-kvm: -drive file=/root/test1.img,if=none,id=drive-scsi0-0-0-0,format=raw,cache=no
ne: could not open disk image /root/test1.img: Permission denied
 [code=1 domain=10]
libguestfs: trace: launch = -1 (error)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x24024a0 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfs41L0JX
[root@intel-i7-12-4 ~]# ll
total 24
-rw-------. 1 root root      2903 Dec 25 10:59 anaconda-ks.cfg
-rw-r--r--. 1 root root     13741 Dec 25 10:59 libguestfs-install.log
-rw-r--r--. 1 root root 104857600 Dec 26 13:48 test1.img
[root@intel-i7-12-4 ~]# ls -Z
-rw-r--r--. root root system_u:object_r:admin_home_t:s0 test1.img
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

the correct context should be,

-rw-r--r--. root root system_u:object_r:svirt_image_t:s0:c39,c571 test1.img
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^





Version-Release number of selected component (if applicable):
libguestfs-1.20.1-4.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. cd /root; guestfish -x -v -N fs 
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 Richard W.M. Jones 2013-01-22 13:03:08 UTC
With SELinux permissive, it still fails for me with exactly
the same error:

libguestfs: error: could not create appliance through libvirt: internal error process exited while connecting to monitor: qemu-kvm: -drive file=/root/test1.img,if=none,id=drive-scsi0-0-0-0,format=raw,cache=none: could not open disk image /root/test1.img: Permission denied
 [code=1 domain=10]

I suspect that because we're running as root, libvirt thinks it's
a good idea to run qemu-kvm as qemu.qemu, and that doesn't have
sufficient (ordinary Unix) permissions to read files in /root.  That
would be a libvirt bug.

I also get a raft of SELinux AVCs, but none of them seem to be
significant failures (audit2allow suggests that no new rules are
needed):

type=VIRT_MACHINE_ID msg=audit(1358859271.394:10528): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c vm-ctx=? img-ctx=? model=stack exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_MACHINE_ID msg=audit(1358859271.395:10529): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c vm-ctx=system_u:system_r:svirt_t:s0:c388,c611 img-ctx=system_u:object_r:svirt_image_t:s0:c388,c611 model=selinux exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_MACHINE_ID msg=audit(1358859271.395:10530): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c vm-ctx=107:107 img-ctx=107:107 model=dac exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.397:10531): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=deny vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=all exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.397:10532): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=major category=pty maj=88 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.398:10533): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/null rdev=01:03 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.398:10534): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/full rdev=01:07 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.398:10535): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/zero rdev=01:05 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.398:10536): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/random rdev=01:08 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.399:10537): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/urandom rdev=01:09 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.399:10538): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/ptmx rdev=05:02 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.399:10539): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/kvm rdev=? acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=failed'
type=VIRT_RESOURCE msg=audit(1358859271.399:10540): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/kqemu rdev=? acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=failed'
type=VIRT_RESOURCE msg=audit(1358859271.399:10541): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/rtc rdev=FE:00 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.400:10542): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=cgroup reason=allow vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c cgroup="/sys/fs/cgroup/devices/libvirt/qemu/guestfs-4c7bl97lnv3hsv30/" class=path path=/dev/hpet rdev=0A:E4 acl=rw exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.663:10543): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=disk reason=start vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c old-disk="?" new-disk="/root/test1.img" exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.663:10544): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=disk reason=start vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c old-disk="?" new-disk="/tmp/libguestfsAqrvni/snapshot1" exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.664:10545): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=mem reason=start vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c old-mem=0 new-mem=512000 exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_RESOURCE msg=audit(1358859271.664:10546): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu resrc=vcpu reason=start vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c old-vcpu=0 new-vcpu=1 exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=success'
type=VIRT_CONTROL msg=audit(1358859271.664:10547): pid=3822 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=qemu op=start reason=booted vm="guestfs-4c7bl97lnv3hsv30" uuid=24b1ad43-96ef-6f8e-745a-f6d804cdd92c vm-pid=-1 exe=2F7573722F7362696E2F6C69627669727464202864656C6574656429 hostname=? addr=? terminal=? res=failed'

Comment 3 Richard W.M. Jones 2013-01-22 13:06:20 UTC
'chown qemu.qemu /root' "fixes" this.

I'm reassigning this bug to libvirt.

Comment 4 Richard W.M. Jones 2013-01-22 13:09:10 UTC
When libguestfs uses libvirt, it specifies a session URI, ie.
qemu:///session.  When libguesfs is root, libvirt decides to run
the qemu-kvm process as qemu.qemu (not root.root).  As a result
qemu cannot open regular files in root-owned directories such
as /root.

Libvirt should run qemu as root.root (when a session URI is
used, not for a system URI obviously).

A simple reproducer is:

su -
cd /root
guestfish -x -v -N fs

Comment 5 Daniel Berrangé 2013-01-22 13:11:41 UTC
There is no such thing as "session" mode when running as root. Libvirt should have rejected the 'qemu:///session' URI if you were root. If libguestfs is running as root, then the qemu:///system URI should be used.

Comment 6 Richard W.M. Jones 2013-01-22 13:19:09 UTC
Let me make a small clarification on comment 4.

It turns out that libguestfs opens libvirt with the NULL URI
(ie. letting libvirt choose a default).  That's a bug in
libguestfs.

HOWEVER forcing the URI to qemu:///session does NOT work:

LIBGUESTFS_ATTACH_METHOD=libvirt:qemu:///session guestfish -x -v -N fs

It doesn't appear to start up a session libvirtd process either.

Comment 7 Richard W.M. Jones 2013-01-22 14:29:59 UTC
I tried adding:

<seclabel type='static' model='dac' relabel='yes'>
  <label>0:0</label>
</seclabel>

However it just fails a bit later on:

libguestfs: error: could not create appliance through libvirt: internal error process exited while connecting to monitor: bind(unix:/var/lib/libvirt/qemu/guestfs-tv37llwj5evejniq.monitor): Permission denied
chardev: opening backend "socket" failed
 [code=1 domain=10]

SELinux is Permissive, and there is no additional information in
the log file.

I've no idea why.  If qemu is running as root, then it has
access to /var/lib/libvirt/qemu.  Perhaps the mode of the
socket is wrong, eg. it's not writable or something.

Comment 8 Richard W.M. Jones 2013-01-22 14:46:50 UTC
Possible patch to libguestfs.  Does not work as discussed
in comment 7.

From c13ab51066ed4e1b13dc64bef16581146e957cdb Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Tue, 22 Jan 2013 14:45:28 +0000
Subject: [PATCH] launch: libvirt: Run qemu as root when libguestfs runs as
 root (RHBZ#890291).

---
 src/launch-libvirt.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/src/launch-libvirt.c b/src/launch-libvirt.c
index 1609b79..1da04b1 100644
--- a/src/launch-libvirt.c
+++ b/src/launch-libvirt.c
@@ -870,6 +870,27 @@ construct_libvirt_xml_seclabel (guestfs_h *g,
                                            BAD_CAST "none"));
     XMLERROR (-1, xmlTextWriterEndElement (xo));
   }
+  else {
+    if (params->is_root) {
+      /* If root, run the qemu subprocess as root too (not qemu.qemu).
+       * See: https://bugzilla.redhat.com/show_bug.cgi?id=890291#c4
+       */
+      XMLERROR (-1, xmlTextWriterStartElement (xo, BAD_CAST "seclabel"));
+      XMLERROR (-1,
+                xmlTextWriterWriteAttribute (xo, BAD_CAST "type",
+                                             BAD_CAST "static"));
+      XMLERROR (-1,
+                xmlTextWriterWriteAttribute (xo, BAD_CAST "model",
+                                             BAD_CAST "dac"));
+      XMLERROR (-1,
+                xmlTextWriterWriteAttribute (xo, BAD_CAST "relabel",
+                                             BAD_CAST "yes"));
+      XMLERROR (-1, xmlTextWriterStartElement (xo, BAD_CAST "label"));
+      XMLERROR (-1, xmlTextWriterWriteString (xo, BAD_CAST "0:0"));
+      XMLERROR (-1, xmlTextWriterEndElement (xo));
+      XMLERROR (-1, xmlTextWriterEndElement (xo));
+    }
+  }
 
   return 0;
 
-- 
1.8.1

Comment 9 Daniel Berrangé 2013-01-22 16:00:12 UTC
>   <label>0:0</label>

I think you'll want to use  'root:qemu' instead of this - the VM needs to be able to access the various dirs under /var/lib/libvirt/ which are owned by qemu:qemu.

Comment 10 Richard W.M. Jones 2013-01-22 16:28:52 UTC
(In reply to comment #9)
> >   <label>0:0</label>
> 
> I think you'll want to use  'root:qemu' instead of this - the VM needs to be
> able to access the various dirs under /var/lib/libvirt/ which are owned by
> qemu:qemu.

I can't use names in the ancient libvirt in Fedora 18 [when are
we going to get a more recent version?]

In any case, neither 0:107 nor 0:36 worked.  Both failed in the
same way as comment 7.

Comment 14 Chris Evich 2013-09-10 18:30:38 UTC
I'm hitting a very similar if not the same problem with virt-install on RHEL-7.0-20130909.n.0:

virt-install --hvm ... --import --disk path=/root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2,bus=virtio,size=10,format=qcow2 ...

which shows in libvirt log:

char device redirected to /dev/pts/3 (label charserial0)
qemu-kvm: -drive file=/root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none: could not open disk image /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2: Permission denied

I made /root mode 0555, and checked all the intervening directories, anything running as qemu.qemu should have access all the way down the tree.  Now,  there's actually a symlink in here

ls -la /root/autotest/client/tests/virt/shared/
...
lrwxrwxrwx.  1 root root   18 Sep  9 16:21 data -> /var/lib/virt_test

So I checked audit.log and it seems it's hitting a problem on the symlink itself:

type=AVC ... pid=12672 comm="qemu-kvm" ... scontext=system_u:system_r:svirt_t:s0:c67,c371 tcontext=unconfined_u:object_r:admin_home_t:s0 tclass=lnk_file

Then I stumbled upon what looks like libvirt CHANGING the file ownership, which doesn't seem appropriate in it's own right, but could ALSO be a factor in the problem if SELinux wasn't getting in the way:

# ls -laZ /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2
-rw-rw----. root root system_u:object_r:var_lib_t:s0   /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2

# chown qemu.qemu /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2

# ls -laZ /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2
-rw-rw----. qemu qemu system_u:object_r:var_lib_t:s0   /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2

# virt-install --hvm ... --import ... --disk path=/root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2,bus=virtio,size=10,format=qcow2 ...

Starting install...
ERROR    internal error: process exited while connecting to monitor: char device redirected to /dev/pts/2 (label charserial0)
qemu-kvm: -drive file=/root/...: could not open disk image /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2: Permission denied

same error, but the file ownership was changed:

# ls -laZ /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2
-rw-rw----. root root system_u:object_r:admin_home_t:s0 /root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2

So, maybe this would break w/o SELinux, if libvirt is changing the image to root.root THEN trying to execute kvm-qemu as qemu.qemu (which now can't read the file)??

Also, it seems completely wrong that it should be changing permissions like this at all.  What if this was an environment where permissions mattered, and libvirt just changed them AGAINST some local security policy?

Comment 15 Chris Evich 2013-09-10 18:48:01 UTC
BTW: It does seem to be SELinux getting in the way here:

# sestatus
...
Current mode:                   permissive

# ls -la /var/lib/virt_test/images/jeos-17-64.qcow2
-rw-rw----. 2 qemu qemu 941555712 Sep 10 14:10 /var/lib/virt_test/images/jeos-17-64.qcow2

# virt-install --hvm ... --import  --disk path=/root/autotest/client/tests/virt/shared/data/images/jeos-17-64.qcow2,bus=virtio,size=10,format=qcow2 ...
...
Domain creation completed.

I am fairly bothered however that it's changing ownership of the file under some conditions. That seems really wrong if I happened to be operating in a security-sensitive environment.

Comment 16 Eric Blake 2013-09-10 18:58:21 UTC
(In reply to Chris Evich from comment #15)

> I am fairly bothered however that it's changing ownership of the file under
> some conditions. That seems really wrong if I happened to be operating in a
> security-sensitive environment.

Anyone with privileges to create a domain is already assumed to have privileges to hose the system in other ways (the fact that you can assign an arbitrary file to a domain, and thereby make libvirt change permissions on that file, is not a security bug unless there is an escalation path where you could not change those permissions any other way except by virtue of libvirt).  If you don't want libvirt changing ownership, you can set dynamic_ownership to 0 in /etc/libvirt/qemu.conf (but you are then responsible for setting it correctly yourself).  Also, we are working towards using ACLs rather than chown, where possible, which will also help in this situation.  But all this talk about changing permissions being against a potential security policy is a red herring unless you can prove that there is a user with enough permission to exploit permission changes to form an escalation attack; and if so, open a new bug for that, so that this bug can stay focused on the original problem.

Comment 17 Chris Evich 2013-09-10 19:16:05 UTC
Oh it's configurable, my bad, and never-mind then, I missed that section in the file.  As long as it can be turned off to respect sensitive environments, that aspect shouldn't be a problem then.

I was initially worried it was causing the problem in this bug, by changing permissions to be inaccessible to qemu.  But I think I found that was just a symptom of the SELinux + symlink AVC denial.  So we're square, nothing to see here, move along, move along :)

Comment 18 Richard W.M. Jones 2013-09-10 20:26:18 UTC
(In reply to Eric Blake from comment #16)
> Also, we are working
> towards using ACLs rather than chown, where possible, which will also help
> in this situation.

Actually I'm hoping the real solution to this will be for libvirtd
to open the files and pass file descriptors down to qemu.  That
should be better than ACLs since it means any intervening directories
won't get in the way.  Is that not still the ultimate plan?

Comment 19 Eric Blake 2013-09-10 20:30:59 UTC
The fact that NFS 4.2 supports SELinux labels may have mitigated some of the immediate need for passing fds down, so fd passing is a bit further down the road; but yes, there is still the goal on the horizon of using fd passing to work around all sorts of permissions setups.

Comment 22 Eric Blake 2015-07-28 15:10:55 UTC
(In reply to Eric Blake from comment #19)
> The fact that NFS 4.2 supports SELinux labels may have mitigated some of the
> immediate need for passing fds down, so fd passing is a bit further down the
> road; but yes, there is still the goal on the horizon of using fd passing to
> work around all sorts of permissions setups.

fd passing is covered more by bug 731134

Comment 26 Richard W.M. Jones 2018-12-10 08:20:08 UTC
*** Bug 1657561 has been marked as a duplicate of this bug. ***

Comment 30 Daniel Berrangé 2018-12-14 15:24:49 UTC
(In reply to Richard W.M. Jones from comment #7)
> I tried adding:
> 
> <seclabel type='static' model='dac' relabel='yes'>
>   <label>0:0</label>
> </seclabel>
> 
> However it just fails a bit later on:
> 
> libguestfs: error: could not create appliance through libvirt: internal
> error process exited while connecting to monitor:
> bind(unix:/var/lib/libvirt/qemu/guestfs-tv37llwj5evejniq.monitor):
> Permission denied
> chardev: opening backend "socket" failed
>  [code=1 domain=10]

Since you first tested this, we changed the paths we store these per-VM resources under. Previously all VMs had their files (eg monitor socket) stored in the same directory, which caused some permissions problems when using different permissisons on individual VMs as you see here. Now we create a dedicated directory per VM and explicitly chown that directory to match the seclabel, so it should work more transparently. 

I tested my own install with <label>0:0</label> and the VM successfully started & co-existed with other VMs runnng as qemu:qemu.

So assuming that also works correctly when you apply your patch from comment #8 to libguestfs, the main remaining gotcha could be the issue of capabilities. Normally a root user would have CAP_DAC_OVERRIDE, but libvirt will have removed that from QEMU. That would prevent the root user reading a file owned by a non-root user, for example, unless it had world-read bit set. We could potentially extend the DAC driver to allow it to retain capabilities.

Comment 31 Daniel Berrangé 2018-12-14 15:38:26 UTC
(In reply to Daniel Berrange from comment #30)
> Since you first tested this, we changed the paths we store these per-VM
> resources under. Previously all VMs had their files (eg monitor socket)
> stored in the same directory, which caused some permissions problems when
> using different permissisons on individual VMs as you see here. Now we
> create a dedicated directory per VM and explicitly chown that directory to
> match the seclabel, so it should work more transparently. 

I believe we probably fixed this particular problem in v1.2.19 with

commit f1f68ca33433825ce0deed2d96f1990200bc6618
Author: Martin Kletzander <mkletzan@redhat.com>
Date:   Fri Aug 7 14:42:31 2015 +0200

    qemu: Fix access to auto-generated socket paths
    
    We are automatically generating some socket paths for domains, but all
    those paths end up in a directory that's the same for multiple domains.
    The problem is that multiple domains can each run with different
    seclabels (users, selinux contexts, etc.).  The idea here is to create a
    per-domain directory labelled in a way that each domain can access its
    own unix sockets.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1146886

Comment 32 Eric Blake 2019-10-09 18:51:45 UTC
Dan, is this still relevant after your work with split daemons?

Comment 33 Richard W.M. Jones 2019-10-09 19:30:04 UTC
I'm very much hoping that Dan's split daemon work will finally mean we
can close this bug, but I'd like someone to test it first to confirm.  It
may require a small change to libguestfs lib/launch-libvirt.c so it calls
a different API.

Comment 34 Daniel Berrangé 2019-10-10 09:25:10 UTC
The daemon split does not introduce a 'session mode'  for root, so no change in that respect.

For libguestfs though, I'm thinking that the new "embedded mode" may well be the better bet

There's a demo here:

  https://www.redhat.com/archives/libvir-list/2019-May/msg00467.html

and i'm working on getting something more production viable for that.

Comment 36 Jaroslav Suchanek 2020-03-18 10:21:56 UTC
I am deferring this bug as this should be solved and superseded by bug 1772943 as is also commented in comment 34.

Please reopen if you disagree and provide justification why this bug should
get enough priority. Most important would be information about impact on
customer or layered product. Please indicate requested target release.


Note You need to log in before you can comment on or make changes to this bug.