Bug 1858260 - SELinux prevents svirt_t read|write on lvm_control_t during VM creation
Summary: SELinux prevents svirt_t read|write on lvm_control_t during VM creation
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.2
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 8.3
Assignee: Michal Privoznik
QA Contact: yafu
URL:
Whiteboard:
: 1822522 1835909 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-17 11:31 UTC by Cédric Jeanneret
Modified: 2020-11-17 17:50 UTC (History)
19 users (show)

Fixed In Version: libvirt-6.6.0-1.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-11-17 17:50:17 UTC
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)
Generated XML for the instance (4.13 KB, text/plain)
2020-07-20 12:06 UTC, Cédric Jeanneret
no flags Details
strace log of startup (634.92 KB, text/plain)
2020-07-20 12:12 UTC, Daniel Berrangé
no flags Details
Libvirt debug log for an instance creation (14.35 KB, text/plain)
2020-07-20 12:40 UTC, Cédric Jeanneret
no flags Details

Description Cédric Jeanneret 2020-07-17 11:31:37 UTC
Description of problem:

On a freshly deployed osp-16.1, creating a VM on a compute node leads to denials in the audit.log:

type=AVC msg=audit(1594984849.260:10946): avc:  denied  { read write } for  pid=44259 comm="qemu-kvm" path="/dev/mapper/control" dev="devtmpfs" ino=15431 scontext=system_u:system_r:svirt_t:s0:c470,c554 tcontext=system_u:object_r:lvm_control_t:s0 tclass=chr_file permissive=0

This is non-blocking though, not really sure what svirt is trying to do in there.

Note the overcloud has been built using the standard OC images, meaning there isn't any LVM on the system

This also happens when we migrate a VM from a compute to another.


Version-Release number of selected component (if applicable):
openstack-selinux-0.8.20-0.20200428133425.3300746.el8ost.noarch
rhosp-director-images-x86_64-16.1-20200714.3.el8ost.noarch


How reproducible:
Always

Steps to Reproduce:
1. Deploy 16.1 with latest puddle/compose
2. Create a VM on a compute
3. Check the audit.log

Actual results:
We see a denial (but the VM is created)

Expected results:
We shouldn't see any denials

Additional info:
The following policy is generated using audit2allow, not sure if it's a good thing or not:
module svirt-lvm 1.0;

require {
        type svirt_t;
        type lvm_control_t;
        class chr_file { read write };
}

#============= svirt_t ==============
allow svirt_t lvm_control_t:chr_file { read write };

Comment 2 Kashyap Chamarthy 2020-07-17 12:12:18 UTC
A few things (but no real root cause yet):

- Given the AVC, DanPB says on IRC that "allowing write access to /dev/mapper/control sounds like a very bad idea"

- So we enabled libvirt debug filters that will also print out debugging for SELinux, but there's nothing in there.  
  I.e. this yields nothing:

    ()[root@oc0-compute1 libvirt]# egrep -i 'lvm|mapper' libvirtd.log
    ()[root@oc0-compute1 libvirt]#

- Nothing related to LVM or /dev/mapper/control in `journalctl` on the relevant Compute node

- But audit.log reports the same AVC:

---
type=AVC msg=audit(1594987242.026:11795): avc:  denied  { read write } for  pid=52726 comm="qemu-kvm" path="/dev/mapper/control" dev="devtmpfs" ino=1217 scontext=system_u:system_r:svirt_t:s0:c170,c695 tcontext=system_u:object_r:lvm_control_t:s0 tclass=chr_file permissive=0
---

Comment 3 Kashyap Chamarthy 2020-07-17 12:13:24 UTC
For the record, that's the full guest QEMU command-line, including libvirt and QEMU versions in use:

---------------------------------------------------------------------------------
2020-07-17 11:20:49.233+0000: starting up libvirt version: 6.0.0, package: 17.2.module+el8.2.0+6629+3fc0f2c2 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2020-05-14-11:03:02, ), qemu version: 4.2.0qemu-kvm-4.2.0-19.module+el8.2.0+6296+6b821950, kernel: 4.18.0-193.6.3.el8_2.x86_64, hostname: oc0-compute0.mydomain.tld
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-1-instance-00000001 \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-instance-00000001/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-instance-00000001/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-instance-00000001/.config \
QEMU_AUDIO_DRV=none \
/usr/libexec/qemu-kvm \
-name guest=instance-00000001,debug-threads=on \
-S \
-object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-instance-00000001/master-key.aes \
-machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,dump-guest-core=off \
-cpu EPYC-IBPB,x2apic=on,tsc-deadline=on,hypervisor=on,tsc-adjust=on,arch-capabilities=on,ssbd=on,cmp-legacy=on,perfctr-core=on,amd-ssbd=on,virt-ssbd=on,rdctl-no=on,skip-l1dfl-vmentry=on,mds-no=on,monitor=off,svm=off \
-m 512 \
-overcommit mem-lock=off \
-smp 1,sockets=1,dies=1,cores=1,threads=1 \
-uuid 8a234ced-b36c-461d-acaa-2658992b24e5 \
-smbios 'type=1,manufacturer=Red Hat,product=OpenStack Compute,version=20.3.1-0.20200626213433.38ee1f3.el8ost,serial=8a234ced-b36c-461d-acaa-2658992b24e5,uuid=8a234ced-b36c-461d-acaa-2658992b24e5,family=Virtual Machine' \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
-blockdev '{"driver":"file","filename":"/var/lib/nova/instances/_base/fbd4b81c3120e48c414d41e26399f983592f0f56","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"cache":{"direct":true,"no-flush":false},"driver":"raw","file":"libvirt-2-storage"}' \
-blockdev '{"driver":"file","filename":"/var/lib/nova/instances/8a234ced-b36c-461d-acaa-2658992b24e5/disk","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}' \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,id=virtio-disk0,bootindex=1,write-cache=on \
-netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 \
-device virtio-net-pci,rx_queue_size=512,host_mtu=1442,netdev=hostnet0,id=net0,mac=fa:16:3e:07:a6:89,bus=pci.0,addr=0x3 \
-add-fd set=3,fd=38 \
-chardev pty,id=charserial0,logfile=/dev/fdset/3,logappend=on \
-device isa-serial,chardev=charserial0,id=serial0 \
-device usb-tablet,id=input0,bus=usb.0,port=1 \
-vnc 172.16.13.102:0 \
-device cirrus-vga,id=video0,bus=pci.0,addr=0x2 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/0 (label charserial0)
2020-07-17T11:20:49.311581Z qemu-kvm: -device cirrus-vga,id=video0,bus=pci.0,addr=0x2: warning: 'cirrus-vga' is deprecated, please use a different VGA card instead
---------------------------------------------------------------------------------

Comment 4 Cédric Jeanneret 2020-07-17 12:27:29 UTC
Some more logs for the context:

type=VIRT_MACHINE_ID msg=audit(1594987241.930:11779): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 vm-ctx=system_u:system_r:svirt_t:s0:c170,c695 img-ctx=system_u:object_r:svirt_image_t:s0:c170,c695 model=selinux exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                                                               
type=VIRT_MACHINE_ID msg=audit(1594987241.930:11780): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 vm-ctx=+107:+107 img-ctx=+107:+107 model=dac exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                                                                                                                                   
type=ANOM_PROMISCUOUS msg=audit(1594987241.980:11781): dev=tap5fda37a3-f0 prom=256 old_prom=0 auid=4294967295 uid=993 gid=1000 ses=4294967295AUID="unset" UID="openvswitch" GID="hugetlbfs"                                                   
type=VIRT_RESOURCE msg=audit(1594987241.984:11782): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=net reason=open vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 net=fa:16:3e:30:64:70 path="/dev/net/tun" rdev=0A:C8 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                                                                                                       
type=VIRT_RESOURCE msg=audit(1594987241.999:11783): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=net reason=open vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 net=fa:16:3e:30:64:70 path="/dev/vhost-net" rdev=0A:EE exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                                                                                                     
type=VIRT_RESOURCE msg=audit(1594987242.018:11784): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=deny vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=all exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                                                                
type=VIRT_RESOURCE msg=audit(1594987242.018:11785): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=major category=pty maj=88 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                                  
type=VIRT_RESOURCE msg=audit(1594987242.018:11786): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/null" rdev=01:03 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                           
type=VIRT_RESOURCE msg=audit(1594987242.018:11787): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/full" rdev=01:07 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                           
type=VIRT_RESOURCE msg=audit(1594987242.018:11788): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/zero" rdev=01:05 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                           
type=VIRT_RESOURCE msg=audit(1594987242.018:11789): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/random" rdev=01:08 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                         
type=VIRT_RESOURCE msg=audit(1594987242.018:11790): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/urandom" rdev=01:09 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                        
type=VIRT_RESOURCE msg=audit(1594987242.018:11791): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/ptmx" rdev=05:02 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                           
type=VIRT_RESOURCE msg=audit(1594987242.018:11792): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/kvm" rdev=0A:E8 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                            
type=VIRT_RESOURCE msg=audit(1594987242.018:11793): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/rtc" rdev=FB:00 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                            
type=VIRT_RESOURCE msg=audit(1594987242.018:11794): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=cgroup reason=allow vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 cgroup="/sys/fs/cgroup/devices/machine/qemu-1-instance-00000002.libvirt-qemu/" class=path path="/dev/hpet" rdev=0A:E4 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"                           

type=AVC msg=audit(1594987242.026:11795): avc:  denied  { read write } for  pid=52726 comm="qemu-kvm" path="/dev/mapper/control" dev="devtmpfs" ino=1217 scontext=system_u:system_r:svirt_t:s0:c170,c695 tcontext=system_u:object_r:lvm_control_t:s0 tclass=chr_file permissive=0

type=USER_ACCT msg=audit(1594987242.063:11796): pid=52735 uid=42435 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='op=PAM:accounting grantors=pam_unix acct="neutron" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=success'UID="unknown(42435)" AUID="unset"
type=USER_CMD msg=audit(1594987242.063:11797): pid=52735 uid=42435 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='cwd="/" cmd=6E657574726F6E2D726F6F7477726170202F6574632F6E657574726F6E2F726F6F74777261702E636F6E66206970206E65746E732065786563206F766E6D6574612D36336438326433612D636262312D343534302D623964632D6530623863653433336235332073797363746C202D77206E65742E697076342E636F6E662E616C6C2E70726F6D6F74655F7365636F6E6461726965733D31 exe="/usr/bin/sudo" terminal=? res=success'UID="unknown(42435)" AUID="unset"
type=CRED_REFR msg=audit(1594987242.063:11798): pid=52735 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=USER_START msg=audit(1594987242.063:11799): pid=52735 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=USER_END msg=audit(1594987242.320:11800): pid=52735 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=CRED_DISP msg=audit(1594987242.320:11801): pid=52735 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=VIRT_RESOURCE msg=audit(1594987242.348:11802): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=disk reason=start vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 old-disk="?" new-disk="/var/lib/nova/instances/350dc719-a54b-4b58-a24b-462a20c17127/disk" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=VIRT_RESOURCE msg=audit(1594987242.348:11803): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=net reason=start vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 old-net="?" new-net="fa:16:3e:30:64:70" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=VIRT_RESOURCE msg=audit(1594987242.348:11804): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=chardev reason=start vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 old-chardev="?" new-chardev="/dev/pts/1" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=VIRT_RESOURCE msg=audit(1594987242.348:11805): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=mem reason=start vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 old-mem=0 new-mem=524288 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=VIRT_RESOURCE msg=audit(1594987242.348:11806): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm resrc=vcpu reason=start vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 old-vcpu=0 new-vcpu=1 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=VIRT_CONTROL msg=audit(1594987242.348:11807): pid=52329 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='virt=kvm op=start reason=booted vm="instance-00000002" uuid=350dc719-a54b-4b58-a24b-462a20c17127 vm-pid=52726 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

Comment 5 Cédric Jeanneret 2020-07-20 12:06:09 UTC
Created attachment 1701742 [details]
Generated XML for the instance

Comment 6 Daniel Berrangé 2020-07-20 12:10:32 UTC
strace of the QEMU startup shows this sequence:

openat(AT_FDCWD, "/proc/thread-self/attr/sockcreate", O_RDWR|O_CLOEXEC) = 3
write(3, "system_u:system_r:svirt_t:s0:c380,c858\0", 39) = 39
close(3)                                = 0
close(3)                                = 0
openat(AT_FDCWD, "/proc/thread-self/attr/sockcreate", O_RDWR|O_CLOEXEC) = 3
close(3)                                = 0
close(3)                                = 0
stat("/dev/mapper/control", {st_mode=S_IFCHR|0600, st_rdev=makedev(10, 236), ...}) = 0
openat(AT_FDCWD, "/dev/mapper/control", O_RDWR) = 3
close(4)                                = 0
stat("/dev/mapper/control", {st_mode=S_IFCHR|0600, st_rdev=makedev(10, 236), ...}) = 0
close(4)                                = 0
stat("/dev/mapper/control", {st_mode=S_IFCHR|0600, st_rdev=makedev(10, 236), ...}) = 0
close(4)                                = 0
openat(AT_FDCWD, "/proc/thread-self/attr/exec", O_RDWR|O_CLOEXEC) = 4
write(4, "system_u:system_r:svirt_t:s0:c380,c858\0", 39) = 39
close(4)                                = 0
close(4)                                = 0
close(4)                                = 0
close(40)                               = 0
close(41)                               = 0
execve("/usr/libexec/qemu-kvm", ["/usr/libexec/qemu-kvm", "-name", "guest=instance-00000003,debug-threads=on", "-S", "-object", "secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-instance-00000003/master-key.aes", "-machine", "pc-i440fx-rhel7.6.0,accel=kvm,usb=off,dump-guest-core=off", "-cpu", "EPYC-IBPB,x2apic=on,tsc-deadline=on,hypervisor=on,tsc-adjust=on,arch-capabilities=on,ssbd=on,cmp-legacy=on,perfctr-core=on,amd-ssbd=on,virt-ssbd=on,rdctl-no=on,skip-l1dfl-vmentry=on,mds-no=on,monitor=off,svm=off", "-m", "512", "-overcommit", "mem-lock=off", "-smp", "1,sockets=1,dies=1,cores=1,threads=1", "-uuid", "a97feed7-1d75-4193-86e6-b4c1e657e03a", "-smbios", "type=1,manufacturer=Red Hat,product=OpenStack Compute,version=20.3.1-0.20200626213433.38ee1f3.el8ost,serial=a97feed7-1d75-4193-86e6-b4c1e657e03a,uuid=a97feed7-1d75-4193-86e6-b4c1e657e03a,family=Virtual Machine", "-no-user-config", "-nodefaults", "-chardev", "socket,id=charmonitor,fd=34,server,nowait", "-mon", "chardev=charmonitor,id=monitor,mode=control", "-rtc", "base=utc,driftfix=slew", "-global", "kvm-pit.lost_tick_policy=delay", "-no-hpet", "-no-shutdown", "-boot", "strict=on", "-device", "piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2", "-blockdev", "{\"driver\":\"file\",\"filename\":\"/var/lib/nova/instances/_base/40b95d117af8543436a404bf09f6690aefaebf6b\",\"node-name\":\"libvirt-2-storage\",\"cache\":{\"direct\":true,\"no-flush\":false},\"auto-read-only\":true,\"discard\":\"unmap\"}", "-blockdev", "{\"node-name\":\"libvirt-2-format\",\"read-only\":true,\"cache\":{\"direct\":true,\"no-flush\":false},\"driver\":\"raw\",\"file\":\"libvirt-2-storage\"}", "-blockdev", "{\"driver\":\"file\",\"filename\":\"/var/lib/nova/instances/a97feed7-1d75-4193-86e6-b4c1e657e03a/disk\",\"node-name\":\"libvirt-1-storage\",\"cache\":{\"direct\":true,\"no-flush\":false},\"auto-read-only\":true,\"discard\":\"unmap\"}", "-blockdev", "{\"node-name\":\"libvirt-1-format\",\"read-only\":false,\"cache\":{\"direct\":true,\"no-flush\":false},\"driver\":\"qcow2\",\"file\":\"libvirt-1-storage\",\"backing\":\"libvirt-2-format\"}", "-device", "virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,id=virtio-disk0,bootindex=1,write-cache=on", "-netdev", "tap,fd=36,id=hostnet0,vhost=on,vhostfd=37", "-device", "virtio-net-pci,rx_queue_size=512,host_mtu=1442,netdev=hostnet0,id=net0,mac=fa:16:3e:7f:db:a3,bus=pci.0,addr=0x3", "-add-fd", "set=3,fd=39", "-chardev", "pty,id=charserial0,logfile=/dev/fdset/3,logappend=on", "-device", "isa-serial,chardev=charserial0,id=serial0", "-device", "usb-tablet,id=input0,bus=usb.0,port=1", "-vnc", "172.16.13.62:1", "-device", "cirrus-vga,id=video0,bus=pci.0,addr=0x2", "-device", "virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5", "-sandbox", "on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny", "-msg", "timestamp=on"], 0x7fb30c037e20 /* 7 vars */) = 0



We can see in this trace that libvirt is opening /dev/mapper/control, and then nothing ever closes FD=3. 

IOW, we're leaking FD=3 into the QEMU process, and then at time of execve  SELinux is reporting the AVC because we've left a read-write FD open and svirt, rightly, isn't allowing it.

I can't immediately see what code in libvirt will be opening this file though

Comment 7 Daniel Berrangé 2020-07-20 12:12:31 UTC
Created attachment 1701743 [details]
strace log of startup

Comment 8 Cédric Jeanneret 2020-07-20 12:40:43 UTC
Created attachment 1701749 [details]
Libvirt debug log for an instance creation

Comment 9 Daniel Berrangé 2020-07-20 12:53:24 UTC
Shortly before /dev/mapper/control is opened, strace logs show this:

unlink("/run/libvirt/qemu/2-instance-00000003.dev/hpet") = -1 ENOENT (No such file or directory)
mknod("/run/libvirt/qemu/2-instance-00000003.dev/hpet", S_IFCHR|0600, makedev(10, 228)) = 0
lchown("/run/libvirt/qemu/2-instance-00000003.dev/hpet", 0, 0) = 0
chmod("/run/libvirt/qemu/2-instance-00000003.dev/hpet", 020600) = 0


Shortly after /dev/mapper/control is opened, strace logs show this:

stat("/dev/hugepages", {st_mode=S_IFDIR|0775, st_size=0, ...}) = 0
stat("/run/libvirt/qemu/2-instance-00000003.hugepages", 0x7fb3191a9ea0) = -1 ENOENT (No such file or directory)
stat("/run/libvirt/qemu", {st_mode=S_IFDIR|0755, st_size=200, ...}) = 0
mkdir("/run/libvirt/qemu/2-instance-00000003.hugepages", 0777) = 0
mount("/dev/hugepages", "/run/libvirt/qemu/2-instance-00000003.hugepages", 0x7fb323fd6d63, MS_MOVE, NULL) = 0


We can match this up with the libvirt debug logs for the instance:

2020-07-20 12:32:27.189+0000: 90277: debug : virFileMakeParentPath:3056 : path=/run/libvirt/qemu/3-instance-00000005.dev/hpet
2020-07-20 12:32:27.189+0000: 90277: debug : virFileMakePathHelper:2993 : path=/run/libvirt/qemu/3-instance-00000005.dev mode=0777
2020-07-20 12:32:27.189+0000: 90277: debug : qemuDomainCreateDeviceRecursive:14642 : Creating dev /dev/hpet
2020-07-20 12:32:27.189+0000: 90277: debug : qemuDomainSetupAllDisks:14933 : Setting up disks
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllDisks:14942 : Setup all disks
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllHostdevs:14975 : Setting up hostdevs
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllHostdevs:14982 : Setup all hostdevs
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllMemories:15006 : Setting up memories
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllMemories:15013 : Setup all memories
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllChardevs:15043 : Setting up chardevs
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllChardevs:15051 : Setup all chardevs
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllGraphics:15107 : Setting up graphics
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllGraphics:15115 : Setup all graphics
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllInputs:15141 : Setting up inputs
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllInputs:15148 : Setup all inputs
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllRNGs:15180 : Setting up RNGs
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupAllRNGs:15188 : Setup all RNGs
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupLoader:15200 : Setting up loader
2020-07-20 12:32:27.190+0000: 90277: debug : qemuDomainSetupLoader:15224 : Setup loader
2020-07-20 12:32:27.190+0000: 90277: debug : virFileMakePathHelper:2993 : path=/run/libvirt/qemu/3-instance-00000005.hugepages mode=0777
2020-07-20 12:32:27.190+0000: 90277: debug : virFileMakePathHelper:2993 : path=/run/libvirt/qemu mode=0777

This points towards  qemuDomainSetupAllDisks as being a likely cause of opening /dev/mapper/control.

This method in turn calls qemuDomainSetupDisk, which calls virDevMapperGetTargets()

This in turn calls into dm_task_create in libdevmapper.so

In the LVM2 package source we see code in the source file device_mapper/ioctl/libdm-iface.c :

#ifdef DM_IOCTLS
static int _control_fd = -1;
static int _hold_control_fd_open = 0;

...snip...

static int _open_and_assign_control_fd(const char *control)
{
        if ((_control_fd = open(control, O_RDWR)) < 0) {
                log_sys_error("open", control);
                return 0;
        }

        return 1;
}
#endif

this matches our strace log.

I think there are multiple bugs here:

 - device-mapper should be setting O_CLOEXEC here to prevent its global FD being left open in child process. 

   This is a potentially very serious flaw as a FD for /dev/mapper/control gives significant privileges over the host and children should never get this

 - libvirt is not calling  dm_lib_release() so it doesn't clean up its global state. This would workaround the lack of use of O_CLOEXEC, but device-mapper should be fixed regardless. It simply isn't acceptable to open FDs without O_CLOEXEC in a multi-threaded application in the modern world.

 - libvirt should actually not be using  libdevice-mapper.so at all in this context. 

   We're running in between fork & exec, which means we're only permitted to use async signal safe functions. Libvirt uses malloc because we know our platforms make malloc safe, but anything else not declared async signal safe is still 100% off limits.

   I see zero evidence that libdevice-mapper is written to be async signal safe. Purely by luck we're not triggering a function that will cause a deadlock, or maybe we are but havent hit the deadlock race yet.

Comment 10 yafu 2020-07-21 07:32:29 UTC
Hi Daniel,

It seems the same issue with Bug 1822522 - avc: denied { read write } for pid=3449 comm="qemu-kvm" path="/dev/mapper/control".
And there is patch for the issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1822522#c6

Comment 13 Michal Privoznik 2020-07-22 09:41:07 UTC
Patches proposed upstream:

https://www.redhat.com/archives/libvir-list/2020-July/msg01500.html

Comment 18 Michal Privoznik 2020-07-27 07:43:14 UTC
I've merged patches upstream:

e450ebb4c6 virDevMapperGetTargets: Don't ignore EBADF
2249455654 virdevmapper: Don't use libdevmapper to obtain dependencies
b8ebbe0545 virDevMapperGetTargetsImpl: Use VIR_AUTOSTRINGLIST
ae5752aabc virdevmapper.c: Join two WITH_DEVMAPPER sections together

v6.6.0-rc1-4-ge450ebb4c6

Comment 19 Zdenek Pytela 2020-07-29 17:31:28 UTC
*** Bug 1835909 has been marked as a duplicate of this bug. ***

Comment 20 Zdenek Pytela 2020-07-29 17:32:04 UTC
*** Bug 1822522 has been marked as a duplicate of this bug. ***

Comment 24 yafu 2020-09-14 01:29:29 UTC
Reproduced with libvirt-daemon-6.0.0-17.3.el8.x86_64.

1.Start a guest without multipath disk:
#virsh start rhel8.3 
Domain rhel8.3 started

# virsh domblklist rhel8.3 
 Target   Source
------------------------------------------------------------------
 vda      /var/lib/libvirt/images/RHEL-8.3-x86_64-latest.qcow2.1

2.Check the audit log:
# ausearch -m avc | grep -i control | grep -i mapper
type=AVC msg=audit(1600045610.463:8082): avc:  denied  { read write } for  pid=14416 comm="qemu-kvm" path="/dev/mapper/control" dev="devtmpfs" ino=24622 scontext=system_u:system_r:svirt_t:s0:c380,c661 tcontext=system_u:object_r:lvm_control_t:s0 tclass=chr_file permissive=0

Verified with libvirt-6.6.0-4.module+el8.3.0+7883+3d717aa8.x86_64.
1.Start a guest without multipath disk:
#virsh start rhel8.3 
Domain rhel8.3 started

# virsh domblklist rhel8.3 
 Target   Source
------------------------------------------------------------------
 vda      /var/lib/libvirt/images/RHEL-8.3-x86_64-latest.qcow2.1

2.Check the audit log:
#ausearch -m avc | grep -i control | grep -i mapper
no output

Comment 27 errata-xmlrpc 2020-11-17 17:50:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5137


Note You need to log in before you can comment on or make changes to this bug.