Bug 798605

Summary: selinux denials with interface type='direct' mode='bridge'
Product: [Fedora] Fedora Reporter: John Mylchreest <jmylchreest>
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: berrange, clalancette, crobinso, dallan, dominick.grift, dougsland, dwalsh, itamar, jforbes, laine, libvirt-maint, mgrepl, urkle, veillard, virt-maint
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: abrt_hash:b71324959c4fc41127faf3b1113cc921e9f61c5f290a2ffc70ba2a267ede6e6b
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-11-28 09:56:52 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Description Flags
"offending vm xml config" none

Description John Mylchreest 2012-02-29 06:23:20 EST
libreport version: 2.0.8
executable:     /usr/bin/python
hashmarkername: setroubleshoot
kernel:         3.2.7-1.fc16.x86_64
reason:         SELinux is preventing /usr/bin/qemu-kvm from read, write access on the None /dev/tap17.
time:           Wed 29 Feb 2012 11:19:07 GMT

:SELinux is preventing /usr/bin/qemu-kvm from read, write access on the None /dev/tap17.
:*****  Plugin catchall (100. confidence) suggests  ***************************
:If you believe that qemu-kvm should be allowed read write access on the tap17 <Unknown> by default.
:Then you should report this as a bug.
:You can generate a local policy module to allow this access.
:allow this access for now by executing:
:# grep qemu-kvm /var/log/audit/audit.log | audit2allow -M mypol
:# semodule -i mypol.pp
:Additional Information:
:Source Context                system_u:system_r:svirt_t:s0:c181,c334
:Target Context                system_u:object_r:device_t:s0
:Target Objects                /dev/tap17 [ None ]
:Source                        qemu-kvm
:Source Path                   /usr/bin/qemu-kvm
:Port                          <Unknown>
:Host                          (removed)
:Source RPM Packages           qemu-system-x86-0.15.1-4.fc16.x86_64
:Target RPM Packages           
:Policy RPM                    selinux-policy-3.10.0-75.fc16.noarch
:Selinux Enabled               True
:Policy Type                   targeted
:Enforcing Mode                Enforcing
:Host Name                     (removed)
:Platform                      Linux (removed) 3.2.7-1.fc16.x86_64 #1 SMP Tue Feb 21
:                              01:40:47 UTC 2012 x86_64 x86_64
:Alert Count                   1
:First Seen                    Wed 29 Feb 2012 11:17:28 GMT
:Last Seen                     Wed 29 Feb 2012 11:17:28 GMT
:Local ID                      d18b4b2b-7c40-4679-aebc-96e6cf96e9b3
:Raw Audit Messages
:type=AVC msg=audit(1330514248.186:477): avc:  denied  { read write } for  pid=23623 comm="qemu-kvm" path="/dev/tap17" dev=devtmpfs ino=472574 scontext=system_u:system_r:svirt_t:s0:c181,c334 tcontext=system_u:object_r:device_t:s0 tclass=chr_filenode=(removed) type=SYSCALL msg=audit(1330514248.186:477): arch=c000003e syscall=59 success=yes exit=0 a0=7fc1b8001680 a1=7fc1b80070c0 a2=7fc1b8000d10 a3=0 items=0 ppid=1 pid=23623 auid=4294967295 uid=107 gid=107 euid=107 suid=107 fsuid=107 egid=107 sgid=107 fsgid=107 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/bin/qemu-kvm" subj=system_u:system_r:svirt_t:s0:c181,c334 key=(null)
:Hash: qemu-kvm,svirt_t,device_t,None,read,write
:audit2allow -R
Comment 1 Daniel Walsh 2012-02-29 14:15:25 EST
This looks like libvirt did not label the device correctly.
Comment 2 Laine Stump 2012-02-29 14:51:59 EST
Please attach the following to the bug report:

1) the output of "virsh dumpxml $guestname --inactive"
2) The last several lines of /var/log/libvirt/qemu/$guestname.log
3) Any messages from /var/log/libvirt/libvirtd.log from around the time of this failure.
Comment 3 John Mylchreest 2012-03-02 06:24:57 EST
Thanks for the reply. Since posting this I've made quite a few changes to the guest and built a few selinux modules around it also.

I'll look to recreate it and provide the detail you're asking for but it should be easily doable with:

* create vm using the virt-manager create wizard
* before beginning installation, alter the configuration
* change the NIC from NAT to macvtap (eth0 in my case), VEPA
* attempt to boot the new guest.

Comment 4 Laine Stump 2012-03-02 15:02:19 EST
This probably isn't relevant to the bug, but I do you actually have a VEPA capable switch, and if so, do you have lldpad installed? These are still very rare in the wild, and 90% of the time I see somebody using VEPA mode, it is in error - if you are connected to any switch that doesn't specifically support VEPA, you should instead use 'bridge' or 'private' mode (depending on whether you want the guests to be able to directly communicate with each other without bouncing all traffic off the switch)
Comment 5 Edward Rudd 2012-06-05 17:11:30 EDT
I'm receiving this error on RHEL 6 as well.. The key is the Selinux deny log entry.   creating a new semodule with the following resolves the issue (run audit2allow -M libvirt-tap on the log in the OP )

module libvirt-tap 1.0;

require {
        type device_t;
        type svirt_t;
        class chr_file { read write };

#============= svirt_t ==============
allow svirt_t device_t:chr_file { read write };
Comment 6 Cole Robinson 2012-06-07 16:38:33 EDT
Edward, are you seeing this issue when trying to start the VM? Can you provide the VM XML of the offending VM? sudo virsh dumpxml $vmname
Comment 7 Edward Rudd 2012-06-07 16:44:23 EDT
Created attachment 590296 [details]
"offending vm xml config"

I've attached the XML setup for the offending VM..
Comment 8 Cole Robinson 2012-06-07 17:08:16 EDT
Thanks Edward.

Here's the network bit:

    <interface type='direct'>
      <mac address='52:54:00:ae:5e:08'/>
      <source dev='eth0' mode='bridge'/>
      <target dev='macvtap0'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Comment 9 Cole Robinson 2012-10-20 18:35:23 EDT
I actually can't reproduce this on F17, so closing as WORKSFORME.

Edward, if you are still seeing this on RHEL6, please file a bug there or follow up with RH support.

If anyone can reproduce on F17+, please reopen.
Comment 10 Laine Stump 2012-10-22 22:16:17 EDT
As a matter of fact, we just recently figured out (on RHEL6, but it is a verified problem in F17, F18, and Rawhide as well) that there was a race condition when macvtap devices are created that can cause an selinux AVC failure if the new device's fd is passed to qemu too quickly. The problem is that the kernel creates macvtap devices with the label device_t, and relies on udev to asynchronously notice this and relabel them to tun_tap_device_t (which is the label that qemu has permissions for). If udev was excessively slow to notice and take action, or libvirt was excessively quick to pass the device's fd off to qemu, qemu would fail in exactly the manner described here.

The solution is to add code to libvirt to relabel each newly created macvtap device with the selinux label *it should have had to begin with* (the effect is just to guarantee the default label is in place before using the fd).

This was reported against RHEL6 in Bug 851981 and fixed upstream in several patches (there were a couple prerequisites, and a few missteps in figuring out the correct fix). Unfortunately it seems non-trivial to backport these patches to the 0.9.11-maint branch for F17 (much less to v0.9.6-maint for F16), since the security driver code went through a major refactoring just before 0.10.0.

These patches *can* easily be applied on top of 0.10.2, though, and that will probably be the maintenance branch for F18, so I'm changing the Fedora version of this bug to F18 and setting it to POST (but not pushing the patches to v0.10.2-maint until verifying that's what will be F18)

commit 9674f2c637114fa6ac0680fe5658a41a62bb34a8
Author: Martin Kletzander <mkletzan@redhat.com>
Date:   Fri Oct 5 16:41:22 2012 +0200

    selinux: Use raw contexts

commit ae368ebfcc4923d0b32e83d4ca96a6f599625785
Author: Guannan Ren <gren@redhat.com>
Date:   Mon Oct 15 17:03:49 2012 +0800

    selinux: add security selinux function to label tapfd
    When using macvtap, a character device gets first created by
    kernel with name /dev/tapN, its selinux context is:
    Shortly, when udev gets notification when new file is created
    in /dev, it will then jump in and relabel this file back to the
    expected default context:
    There is a time gap happened.
    Sometimes, it will have migration failed, AVC error message:
    type=AVC msg=audit(1349858424.233:42507): avc:  denied  { read write } for
    pid=19926 comm="qemu-kvm" path="/dev/tap33" dev=devtmpfs ino=131524
    tcontext=system_u:object_r:device_t:s0 tclass=chr_file
    This patch will label the tapfd device before qemu process starts:
    system_u:object_r:tun_tap_device_t:MCS(MCS from seclabel->label)

commit 89b63f0ad448a0442f4afc5489748e2cc829e527
Author: Guannan Ren <gren@redhat.com>
Date:   Wed Oct 17 11:23:19 2012 +0800

    selinux: fix wrong tapfd relablling
    It should relabel tapfd of virtual network of type VIR_DOMAIN_NET_TYPE_DIRECT
    (commit ae368ebfcc4923d0b32e83d4ca96a6f599625785 introduced this bug)
    Caution: The context of the two hunks is identical other than indentation.
    Please be extremely cautious of where the patch gets applied.

commit 6676c1fc8fb297de998f96862cbe2396355b2777
Author: Martin Kletzander <mkletzan@redhat.com>
Date:   Mon Oct 15 14:04:36 2012 +0200

    selinux: Use raw contexts 2
    In commit 9674f2c637114fa6ac0680fe5658a41a62bb34a8, I forgot to change
    selabel_lookup with the other functions, so this one-liner does exactly

commit d37a3a1d6c6508f235965185453602ba310cc66e
Author: Guannan Ren <gren@redhat.com>
Date:   Wed Oct 17 11:28:28 2012 +0800

    selinux: remove unused variables in socket labelling

commit 4492ef7f485a7d42d84a714d2150e648b11e2740
Author: Guannan Ren <gren@redhat.com>
Date:   Fri Oct 19 16:44:30 2012 +0800

    selinux: relabel tapfd in qemuPhysIfaceConnect
    Relabeling tapfd right after the tap device is created.
    qemuPhysIfaceConnect is common function called both for static
    netdevs and for hotplug netdevs.
Comment 11 Cole Robinson 2012-10-23 11:40:53 EDT
Cool Laine, thanks for the diligence!
Comment 12 Fedora Update System 2012-10-27 18:31:44 EDT
libvirt- has been submitted as an update for Fedora 18.
Comment 13 Fedora Update System 2012-10-28 12:33:58 EDT
Package libvirt-
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing libvirt-'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
Comment 14 Fedora Update System 2012-10-30 21:20:51 EDT
libvirt- has been submitted as an update for Fedora 18.
Comment 15 Cole Robinson 2012-11-28 09:56:52 EST
libvirt- now in stable