RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1210263 - Need SELinux policy updates for InfiniBand: RDMA migration fails with SELinux
Summary: Need SELinux policy updates for InfiniBand: RDMA migration fails with SELinux
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: selinux-policy
Version: 7.2
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Lukas Vrabec
QA Contact: Marek Haicman
URL:
Whiteboard:
Depends On:
Blocks: 1331802
TreeView+ depends on / blocked
 
Reported: 2015-04-09 09:59 UTC by Dr. David Alan Gilbert
Modified: 2016-11-04 02:18 UTC (History)
16 users (show)

Fixed In Version: selinux-policy-3.13.1-86.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1331802 (view as bug list)
Environment:
Last Closed: 2016-11-04 02:18:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2283 0 normal SHIPPED_LIVE selinux-policy bug fix and enhancement update 2016-11-03 13:36:25 UTC

Description Dr. David Alan Gilbert 2015-04-09 09:59:30 UTC
Description of problem:
In SELinux enforcing mode, an RDMA migration fails with:
librdmacm: Fatal: unable to open /dev/infiniband/rdma_cm

in permissive we see in audit.log (on the destination):
type=VIRT_RESOURCE msg=audit(1428573029.576:1228): pid=1840 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:virtd_t:s0-s0:c0.c1023 msg='virt=kvm resrc=cgroup reason=allow vm="rhel71on6" uuid=756bdee2-1f44-494b-91c4-a91c0f4d9cca cgroup="/sys/fs/cgroup/devices/machine.slice/machine-qemu\x2drhel71on6.scope/" class=major category=pty maj=88 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'
type=AVC msg=audit(1428573030.167:1229): avc:  denied  { read write } for  pid=17643 comm="qemu-kvm" name="rdma_cm" dev="devtmpfs" ino=27794 scontext=system_u:system_r:svirt_t:s0:c481,c874 tcontext=system_u:object_r:infiniband_device_t:s0 tclass=chr_file
type=AVC msg=audit(1428573030.167:1229): avc:  denied  { open } for  pid=17643 comm="qemu-kvm" path="/dev/infiniband/rdma_cm" dev="devtmpfs" ino=27794 scontext=system_u:system_r:svirt_t:s0:c481,c874 tcontext=system_u:object_r:infiniband_device_t:s0 tclass=chr_file
type=SYSCALL msg=audit(1428573030.167:1229): arch=c000003e syscall=2 success=yes exit=29 a0=7f9581ab7c5c a1=80002 a2=7f958a08bf90 a3=7fffaf8a3060 items=0 ppid=1 pid=17643 auid=4294967295 uid=107 gid=107 euid=107 suid=107 fsuid=107 egid=107 sgid=107 fsgid=107 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/libexec/qemu-kvm" subj=system_u:system_r:svirt_t:s0:c481,c874 key=(null)

[root@virtlab414 ~]# ls -lZ /dev/infiniband/
crw-------. root root system_u:object_r:infiniband_device_t:s0 issm0
crw-------. root root system_u:object_r:infiniband_device_t:s0 issm1
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 rdma_cm
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 ucm0
crw-------. root root system_u:object_r:infiniband_device_t:s0 umad0
crw-------. root root system_u:object_r:infiniband_device_t:s0 umad1
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 uverbs0


Version-Release number of selected component (if applicable):
[root@virtlab414 ~]# rpm -qa | grep libvirt
libvirt-daemon-driver-interface-1.2.8-16.el7_1.2.x86_64
libvirt-client-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-secret-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-qemu-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-nwfilter-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-network-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-nodedev-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-kvm-1.2.8-16.el7_1.2.x86_64
libvirt-daemon-driver-storage-1.2.8-16.el7_1.2.x86_64
[root@virtlab414 ~]# rpm -qa | grep qemu
qemu-guest-agent-2.1.0-4.el7.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.1.x86_64
libvirt-daemon-driver-qemu-1.2.8-16.el7_1.2.x86_64
qemu-kvm-common-rhev-2.1.2-23.el7_1.1.x86_64
qemu-kvm-rhev-debuginfo-2.1.2-23.el7_1.1.x86_64
qemu-kvm-tools-rhev-2.1.2-23.el7_1.1.x86_64
qemu-img-rhev-2.1.2-23.el7_1.1.x86_64
ipxe-roms-qemu-20130517-6.gitc4bce43.el7.noarch
[root@virtlab414 ~]# rpm -qa | grep policy
selinux-policy-targeted-3.13.1-23.el7.noarch
selinux-policy-3.13.1-23.el7.noarch
policycoreutils-2.2.5-15.el7.x86_64

(This is a 7 upgraded to 7.1 host)

How reproducible:
100%

Steps to Reproduce:
1. Take two hosts with RDMA cards
2. Start a VM on one
3. virsh migrate --live --migrateuri rdma://ibpair rhel71on6 qemu+ssh://ibpair/system

   (where ibpair is the name of the other host)

Actual results:
error: internal error: early end of file from monitor: possible problem:
librdmacm: Fatal: unable to open /dev/infiniband/rdma_cm
librdmacm: Fatal: unable to open /dev/infiniband/rdma_cm
RDMA ERROR: could not create rdma event channel
2015-04-09T09:43:28.047845Z qemu-kvm: -incoming rdma:[::]:49152: RDMA ERROR: could not create rdma event channel


Expected results:
A happy migration.

Additional info:

Comment 2 Jiri Denemark 2015-04-14 13:30:27 UTC
Is this something that svirt_t processes should be able to access (perhaps guarded with a new sebool) or should libvirt somehow relabel the file? I don't know enough about Infiniband and RDMA, but relabeling the file seems wrong to me because QEMU may not be the only user of RDMA.

Comment 3 Jiri Denemark 2015-04-20 14:46:29 UTC
Mirek Grepl confirmed a new sebool would be a better solution; moving to selinux-policy.

Comment 4 Miroslav Grepl 2015-04-27 11:24:50 UTC
I am also not sure about /dev/infiniband purpose. I need to do some research.

Should we label FD?

Comment 5 Dr. David Alan Gilbert 2015-05-05 11:03:18 UTC
(In reply to Miroslav Grepl from comment #4)
> I am also not sure about /dev/infiniband purpose. I need to do some research.
> 
> Should we label FD?

Infiniband is hardware that provides an RDMA transport between hosts.  As for the meaning of each of the files in /dev/inifiniband I don't know; dledford is a good contact for the details of how it's supposed to work.

Comment 9 Doug Ledford 2015-12-03 16:28:51 UTC
All of the infiniband files are basically command pipes.  You use the file to send commands to kernel modules.  The permissions on the files are intentionally.  Those with world writable permissions accept commands that any user is allowed to send.  Those with root only write permissions are files that accept privileged commands.  For the purposes of live migration, it would make sense that qemu would have access to the world writable files.  The root only writable files are not needed by qemu/kvm and would present a security risk if it were allowed to access them freely.

It would probably be best if we split the infiniband SElinux type into two types infiniband_user_t and infiniband_mgmt_t.  The world writable files would be infiniband_user_t and access should be freely allowed from other selinux contexts.  The infiniband_mgmt_t should be limited to certain applications (I can provide a list if needed).

Comment 10 Miroslav Grepl 2015-12-18 09:33:56 UTC
Moving to libvirt to have a discussion about possible labeling.

Comment 11 Jiri Denemark 2016-02-24 11:34:19 UTC
The labeling sounds OK to me, but IIUC the labeling will be static and not managed by libvirt. And libvirt will not even need to know what the labeling is. That said it's really up to infiniband and selinux-policy guys to discuss and implement the right labeling.

Comment 12 Lukas Vrabec 2016-04-28 10:07:39 UTC
(In reply to Doug Ledford from comment #9)
> All of the infiniband files are basically command pipes.  You use the file
> to send commands to kernel modules.  The permissions on the files are
> intentionally.  Those with world writable permissions accept commands that
> any user is allowed to send.  Those with root only write permissions are
> files that accept privileged commands.  For the purposes of live migration,
> it would make sense that qemu would have access to the world writable files.
> The root only writable files are not needed by qemu/kvm and would present a
> security risk if it were allowed to access them freely.
> 
> It would probably be best if we split the infiniband SElinux type into two
> types infiniband_user_t and infiniband_mgmt_t.
Could you define which devices should have user and which mgmt type? 
[root@virtlab414 ~]# ls -lZ /dev/infiniband/
crw-------. root root system_u:object_r:infiniband_device_t:s0 issm0
crw-------. root root system_u:object_r:infiniband_device_t:s0 issm1
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 rdma_cm
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 ucm0
crw-------. root root system_u:object_r:infiniband_device_t:s0 umad0
crw-------. root root system_u:object_r:infiniband_device_t:s0 umad1
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 uverbs0



  The world writable files
> would be infiniband_user_t and access should be freely allowed from other
> selinux contexts.  The infiniband_mgmt_t should be limited to certain
> applications (I can provide a list if needed).
Could you provide a list? 

Thank you.

Comment 13 Lukas Vrabec 2016-06-14 11:40:13 UTC
Hi dledford, 

Could you check comment12 ? 

Thank you.

Comment 14 Doug Ledford 2016-06-22 16:21:30 UTC
I already answered which files should have which SELinux types:

world writeable (mode 666) files should have normal IB type
root writeable (mode 600) files should have IB mgmt type

Apps that will need IB mgmt type (at a minimum, I might have missed some):

From the opensm package:

/usr/sbin/opensm
/usr/sbin/osmtest

from ibutils:

/usr/bin/*

from ibacm:

/usr/bin/*
/usr/sbin/*

from infiniband-diags:

/usr/sbin/*

from srptools:

/usr/sbin/srp_daemon
/usr/sbin/ibsrpdm

from perftest:

unknown, will need to test it and see what breaks without having mgmt type and decide if we want to give mgmt type or keep the breakage

from mvapich2:

ditto perftest

opa-basic-tools:

unknown, will have to test and make recommendations

opa-address-resolution:

won't need a change, it will get it's type from ibacm as it is just a plugin library that ibacm uses

opa-fm:

will need testing to confirm, but I suspect it needs /usr/sbin/*, /usr/lib/opa-fm/bin/*, /usr/lib/opa-fm/runtime/*

That's the rough list.  It might need additional tweaks based on testing, and even though I think I have all of the umad using apps installed at the moment, I could have missed some.

Comment 15 Lukas Vrabec 2016-06-29 12:49:31 UTC
I split the devices as was mentioned in comment14. Can I make scratch builds for testing and collecting some AVCs?

Comment 16 Lukas Vrabec 2016-06-29 13:24:35 UTC
Attaching scratch builds:

http://download.eng.brq.redhat.com/scratch/lvrabec/scratch_selinux_policy_infiniband/

Comment 17 Lukas Vrabec 2016-07-04 13:33:42 UTC
Could somebody test it and collect AVC denials? 

Thank you.

Comment 20 Dr. David Alan Gilbert 2016-07-07 11:09:21 UTC
Hi Lukas,
  It still seems to be unhappy with your packages;

(again this is from the destination side):

type=AVC msg=audit(1467889442.717:292): avc:  denied  { read write } for  pid=3385 comm="qemu-kvm" name="rdma_cm" dev="devtmpfs" ino=18626 scontext=system_u:system_r:svirt_t:s0:c333,c758 tcontext=system_u:object_r:infiniband_device_t:s0 tclass=chr_file
type=AVC msg=audit(1467889442.717:293): avc:  denied  { read write } for  pid=3385 comm="qemu-kvm" name="rdma_cm" dev="devtmpfs" ino=18626 scontext=system_u:system_r:svirt_t:s0:c333,c758 tcontext=system_u:object_r:infiniband_device_t:s0 tclass=chr_file

qemu-kvm-rhev-2.6.0-9.el7.x86_64
libvirt-1.3.5-1.el7.x86_64
selinux-policy-3.13.1-82.el7.5.noarch
selinux-policy-targeted-3.13.1-82.el7.5.noarch

[root@virtlab403 ~]# ls -lZ /dev/infiniband/
crw-------. root root system_u:object_r:infiniband_mgmt_device_t:s0 issm0
crw-------. root root system_u:object_r:infiniband_mgmt_device_t:s0 issm1
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 rdma_cm
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 ucm0
crw-------. root root system_u:object_r:infiniband_mgmt_device_t:s0 umad0
crw-------. root root system_u:object_r:infiniband_mgmt_device_t:s0 umad1
crw-rw-rw-. root root system_u:object_r:infiniband_device_t:s0 uverbs0

Comment 21 Lukas Vrabec 2016-07-07 12:21:58 UTC
Hi, 

These AVCs are expected, it's easy fix. I see fixed labeling here. That's good. I'm going to fix these AVCs asap.

Thank you for testing.

Comment 32 Xuesong Zhang 2016-09-07 05:09:45 UTC
hi, 

Is there any other issues with this fix selinux build? Does thie bug can be tested and verified now.
We are verifying one libvirt testonly bug 1331802, which is depend on this selinux bug. If no other issues for this selinux bug, we'd like to change our libvirt bug to verify status. Thanks.
Following is the verify result and steps for your reference:
https://bugzilla.redhat.com/show_bug.cgi?id=1331802#c2

Comment 35 Marek Haicman 2016-09-12 11:19:57 UTC
Functional testing verified the version selinux-policy-3.13.1-86.el7 has necessary rules.

Moving to verified.

Comment 38 errata-xmlrpc 2016-11-04 02:18:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2283.html


Note You need to log in before you can comment on or make changes to this bug.