Bug 842536

Summary: [FCoE] fcoe-utils cannot create FCoE session with SELinux enabled.
Product: Red Hat Enterprise Linux 7 Reporter: Gris Ge <fge>
Component: selinux-policyAssignee: Miroslav Grepl <mgrepl>
Status: CLOSED CURRENTRELEASE QA Contact: Milos Malik <mmalik>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.0CC: chaowang, dwalsh, edwardn, mbanas, mmalik, ohudlick, pfrields, qcai, salmy, xiaoli
Target Milestone: alphaKeywords: Regression, TestBlocker
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 10:21:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
SELinux log when creating FCoE
none
sealert.log none

Description Gris Ge 2012-07-24 06:58:36 UTC
Description of problem:

Using the same command of RHEL 6 to setup FCoE, but no FCoE initiator created.
Commands are:
====
yum install -y fcoe-utils
service fcoe start
service lldpad start
ifconfig p2p1 up
sleep 5
dcbtool sc p2p1 dcb on
sleep 5
dcbtool sc p2p1 pfc e:1 a:1 w:1
dcbtool sc p2p1 app:fcoe e:1 a:1 w:1

ifconfig p2p2 up
sleep 5
dcbtool sc p2p2 dcb on
sleep 5
dcbtool sc p2p2 pfc e:1 a:1 w:1
dcbtool sc p2p2 app:fcoe e:1 a:1 w:1

cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-p2p1 -f
cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-p2p2 -f

service fcoe restart
====

These are the outputs.
===
storageqe-13 login: [ 1468.890269] netlink: 12 bytes leftover after parsing attributes.
[ 1468.922175] netlink: 12 bytes leftover after parsing attributes.
[ 1468.956222] netlink: 12 bytes leftover after parsing attributes.
[ 1468.987472] netlink: 12 bytes leftover after parsing attributes.
[ 1469.186125] 8021q: 802.1Q VLAN Support v1.8
[ 1469.207325] 8021q: adding VLAN 0 to HW filter on device em0
[ 1469.234549] 8021q: adding VLAN 0 to HW filter on device em1
[ 1469.262514] 8021q: adding VLAN 0 to HW filter on device p2p1
[ 1469.290021] 8021q: adding VLAN 0 to HW filter on device p2p2
[ 1470.353778] ixgbe 0000:07:00.1: Multiqueue Enabled: Rx Queue count = 32, Tx Queue count = 32
[ 1471.375929] ixgbe 0000:07:00.1: p2p2: detected SFP+: 6
[ 1471.512365] ixgbe 0000:07:00.0: Multiqueue Enabled: Rx Queue count = 32, Tx Queue count = 32
[ 1472.327738] DMA-API: debugging out of memory - disabling
[ 1472.534455] ixgbe 0000:07:00.0: p2p1: detected SFP+: 5
[ 1472.843166] ixgbe 0000:07:00.1: p2p2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 1473.793954] ixgbe 0000:07:00.0: p2p1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 1474.947762] ixgbe 0000:07:00.1: Multiqueue Enabled: Rx Queue count = 32, Tx Queue count = 32
[ 1475.707074] ixgbe 0000:07:00.1: p2p2: detected SFP+: 6
[ 1477.085911] ixgbe 0000:07:00.0: Multiqueue Enabled: Rx Queue count = 32, Tx Queue count = 32
[ 1477.171185] ixgbe 0000:07:00.1: p2p2: NIC Link is Up 10 Gbps, Flow Control: None
[ 1477.846267] ixgbe 0000:07:00.0: p2p1: detected SFP+: 5
[ 1479.309709] ixgbe 0000:07:00.0: p2p1: NIC Link is Up 10 Gbps, Flow Control: None
[ 1489.509292] netlink: 12 bytes leftover after parsing attributes.
[ 1489.541418] netlink: 12 bytes leftover after parsing attributes.
[ 1489.574037] netlink: 12 bytes leftover after parsing attributes.
======


Tried to use 'fipvlan -c p2p1' to manually create VLAN sub-interface, and the use 'fcoeadm -c p2p1.802'. But fcoeadm failed with internal error, strace is:
=====
openat(AT_FDCWD, "/sys", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
brk(0)                                  = 0x7f2d66c1d000
brk(0x7f2d66c46000)                     = 0x7f2d66c46000
brk(0)                                  = 0x7f2d66c46000
brk(0)                                  = 0x7f2d66c46000
brk(0)                                  = 0x7f2d66c46000
brk(0x7f2d66c3e000)                     = 0x7f2d66c3e000
brk(0)                                  = 0x7f2d66c3e000
close(3)                                = 0
socket(PF_FILE, SOCK_DGRAM, 0)          = 3
bind(3, {sa_family=AF_FILE, NULL}, 2)   = 0
connect(3, {sa_family=AF_FILE, sun_path="/var/run/fcm/fcm_clif"}, 110) = 0
sendto(3, "\1\0\0\0p2p1.802\0\0\0\0\0\0\0\0", 20, 0, NULL, 0) = 20
select(4, [3], NULL, NULL, {30, 0})     = 0 (Timeout)
close(3)                                = 0
write(2, "fcoeadm: Internal error\n", 24fcoeadm: Internal error
=====

Tried to use sysfs to create fcoe session via this command:
=====
echo p2p1.802 > /sys/module/fcoe/parameters/create
=====

BUT file '/sys/module/fcoe/parameters/create' doesn't exists.

Version-Release number of selected component (if applicable):
fcoe-utils-1.0.22-2.el7.x86_64
kernel-3.3.0-0.20.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Check command above.
2.
3.
  
Actual results:
No FCoE session created.

Expected results:
FCoE session created automatically after lldpad.

Additional info:

This is basic function of fcoe-utils, must be fixed in beta release, requesting beta-blocker.

Comment 1 Gris Ge 2012-07-27 06:09:59 UTC
Workaround found:

===
fipvlan -c p2p1
fipvlan -c p2p2
echo p2p1.802 > /sys/module/libfcoe/parameters/create
echo p2p2.802 > /sys/module/libfcoe/parameters/create
===

The new sysfs file for soft FCoE creation changed to
===
/sys/module/libfcoe/parameters/create
#           ^^^^^^^
===

So current problem is:

1. dcbtool didn't create sub interface via fipvlan. Expected interface name should be
   p2p1.<VLAN_ID>-fcoe

2. fcoeadm cannot create fcoe session automatically.

Comment 2 WANG Chao 2012-09-04 10:19:25 UTC
Reproduced on rhel7 pre-beta compose(0903.n.1),

fcoe-utils-1.0.24-1.el7.x86_64
kernel-3.5.0-0.24.el7

But can work around regarding comment 1

Comment 3 Petr Šabata 2012-11-28 12:30:16 UTC
Note: I've just updated fcoe-utils to 1.0.25 in el7.

Comment 4 Gris Ge 2013-02-17 04:30:33 UTC
Petr Šabata,

Issue still exists on fcoe-utils-1.0.25-2.el7.x86_64.

Comment 9 Petr Šabata 2013-02-26 10:08:29 UTC
This is blocked by SELinux.
Everything seems to work properly with SELinux disabled.

Denials found in the audit.log:
type=AVC msg=audit(1361872880.616:321): avc:  denied  { create } for  pid=16387 comm="fcoemon" scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=packet_socket
type=AVC msg=audit(1361872880.617:322): avc:  denied  { create } for  pid=16387 comm="fcoemon" scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=packet_socket
type=AVC msg=audit(1361872944.653:323): avc:  denied  { create } for  pid=16387 comm="fcoemon" scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=udp_socket
type=AVC msg=audit(1361872944.653:324): avc:  denied  { write } for  pid=16387 comm="fcoemon" name="create" dev="sysfs" ino=17985 scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:object_r:sysfs_t:s0 tclass=file
type=AVC msg=audit(1361872944.653:325): avc:  denied  { sendto } for  pid=16387 comm="fcoemon" path=003030303039 scontext=system_u:system_r:fcoemon_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=unix_dgram_socket
type=AVC msg=audit(1361873064.798:327): avc:  denied  { sendto } for  pid=16387 comm="fcoemon" path=003030303061 scontext=system_u:system_r:fcoemon_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=unix_dgram_socket
type=AVC msg=audit(1361873105.291:331): avc:  denied  { create } for  pid=16423 comm="fcoemon" scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=packet_socket
type=AVC msg=audit(1361873105.291:331): avc:  denied  { net_raw } for  pid=16423 comm="fcoemon" capability=13  scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=capability
type=AVC msg=audit(1361873105.291:332): avc:  denied  { ioctl } for  pid=16423 comm="fcoemon" path="socket:[28315]" dev="sockfs" ino=28315 scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=packet_socket
type=AVC msg=audit(1361873105.300:333): avc:  denied  { setopt } for  pid=16423 comm="fcoemon" scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=packet_socket
type=AVC msg=audit(1361873105.300:334): avc:  denied  { bind } for  pid=16423 comm="fcoemon" scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=packet_socket
type=AVC msg=audit(1361873105.374:335): avc:  denied  { create } for  pid=16423 comm="fcoemon" scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=udp_socket
type=AVC msg=audit(1361873105.374:336): avc:  denied  { ioctl } for  pid=16423 comm="fcoemon" path="socket:[36592]" dev="sockfs" ino=36592 scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:system_r:fcoemon_t:s0 tclass=udp_socket
type=AVC msg=audit(1361873105.374:337): avc:  denied  { write } for  pid=16423 comm="fcoemon" name="create" dev="sysfs" ino=17985 scontext=system_u:system_r:fcoemon_t:s0 tcontext=system_u:object_r:sysfs_t:s0 tclass=file

Comment 10 Miroslav Grepl 2013-02-26 14:44:01 UTC
We need to add fixes for fcoemon_t policy.

type=AVC msg=audit(1361872944.653:325): avc:  denied  { sendto } for  pid=16387 comm="fcoemon" path=003030303039 scontext=system_u:system_r:fcoemon_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=unix_dgram_socket
type=AVC msg=audit(1361873064.798:327): avc:  denied  { sendto } for  pid=16387 comm="fcoemon" path=003030303061 scontext=system_u:system_r:fcoemon_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=unix_dgram_socket

Any idea with which process does fcoemon communicate?

Comment 11 Milos Malik 2013-02-26 15:57:14 UTC
My guess is lldpad (lldpad_t).

Comment 12 Milos Malik 2013-02-26 16:22:34 UTC
Hi guys, could you run the reproducer in permissive mode, gather all AVCs and attach them here? I don't know the configuration of p2p1 and p2p2 devices.

Comment 13 Gris Ge 2013-02-28 08:20:36 UTC
Created attachment 703836 [details]
SELinux log when creating FCoE

The server is storageqe-13.rhts.eng.bos.redhat.com with default password.

If this log does not meet your need, please feel free to login and check around.

Thanks.

Confirmed, with SELinux disabled, fcoe-utils can create FCoE session as expected.

Comment 14 Gris Ge 2013-02-28 08:42:40 UTC
Changed the title of this bug as bnx2fc also fail with SElinux issue.

Comment 15 Miroslav Grepl 2013-02-28 12:47:42 UTC
(In reply to comment #11)
> My guess is lldpad (lldpad_t)

But we see unconfined_t. Maybe it has been started by hand. 

I added some fixes to rawhide.

Comment 16 Milos Malik 2013-02-28 15:22:31 UTC
Could it be fixed in selinux-policy 3.11.1 ?

Comment 23 Gris Ge 2013-04-17 10:52:07 UTC
FCoE session automatically created with these SELinux rpms:
===
libselinux-utils-2.1.13-3.el7.x86_64
libselinux-2.1.13-3.el7.x86_64
libselinux-python-2.1.13-3.el7.x86_64
selinux-policy-devel-3.11.1-75.el7.noarch
selinux-policy-3.12.1-29.el7.noarch
selinux-policy-targeted-3.12.1-29.el7.noarch
selinux-policy-doc-3.12.1-29.el7.noarch
===

But be advised, RHEL-7.0-20130403.0 repo is still holding the old
SELinux.

Once new selinux rpms goes to any RTT repo, will try again.

Comment 24 edwardn 2014-01-16 02:20:05 UTC
It looks like we are still seeing this issue with bnx2fc and SELinux enabled with RH7.0 SS1 release.  Does anyone know if this is supposed to be fixed?  Thanks.

Comment 25 Miroslav Grepl 2014-01-16 09:56:16 UTC
(In reply to edwardn from comment #24)
> It looks like we are still seeing this issue with bnx2fc and SELinux enabled
> with RH7.0 SS1 release.  Does anyone know if this is supposed to be fixed? 
> Thanks.

Are you getting AVC msgs?

Comment 26 edwardn 2014-01-16 18:32:51 UTC
(In reply to Miroslav Grepl from comment #25) 
> Are you getting AVC msgs?

Seeing the following message when running the fcoeadm command below.  

Jan 16 10:22:02 b154 setroubleshoot: SELinux is preventing /usr/sbin/fcoemon from sendto access on the unix_dgram_socket @fcm_clif/139897063048592. For complete SELinux messages. run sealert -l 3d667027-f565-4246-97be-a2876223c66a

[root@b154 ~]# fcoeadm -c em2
fcoeadm: Internal error
Try 'fcoeadm --help' for more information.

Attached log (sealert.log) is the output when running "sealert -l 3d667027-f565-4246-97be-a2876223c66a".  If selinux is disabled in /etc/selinux/config, problem is not seen.

Comment 27 edwardn 2014-01-16 18:33:54 UTC
Created attachment 851218 [details]
sealert.log

Comment 28 Miroslav Grepl 2014-01-17 10:04:23 UTC
(In reply to edwardn from comment #27)
> Created attachment 851218 [details]
> sealert.log

We have another bug for this issue.

Comment 31 Ludek Smid 2014-06-13 10:21:49 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.