Bug 2048251

Summary: Selinux is not allowing SCTP connection setup between inter pod communication in enforcing mode
Product: Red Hat Enterprise Linux 8 Reporter: Jitendra Pradhan <jpradhan>
Component: kernelAssignee: Ondrej Mosnacek <omosnace>
kernel sub component: SELinux QA Contact: Milos Malik <mmalik>
Status: CLOSED ERRATA Docs Contact: Jan Fiala <jafiala>
Severity: urgent    
Priority: urgent CC: amkulkar, arghosh, blitton, bzvonar, dwalsh, eglottma, jbainbri, jnovy, Lgrunewa, lvrabec, mleonard, mmalik, omosnace, rjamadar, scorcora, ssekidde, sujagtap, tsweeney, xzhou, ykashtan, zpytela
Version: 8.5Keywords: AutoVerified, Triaged, ZStream
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-4.18.0-372.1.1.el8 Doc Type: Bug Fix
Doc Text:
.SELinux no longer incorrectly denies SCTP communication Previously, SELinux incorrectly denied SCTP communication between containers in enforcing mode when using an SCTP socket to both receive and initiate SCTP associations. As a consequence, when an association was first received, then initiated, and then received again, SELinux incorrectly denied the last association. With this update, the SCTP and SELinux kernel code has been fixed to correctly handle socket peer label initialization on outgoing associations. As a result, SELinux no longer performs incorrect AVC checks when an SCTP association is being set up.
Story Points: ---
Clone Of:
: 2054116 2054117 (view as bug list) Environment:
Last Closed: 2022-05-10 15:12:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2015525    
Bug Blocks: 2054116, 2054117    
Attachments:
Description Flags
Tarball with reproducer
none
Tarball with reproducer v2
none
Tarball with reproducer v3 none

Description Jitendra Pradhan 2022-01-30 15:20:49 UTC
Description of problem:

Selinux won't allow SCTP communication between containers in enforcing mode.

Version-Release number of selected component (if applicable):

RHEL 8.5 

How reproducible:

sctp association can't be set up from container up(49157) to container cp(49156),but audit log shows that cp pod/cp container to cp pod/mp container is denied.

The count of discard SCTP INIT is 9 and the count of denied association also 9.


Expected results:
SCTP connection between two pod should work in enforcing mode of SELinux.



Additional info:

Please refer the case #03134669 for more information like sosreport, must-gather, strace, perf report and etc. It has all the requisites to proceed further.

Also earlier there was a bugzilla  https://bugzilla.redhat.com/show_bug.cgi?id=1769469 which was for same cause but for the permissive mode of SELinux. Now our customer is not facing any issue for permissive mode.

Comment 3 Daniel Walsh 2022-01-31 17:29:48 UTC
I need AVC messages to see if this is already fixed.

Comment 4 milti leonard 2022-01-31 18:35:47 UTC
@dwalsh, the support case [1] contains the SRs where you should be able to find the AVC msgs youre requesting.

[1] https://access.redhat.com/support/cases/internal/#/case/03134669

Comment 15 Daniel Walsh 2022-02-07 15:37:12 UTC
This is very strange, you have unlabeled_t attempting to talk to unlabeled_t.  Unlabeled processes should not be showing up on this system


It is my feeling that this system did something to update selinux policy which ended up screwing up the labeling system.

Comment 18 Yuval Kashtan 2022-02-10 07:19:30 UTC
based on C#17
this BZ is, for the very least, blocked by 2015525

Comment 23 Ondrej Mosnacek 2022-02-12 18:46:48 UTC
(Switching component to kernel to avoid confusion, as that's where the bug is.)

Comment 24 Ondrej Mosnacek 2022-02-13 10:17:31 UTC
So, I believe I have now figured out how exactly this bug is triggered. Basically, the denial will happen after this sequence of events on the same SCTP socket:
1. An association request is received on the socket from a remote peer (and the assoc is opened).
2. The same socket establishes a new association with a remote peer (can be the same one as in 1.).
3. The same socket receives another association request from a remote peer (again doesn't matter what peer, as long as it's a new assoc).

Step 3 is where the denial happens. The root of the problem is that the author of the SELinux SCTP support patches failed to account for the fact that with SCTP SEQPACKET sockets it is possible to both open and receive associations without switching between listening and connected states. In contrast, BZ 2015525 is specific to client-side peeloff, which was also not accounted for in the original patches.

Good news is that the patches proposed for BZ 2015525 will fix this bug (I verified this on a Fedora kernel with the patches applied), too, so there are no additional patches needed. I will attach a basic reproducer that triggers the issue shortly.

Comment 25 Ondrej Mosnacek 2022-02-13 10:22:08 UTC
Created attachment 1860859 [details]
Tarball with reproducer

Comment 26 Ondrej Mosnacek 2022-02-13 10:32:05 UTC
Created attachment 1860860 [details]
Tarball with reproducer v2

(Fixed missing Makefile in the tarball.)

Comment 28 Yuval Kashtan 2022-02-14 07:50:25 UTC
this selinux policy is able to workaround (both of) the issue:
```
module sctp_wa 1.0;

require {
	type kernel_t;
	type unlabeled_t;
	type container_t;
	class system module_request;
	class sctp_socket { association read };
}

#============= container_t ==============

allow container_t kernel_t:system module_request;
allow container_t unlabeled_t:sctp_socket read;
allow unlabeled_t self:sctp_socket association;
```

this can be applied with
https://github.com/yuvalk/ocp-security/blob/master/sctp/selinux_custom.deployment.yaml

Comment 35 Ondrej Mosnacek 2022-02-14 15:55:46 UTC
Created attachment 1861001 [details]
Tarball with reproducer v3

I found a bug in the reproducer which caused it to fail spuriously. Attaching a v3, which should fix this. (In our test case derived from the reproducer this is already fixed.)

Comment 36 Ondrej Mosnacek 2022-02-15 07:45:59 UTC
(In reply to Ondrej Mosnacek from comment #24)
> So, I believe I have now figured out how exactly this bug is triggered.
> Basically, the denial will happen after this sequence of events on the same
> SCTP socket:
> 1. An association request is received on the socket from a remote peer (and
> the assoc is opened).
> 2. The same socket establishes a new association with a remote peer (can be
> the same one as in 1.).
> 3. The same socket receives another association request from a remote peer
> (again doesn't matter what peer, as long as it's a new assoc).

There seem to be some misunderstandings, so let me try to describe the root cause better...

Suppose we have an SCTP SEQPACKET socket (say, struct sock *sk). Initially (after going through selinux_socket_post_create()), the socket will have:

sk->sk_security->peer_sid = SECINITSID_UNLABELED
sk->sk_security->sctp_assoc_state = SCTP_ASSOC_UNSET

Now suppose the socket receives a message from another peer. This creates a new (first) association and selinux_sctp_assoc_request() is called(*) to initialize it on the SELinux side. This call (among other things) sets:

sk->sk_security->sctp_assoc_state = SCTP_ASSOC_SET
sk->sk_security->peer_sid = SECINITSID_UNLABELED (i.e. remains unchanged)

Then let's suppose this association is peeled off and then through this same socket a message is sent to a remote peer, establishing another association(**) - now via sctp_sf_do_5_1E_ca(), and selinux_inet_conn_established() is called, which happens to set (via selinux_skb_peerlbl_sid() -> security_net_peersid_resolve():

sk->sk_security->peer_sid = SECINITSID_NULL

Now again, let's suppose this association is peeled off and this time the same socket recieves another message to create another new association. Now selinux_sctp_assoc_request() is called again via sctp_sf_do_5_1B_init(), BUT this time the "} else if  (sksec->peer_sid != peer_sid) {" condition evaluates to true, because sk->sk_security->peer_sid == SECINITSID_NULL, while the freshly computed peer_sid == SECINITSID_UNLABELED. And in this branch we get to the avc_has_perm() call, which then does a check for the access vector (SECINITSID_NULL, SECINITSID_UNLABELED, sctp_socket, association), for which there is no matching rule in the policy, and thus a denial happens.

Then why does the AVC have 2x unlabeled_t? This is because SECINITSID_NULL is defined to be interpreted as SECINITSID_UNLABELED (kind of a "fallback", since SECINITSID_NULL doesn't represent any valid SELinux context.

In the reproducer, I wanted to avoid the need to use multiple peers, so all of the associations are set up with the same peer. To ensure that the association gets terminated between the steps, I always peel it off right after creating and close the peeled off sockets before the next step. This is enough to trigger the denial as reported by the customer.

The fact that the reproducer makes use of SCTP_PEELOFF means that it is also susceptible to BZ 2015525, so it is expected that it will behave differently when BZ 2015525 is unfixed vs. fixed/worked-around. But even with workaround in place for BZ 2015525 (e.g. running under the spc_t context), it should produce the denial that is the subject of this BZ.

However, we have been doing more experiments with Yuval and it seems the reproducer is still unstable - sometimes it fails too early with a weird error. We figured out that adding some extra delays helps. I will no longer maintain the reproducer here in the BZ, but Yuval maintains an up-to-date OCP-ready reproducer here:
https://github.com/yuvalk/ocp-security/tree/master/sctp

The kernel patches for BZ 2015525 fix also this issue, since they ensure that sksec->peer_sid is never SECINITSID_NULL, and thus the odd NULL<->UNLABELED comparison cannot happen. With peer labeling disabled sksec->peer_sid will always be SECINITSID_UNLABELED after the fix.

(*) This actually happens twice (once from sctp_sf_do_5_1B_init() and then from sctp_sf_do_5_1D_ce()), but the second time these fields remain unchanged.
(**) Note that this can happen after the first one has been torn down, it doesn't matter.

Comment 54 errata-xmlrpc 2022-05-10 15:12:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: kernel security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1988