It's part of kernel-modules-extra which IMO was a well-intentioned but ultimately ill-advised attempt to support smaller installations.  (Anyways at this point RHCOS is huge due to kubelet and kernel-modules-extra weighs in at 600k, a whole 0.3% of the size of hyperkube).

(In reply to Colin Walters from comment #1)
> It's part of kernel-modules-extra which IMO was a well-intentioned but
> ultimately ill-advised attempt to support smaller installations.  (Anyways
> at this point RHCOS is huge due to kubelet and kernel-modules-extra weighs
> in at 600k, a whole 0.3% of the size of hyperkube).

To be clear, kernel-modules-extra was introduced in Fedora as a compromise for modules we really wanted to disable entirely but didn't have the gumption to follow through on.  It is one of my biggest regrets from when I did the kernel repackaging, and I would completely delete it if I could go back in time.

It got carried into RHEL 8 from there and adjusted.  SCTP probably doesn't belong there.

The overall kernel-core/kernel-modules split actually does help in many cases, but clearly it depends on what each environment needs.  The Fedora version was fine for VMs to only have kernel-core.  I would think something similar could work for CoreOS, but I haven't followed what RHEL did.

> It got carried into RHEL 8 from there and adjusted.  SCTP probably doesn't belong there.

Yeah, probably.  But SCTP is in an interesting space adoption wise; feels ultimately like HTTP2 is taking over a lot of that space, and ultimately HTTP3 will take that farther.

>   The Fedora version was fine for VMs to only have kernel-core.  I would think something similar could work for CoreOS

We don't want to have separate builds for physical/virtual.

(In reply to Colin Walters from comment #4)
> Yeah, probably.  But SCTP is in an interesting space adoption wise; feels
> ultimately like HTTP2 is taking over a lot of that space, and ultimately
> HTTP3 will take that farther.

For sure, the influence of SCTP on QUIC is unmistakable. Regardless, SCTP is widely used in the telco space, and that won't be changing anytime soon.

https://gitlab.cee.redhat.com/coreos/redhat-coreos/merge_requests/380

I think sctp.ko is considered by some to be in the set of kernel modules that haven't had enough eyes on them and so are more likely to have undiscovered security holes. It might be good if admins had to explicitly enable it.

So the linked CoreOS MR brings in the kernel-modules-extra package, but that includes a file /etc/modprobe.d/sctp-blacklist.conf blacklisting the module. Plus, our selinux policy blocks containers from being able to cause modules to be autoloaded even if they aren't blacklisted:

    danw@p50:~> oc exec test-pod -- nc -l --sctp -p 9999
    Ncat: Unable to open any listening sockets. QUITTING.
    command terminated with exit code 2

with the kernel logging:

    [ 1957.432504] audit: type=1400 audit(1561323434.328:7): avc:  denied  { module_request } for  pid=68007 comm="nc" kmod="net-pf-10-proto-132-type-1" scontext=system_u:system_r:container_t:s0:c519,c876 tcontext=system_u:system_r:kernel_t:s0 tclass=system permissive=0
    [ 1957.449268] audit: type=1400 audit(1561323434.328:8): avc:  denied  { module_request } for  pid=68007 comm="nc" kmod="net-pf-10-proto-132" scontext=system_u:system_r:container_t:s0:c519,c876 tcontext=system_u:system_r:kernel_t:s0 tclass=system permissive=0


So to enable SCTP, an administrator will need to unblacklist it and explicitly modprobe it. What is the right way for them to do that in RHCOS?

> So to enable SCTP, an administrator will need to unblacklist it and explicitly modprobe it. What is the right way for them to do that in RHCOS?

We will definitely need a KCS entry for this.  The general mechanism to manage RHCOS is MachineConfig:
https://github.com/openshift/machine-config-operator/#applying-configuration-changes-to-the-cluster

In this case it'd be a MachineConfig file that replaced /etc/modprobe.d/sctp-blacklist.conf with an empty file,
and also a systemd unit that runs `modprobe sctp` and is Before=kubelet.service.

Using release 4.2.0-0.nightly-2019-06-26-081712, I was able to confirm that the sctp module is available in the OS and can be loaded.

```
$ oc get clusterversion                                                                                           
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.2.0-0.nightly-2019-06-26-081712   True        False         3h15m   Cluster version is 4.2.0-0.nightly-2019-06-26-081712                                                                                

$ oc get node                                                                                                     
NAME                                         STATUS   ROLES    AGE     VERSION
ip-10-0-133-132.us-west-2.compute.internal   Ready    master   3h32m   v1.14.0+04ae0f405
ip-10-0-136-252.us-west-2.compute.internal   Ready    worker   3h24m   v1.14.0+04ae0f405
ip-10-0-150-76.us-west-2.compute.internal    Ready    master   3h32m   v1.14.0+04ae0f405
ip-10-0-151-188.us-west-2.compute.internal   Ready    worker   3h24m   v1.14.0+04ae0f405
ip-10-0-160-45.us-west-2.compute.internal    Ready    master   3h32m   v1.14.0+04ae0f405
ip-10-0-174-25.us-west-2.compute.internal    Ready    worker   3h24m   v1.14.0+04ae0f405

$ oc debug node/ip-10-0-136-252.us-west-2.compute.internal                                                        
Starting pod/ip-10-0-136-252us-west-2computeinternal-debug ...
To use host binaries, run `chroot /host`
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# rpm-ostree status
State: idle
AutomaticUpdates: disabled
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c4a8d13f7d441df335ee2b434887ab4ae0b3cf1bbd3ac29d70297779cd900e01                                                                                    
              CustomOrigin: Managed by pivot tool
                   Version: 420.8.20190625.0 (2019-06-25T20:28:05Z)

  pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dd3bd07c9a4fd39d8039b137fa161c3a5b38a2855e8df9f6c16946ccdf7b3f31                                                                                    
              CustomOrigin: Provisioned from oscontainer
                   Version: 420.8.20190624.0 (2019-06-24T00:25:32Z)

sh-4.4# rpm -q kernel-modules-extra
kernel-modules-extra-4.18.0-80.4.2.el8_0.x86_64
sh-4.4# rpm -ql kernel-modules-extra | grep sctp
/etc/modprobe.d/sctp-blacklist.conf
/etc/modprobe.d/sctp_diag-blacklist.conf
/lib/modules/4.18.0-80.4.2.el8_0.x86_64/kernel/net/sctp/sctp.ko.xz
/lib/modules/4.18.0-80.4.2.el8_0.x86_64/kernel/net/sctp/sctp_diag.ko.xz
sh-4.4# modprobe sctp
sh-4.4# lsmod sctp
Usage: lsmod
sh-4.4# lsmod | grep sctp
sctp                  385024  8
libcrc32c              16384  5 nf_conntrack,nf_nat,openvswitch,xfs,sctp
```

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922