Bug 1717461

Summary: Unsupported value: "SCTP" when deploying service
Product: OpenShift Container Platform Reporter: Weibin Liang <weliang>
Component: NetworkingAssignee: Casey Callendrello <cdc>
Networking sub component: openshift-sdn QA Contact: Weibin Liang <weliang>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: aos-bugs, augol, danw, eparis, fsimonce, nhorman, william.caban
Version: 4.1.0   
Target Milestone: ---   
Target Release: 4.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-01-23 11:04:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1769469, 1769474, 1774382, 1779790, 1779794    
Bug Blocks: 1771572    

Description Weibin Liang 2019-06-05 14:23:36 UTC
Description of problem:
According to https://github.com/openshift/origin/pull/22883, Kubernetes 1.12 adds beta support for SCTP. PR 22883 add this support in openshift. When deploying the service using protocol SCTP, deploying fail.

Version-Release number of selected component (if applicable):
node.openshift.io/os_version=4.1
Kernel Version:                          4.18.0-80.1.2.el8_0.x86_64
Container Runtime Version:               cri-o://1.13.9-1.rhaos4.1.gitd70609a.el8
Kubelet Version:                         v1.13.4+cb455d664
Kube-Proxy Version:                      v1.13.4+cb455d664


How reproducible:
Always

Steps to Reproduce:
1. create service yaml file to use SCTP protocol
2. oc create -f svc.yaml 

Actual results:
[root@dhcp-41-193 Network]# oc create -f svc.yaml 
The Service "red-service" is invalid: spec.ports[0].protocol: Unsupported value: "SCTP": supported values: "TCP", "UDP"

Expected results:
SCTP should be supported in Kubernetes 1.13.4

Additional info:
[root@dhcp-41-193 Network]# oc describe node ip-10-0-143-200.us-east-2.compute.internal | grep -i vers
node.openshift.io/os_version=4.1
 Kernel Version:                          4.18.0-80.1.2.el8_0.x86_64
 Container Runtime Version:               cri-o://1.13.9-1.rhaos4.1.gitd70609a.el8
 Kubelet Version:                         v1.13.4+cb455d664
 Kube-Proxy Version:                      v1.13.4+cb455d664

Comment 1 zhaozhanqi 2019-06-06 03:01:14 UTC
hi, Weibin

 this feature should be 4.2 feature. it should NOT be supported in 4.1 since the above PR did not merged to `release-4.1` branch.

Comment 2 Dan Winship 2019-06-10 15:28:15 UTC
I suppose you need to enable the SCTPSupport feature gate. I'm not sure how you do that in 4.x

Comment 3 Weibin Liang 2019-06-10 16:03:28 UTC
Should https://github.com/openshift/origin/pull/22883 enable the SCTPSupport feature gate?

Comment 4 Casey Callendrello 2019-07-15 20:08:42 UTC
So, now that Dan has gone off on his year walk to determine just what the implication of the feature gate is, I'm pretty sure it's safe to enable.

So, for 4.2, let's add the SCTP feature gate. I'll try and squeeze this one in.

Comment 5 Dan Winship 2019-07-15 20:40:03 UTC
FTR: https://groups.google.com/d/msg/kubernetes-sig-network/KSi0DI-Gw80/fNpWFnAdCAAJ

OCP is protected both by SELinux and by having sctp blacklisted by default.

Comment 6 Casey Callendrello 2019-08-13 09:52:03 UTC
This fell off the radar for 4.2. We can do it for 4.3.

Comment 7 Casey Callendrello 2019-11-06 15:07:21 UTC
https://github.com/openshift/api/pull/504

Comment 8 Eric Paris 2019-11-21 15:32:38 UTC
The API object should be allowed, by default, in 4.3 nightlies.

Comment 10 Weibin Liang 2019-11-26 15:49:54 UTC
Still wait for https://bugzilla.redhat.com/show_bug.cgi?id=1769469 to be verified first

Comment 11 Weibin Liang 2019-12-17 16:29:14 UTC
Fellow the steps in https://github.com/williamcaban/ocp4-sctp and test in 4.3.0-0.nightly-2019-12-13-180405, the verification failed:
[root@dhcp-41-193 FILE]# oc rsh sctpserver-55f7d4fc78-sklxb
sh-5.0# checksctp
checksctp: Protocol not supported


@fpaoline, I still need your workaround to make SCTP works:
oc create -f https://raw.githubusercontent.com/fedepaol/sctp_setup/master/02-enable_sctp_module.yaml
oc create -f https://raw.githubusercontent.com/fedepaol/sctp_setup/master/01-sctp_policy_ds.yaml
[root@dhcp-41-193 FILE]# oc rsh sctpserver-55f7d4fc78-xm5hf
sh-5.0# checksctp
SCTP supported

Comment 12 Dan Winship 2019-12-17 17:05:20 UTC
SCTP is not supposed to be enabled *by default*, it's just expected that it can be enabled *at all* now. The expected behavior is:

  1. You can run "modprobe sctp" as root on a node (not in a container), and it will work
     (as opposed to erroring out and printing "modprobe: FATAL: Module sctp not found in
     directory ...")

     For actual end users, I think we expect them to do this by doing something like
     creating a MachineConfig object to create a systemd unit file that just runs
     "modprobe sctp". I don't think we've documented any particular approach.

     You do not need to override the sctp-blacklist.conf file (and should not do so).

  2. After loading the sctp module, containers can create sctp sockets

     I believe the policy in 01-sctp_policy_ds.yaml is to work around bug 1769469.
     This is supposed to already be fixed in current 4.3 builds and so that
     workaround should not be needed.

Comment 13 Eric Paris 2019-12-17 17:11:48 UTC
This is fixed when SCTP works after the kernel module is loaded. Either explicitly as winship indicates or implicitly after (manually) unblacklisting it is acceptable. But it requires the human to do $something to enable SCTP. it should not work 'out of the box'

getting the module loaded should be the only thing the human needs to do.

Comment 14 Dan Winship 2019-12-17 17:31:05 UTC
Unblacklisting the module is *not* sufficient to make things work, because our selinux policy blocks containers from causing kernel modules to be autoloaded anyway (which is good because, eg, there is no reason for containers to be causing bluetooth.ko or appletalk.ko or whatever to be autoloaded either).

You could do "unblacklist the module and then run an sctp client or server not in a container to cause sctp to be autoloaded", but that's not really any simpler than "manually modprobe the module".

Comment 15 Weibin Liang 2019-12-17 20:02:12 UTC
Run "modprobe sctp" in each node first, then verification passed on 4.3.0-0.nightly-2019-12-13-180405


[root@dhcp-41-193 FILE]# oc rsh pod/sctpserver-55f7d4fc78-lvcdn
sh-5.0# checksctp
SCTP supported
sh-5.0# sctp_test -H localhost -P 30100 -l
local:addr=::, port=rwp, family=10
seed = 1576612563

Starting tests...
	socket(SOCK_SEQPACKET, IPPROTO_SCTP)  ->  sk=3
	bind(sk=3, [a:::,p:rwp])  --  attempt 1/10
	listen(sk=3,backlog=100)
Server: Receiving packets.
	recvmsg(sk=3) 

[root@dhcp-41-193 FILE]# oc rsh pod/sctpserver-55f7d4fc78-lvcdn
sh-5.0# netstat --sctp -atplun
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
sctp                :::30100                                        LISTEN      12/sctp_test        
sh-5.0# exit
exit
[root@dhcp-41-193 FILE]# oc create -f https://raw.githubusercontent.com/weliang1/Openshift_Networking/master/Features/SCTP/sctp-app-svc.yaml
service/sctpserver created
[root@dhcp-41-193 FILE]# oc get svc
NAME         TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)            AGE
sctpserver   NodePort   172.30.167.248   <none>        30100:30100/SCTP   3s

Comment 17 errata-xmlrpc 2020-01-23 11:04:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062