Bug 2050632

Summary: [kube-apiserver] emit warning for breaking API change in dual-stack Services in 1.23 / 4.10
Product: OpenShift Container Platform Reporter: Andreas Karis <akaris>
Component: kube-apiserverAssignee: Andreas Karis <akaris>
Status: CLOSED ERRATA QA Contact: jmekkatt
Severity: low Docs Contact:
Priority: medium    
Version: 4.9CC: akaris, anusaxen, aos-bugs, danw, jboxman, mfojtik, sttts, trozet, wking, xxia
Target Milestone: ---   
Target Release: 4.9.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Cause A rewrite in the kube-apiserver causes a user facing API change. Starting in kube-apiserver 1.23 / OpenShift 4.10, users MUST explicitly specify either "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack" for DualStack services to be valid. Consequence Users cannot create DualStack services (services with either 2 different ipFamilies or 2 different clusterIPs) without explicitly specifying "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack". Fix Red Hat OpenShift Container Platform 4.8 and 4.9 now emit a warning that warns of the imminent API change. Result Users cannot create DualStack services without explicitly specifying "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack" starting with Red Hat OpenShift Container Platform 4.10. However, users will be warned in previous versions of Red Hat OpenShift Container Platform about the upcoming API change.
Story Points: ---
Clone Of: 2045576 Environment:
Last Closed: 2022-05-03 07:35:33 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2048645    
Bug Blocks:    

Comment 5 jmekkatt 2022-04-19 12:37:46 UTC
Verification steps:
 
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.9.0-0.nightly-2022-04-18-075504   True        False         4h43m   Cluster version is 4.9.0-0.nightly-2022-04-18-075504
 
 
Deploy a cluster with IPv4/IPv6 dualstack.
Create inline services.
$ cat <<'EOF' | oc apply -f -
 apiVersion: v1
 kind: Service
 metadata:
   name: dualstack-ipfamilies-no-ipfamilypolicy
 spec:
   type: ClusterIP
   selector:
     app: test
   ipFamilies:
      - IPv6
      - IPv4
   ports:
     - port: 80
       targetPort: 80
 EOF
W0419 17:46:28.379686   54500 warnings.go:70] service.spec.ipFamilyPolicy must be RequireDualStack or PreferDualStack when multiple 'ipFamilies' are specified, this operation will fail starting with Red Hat OpenShift Platform 4.10.
service/dualstack-ipfamilies-no-ipfamilypolicy created
 
$ cat <<'EOF' | oc apply -f -
 apiVersion: v1
 kind: Service
 metadata:
   name: dualstack-ipfamilies-ipfamilypolicy-requiredualstack
 spec:
   type: ClusterIP
   selector:
     app: test
   ipFamilyPolicy: RequireDualStack
   ipFamilies:
      - IPv6
      - IPv4
   ports:
     - port: 80
       targetPort: 80
 EOF
service/dualstack-ipfamilies-ipfamilypolicy-requiredualstack created
 
$ oc get svc | grep dualstack
dualstack-ipfamilies-ipfamilypolicy-requiredualstack   ClusterIP      fd02::7077     <none>                                 80/TCP    5s
dualstack-ipfamilies-no-ipfamilypolicy                 ClusterIP      fd02::39c9     <none>                                 80/TCP    40s
 
 
$ oc get events | grep dualstack
3m7s        Warning   ImplicitDualStackIpFamilyPolicy   service/dualstack-ipfamilies-no-ipfamilypolicy   DualStack ipFamilyPolicy set implicitly for service default/dualstack-ipfamilies-no-ipfamilypolicy
 
Results : Only event for dualstack-ipfamilies-no-ipfamilypolicy and no event for dualstack-ipfamilies-ipfamilypolicy-requiredualstack and works as expected.
 
 
Repeat with clusterIPs - make sure to choose correct IP addresses from within the cluster's IPv4 and IPv6 service range
Create inline services.
$ cat <<'EOF' | oc apply -f -
 apiVersion: v1
 kind: Service
 metadata:
   name: dualstack-clusterips-ipfamilypolicy-dualstack
 spec:
   type: ClusterIP
   selector:
     app: test
   ipFamilyPolicy: PreferDualStack
   clusterIP: fd02::dcf9
   clusterIPs:
   - fd02::dcf9
   - 172.30.0.3
   ports:
     - port: 80
       targetPort: 80
 EOF
service/dualstack-clusterips-ipfamilypolicy-dualstack created
$ cat <<'EOF' | oc apply -f -
 apiVersion: v1
 kind: Service
 metadata:
   name: dualstack-clusterips-no-ipfamilypolicy
 spec:
   type: ClusterIP
   selector:
     app: test
   clusterIP: fd02::cb76
   clusterIPs:
   - fd02::cb76
   - 172.30.0.4
   ports:
     - port: 80
       targetPort: 80
 EOF
 
W0419 17:22:59.301830   52327 warnings.go:70] service.spec.ipFamilyPolicy must be RequireDualStack or PreferDualStack when multiple 'ipFamilies' are specified, this operation will fail starting with Red Hat OpenShift Platform 4.10.
service/dualstack-clusterips-no-ipfamilypolicy created
 
$ oc get services | grep dualstack
NAME                                            TYPE           CLUSTER-IP   EXTERNAL-IP                            PORT(S)   AGE
dualstack-clusterips-ipfamilypolicy-dualstack   ClusterIP      fd02::dcf9   <none>                                 80/TCP    3m18s
dualstack-clusterips-no-ipfamilypolicy          ClusterIP      fd02::cb76   <none>                                 80/TCP    48s
 
 
$ oc get events | grep dualstack
2m37s       Warning   ImplicitDualStackIpFamilyPolicy   service/dualstack-clusterips-no-ipfamilypolicy            DualStack ipFamilyPolicy set implicitly for service default/dualstack-clusterips-no-ipfamilypolicy
 
Results : Only events for dualstack-clusterips-no-ipfamilypolicy and no event for dualstack-clusterips-ipfamilypolicy-dualstack, working as expected.
 
Create inline services with SingleStack :
$ cat <<'EOF' | oc apply -f -
 apiVersion: v1
 kind: Service
 metadata:
   name: singlestack-clusterip-ipfamilypolicy-singlestack
 spec:
   type: ClusterIP
   selector:
     app: test
   ipFamilyPolicy: SingleStack
   clusterIP: fd02::dcf9
   ports:
     - port: 80
       targetPort: 80
 EOF
service/singlestack-clusterip-ipfamilypolicy-singlestack created
 
$ cat <<'EOF' | oc apply -f -
 apiVersion: v1
 kind: Service
 metadata:
   name: singlestack-ipfamilypolicy-singlestack
 spec:
   type: ClusterIP
   selector:
     app: test
   ipFamilyPolicy: SingleStack
   ports:
     - port: 80
 EOF
service/singlestack-ipfamilypolicy-singlestack created
 
cat <<'EOF' | oc apply -f -
 apiVersion: v1
 kind: Service
 metadata:
   name: singlestack-clusterip-noipfamilypolicy
 spec:
   type: ClusterIP
   selector:
     app: test
   clusterIP: fd02::cb76
   ports:
     - port: 80
       targetPort: 80
 EOF
service/singlestack-clusterip-noipfamilypolicy created
 
 
$ oc get svc | grep singlestack
singlestack-clusterip-ipfamilypolicy-singlestack   ClusterIP      fd02::dcf9     <none>                                 80/TCP    12m
singlestack-clusterip-noipfamilypolicy             ClusterIP      fd02::cb76     <none>                                 80/TCP    69s
singlestack-ipfamilypolicy-singlestack             ClusterIP      172.30.32.60   <none>                                 80/TCP    2m7s
 
 
$ oc get events | grep singlestack
 
Results :  No events should be for 3 singlestack services and works as expected.
 
Hence all checks works as per expectation, moved bug to verified.

Comment 10 errata-xmlrpc 2022-05-03 07:35:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.9.31 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1605