Bug 2048645 - Placeholder: breaking API change in dual-stack Services in 1.23 / 4.10
Summary: Placeholder: breaking API change in dual-stack Services in 1.23 / 4.10
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.10.0
Assignee: Andreas Karis
QA Contact: Anurag saxena
URL:
Whiteboard:
Depends On:
Blocks: 2045576 2050632
TreeView+ depends on / blocked
 
Reported: 2022-01-31 16:37 UTC by Andreas Karis
Modified: 2022-03-17 17:20 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Cause A rewrite in the kube-apiserver causes a user facing API change. Starting in kube-apiserver 1.23 / OpenShift 4.10, users MUST explicitly specify either "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack" for DualStack services to be valid. Consequence Users cannot create DualStack services (services with either 2 different ipFamilies or 2 different clusterIPs) without explicitly specifying "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack". Fix Red Hat OpenShift Container Platform 4.8 and 4.9 now emit a warning that warns of the imminent API change. Result Users cannot create DualStack services without explicitly specifying "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack" starting with Red Hat OpenShift Container Platform 4.10. However, users will be warned in previous versions of Red Hat OpenShift Container Platform about the upcoming API change.
Clone Of:
Environment:
Last Closed: 2022-01-31 16:38:08 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Andreas Karis 2022-01-31 16:37:15 UTC
this is the 4.n+1 bug placeholder for https://bugzilla.redhat.com/show_bug.cgi?id=2045576

We do not expect a fix for 4.10 as 2045576 is a 4.9 and 4.8, downstream fix only, hence I am directly going to mark this here as closed currentrelease

=================================

This bug was initially created as a copy of Bug #2045576

I am copying this bug because: 



kube 1.23 introduced a breaking API change in dual-stack services which I'm just noticing now...

In kube 1.21 and 1.22 (OCP 4.8 and 4.9), the apiserver would default the value of `ipFamilyPolicy` to `RequireDualStack` if you created a Service with two `ipFamilies` or two `clusterIPs` but no explicitly-specified `ipFamilyPolicy`:

    kind: Service
    apiVersion: v1
    metadata:
      name: my-service
    spec:
      type: ClusterIP
      ipFamilies:
        - IPv6
        - IPv4
      ports:
      - port: 80
      selector:
        foo: bar

or

    kind: Service
    apiVersion: v1
    metadata:
      name: my-service
    spec:
      type: ClusterIP
      clusterIPs:
        - 172.30.0.99
        - fd02::9999
      ports:
      - port: 80
      selector:
        foo: bar

This turned out to have some tricky and possibly unfixable broken edge cases, so in 1.23 / 4.10, you MUST explicitly specify either "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack" for the service to be valid.

(This was fallout from a MASSIVE rewrite of the apiserver Service-handling code, https://github.com/kubernetes/kubernetes/pull/96684.)


It is hard to say whether any users are actually creating services in this way. Although this behavior was described in the KEP, it never appeared in the official documentation, which always implied that you had to explicitly provide an `ipFamilyPolicy` value (https://github.com/kubernetes/website/blob/release-1.22/content/en/docs/concepts/services-networking/dual-stack.md#services). (It doesn't actually say "you MUST specify ipFamilyPolicy", but it never suggests that it's possible to omit it, and doesn't describe what would happen if you did.)


If we are really concerned about this as an API break, then we could add a mutating web hook to fix things, but presumably we'd have to maintain it forever.

A simpler fix might be to just modify 4.8 and 4.9 to warn loudly if the user tries to create such a service?

We also need to release-note this, and should explicitly mention it to known large dual-stack-using customers.


Note You need to log in before you can comment on or make changes to this bug.