Bug 2047676 - breaking API change in dual-stack Services in 1.23 / 4.10
Summary: breaking API change in dual-stack Services in 1.23 / 4.10
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.9
Hardware: Unspecified
OS: Unspecified
urgent
low
Target Milestone: ---
: 4.9.z
Assignee: Andreas Karis
QA Contact: Anurag saxena
URL:
Whiteboard:
Depends On: 2045576
Blocks: 2047687
TreeView+ depends on / blocked
 
Reported: 2022-01-28 09:37 UTC by Andreas Karis
Modified: 2022-03-17 17:21 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Deprecated Functionality
Doc Text:
Cause A rewrite in the kube-apiserver causes a user facing API change. Starting in kube-apiserver 1.23 / OpenShift 4.10, users MUST explicitly specify either "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack" for DualStack services to be valid. Consequence Users cannot create DualStack services (services with either 2 different ipFamilies or 2 different clusterIPs) without explicitly specifying "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack". Fix Red Hat OpenShift Container Platform 4.8 and 4.9 now emit a warning that warns of the imminent API change. Result Users cannot create DualStack services without explicitly specifying "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack" starting with Red Hat OpenShift Container Platform 4.10. However, users will be warned in previous versions of Red Hat OpenShift Container Platform about the upcoming API change.
Clone Of: 2045576
: 2047687 (view as bug list)
Environment:
Last Closed: 2022-01-28 15:25:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Andreas Karis 2022-01-28 09:37:52 UTC
+++ This bug was initially created as a clone of Bug #2045576 +++

kube 1.23 introduced a breaking API change in dual-stack services which I'm just noticing now...

In kube 1.21 and 1.22 (OCP 4.8 and 4.9), the apiserver would default the value of `ipFamilyPolicy` to `RequireDualStack` if you created a Service with two `ipFamilies` or two `clusterIPs` but no explicitly-specified `ipFamilyPolicy`:

    kind: Service
    apiVersion: v1
    metadata:
      name: my-service
    spec:
      type: ClusterIP
      ipFamilies:
        - IPv6
        - IPv4
      ports:
      - port: 80
      selector:
        foo: bar

or

    kind: Service
    apiVersion: v1
    metadata:
      name: my-service
    spec:
      type: ClusterIP
      clusterIPs:
        - 172.30.0.99
        - fd02::9999
      ports:
      - port: 80
      selector:
        foo: bar

This turned out to have some tricky and possibly unfixable broken edge cases, so in 1.23 / 4.10, you MUST explicitly specify either "ipFamilyPolicy: PreferDualStack" or "ipFamilyPolicy: RequireDualStack" for the service to be valid.

(This was fallout from a MASSIVE rewrite of the apiserver Service-handling code, https://github.com/kubernetes/kubernetes/pull/96684.)


It is hard to say whether any users are actually creating services in this way. Although this behavior was described in the KEP, it never appeared in the official documentation, which always implied that you had to explicitly provide an `ipFamilyPolicy` value (https://github.com/kubernetes/website/blob/release-1.22/content/en/docs/concepts/services-networking/dual-stack.md#services). (It doesn't actually say "you MUST specify ipFamilyPolicy", but it never suggests that it's possible to omit it, and doesn't describe what would happen if you did.)


If we are really concerned about this as an API break, then we could add a mutating web hook to fix things, but presumably we'd have to maintain it forever.

A simpler fix might be to just modify 4.8 and 4.9 to warn loudly if the user tries to create such a service?

We also need to release-note this, and should explicitly mention it to known large dual-stack-using customers.


Note You need to log in before you can comment on or make changes to this bug.