Bug 1801415

Summary: ingress-to-route controller uses deprecated extensions/v1beta1 API
Product: OpenShift Container Platform Reporter: Miciah Dashiel Butler Masters <mmasters>
Component: NetworkingAssignee: Dan Mace <dmace>
Networking sub component: router QA Contact: Arvind iyengar <aiyengar>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: aiyengar, aos-bugs
Version: 4.4   
Target Milestone: ---   
Target Release: 4.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: The ingress-to-route controller used the ingresses resource from the extensions/v1beta1 API group. Using the ingresses resource from this API group was deprecated in Kubernetes 1.18. Consequence: The ingress-to-route controller was using a deprecated API. Fix: The ingress-to-route controller was updated to use the ingresses resource from the networking.k8s.io/v1beta1 API group. Result: The ingress-to-route controller no longer uses the deprecated ingress API.
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-07-13 17:14:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Miciah Dashiel Butler Masters 2020-02-10 20:15:27 UTC
Description of problem:

The ingress-to-route controller uses the ingress API from the extensions/v1beta1 API group[1]; using ingress from this API group is deprecated in Kubernetes 1.18 in favor of using ingress from the networking.k8s.io/v1beta API group[2].

1. https://github.com/openshift/openshift-controller-manager/blob/8417a9a2d6bac7d7ae43c463d7b52c8aad2b0dbe/pkg/route/ingress/ingress.go#L12
2. https://github.com/kubernetes/kubernetes/pull/74057


Additional info:

The upstream ingress-nginx project has already performed a similar migration: https://github.com/kubernetes/ingress-nginx/pull/4127/commits/84102eec2ba270f624c57023aab59aab4471178e "Migrate to new networking.k8s.io/v1beta1 package".

Comment 1 Dan Mace 2020-02-11 19:35:44 UTC
Pretty sure we can defer this to 4.5 (Kube 1.18). Please correct me if I'm wrong.

Comment 4 Arvind iyengar 2020-05-08 12:27:52 UTC
The fix after merge originally made into "4.5.0-0.nightly-2020-04-29-223453" release version. At the time of writing, the functionality has been verified in "4.5.0-0.nightly-2020-05-06-003431" release:
----
Server Version: 4.5.0-0.nightly-2020-05-06-003431
Kubernetes Version: v1.18.0-rc.1
----

we note that the openshift-controller-manager uses the new "networking.k8s.io" API group and there are no requests with the "openshift-controller-manager-sa" service account for the "ingresses" resource with the "extensions/v1beta1" API group.

Excerpts from extracted audit logs:
-------
$ zcat  must-gather.local.2085467479204652461/quay-io-openshift-release-dev-ocp-v4-0-art-dev-sha256-53cc66fe93fcee37285748f191975eccc56ac244f225613432e1a5d50c67d940/audit_logs/kube-apiserver/ip-10-0-1* | grep -i "openshift-controller-manager-sa" | grep -i "extensions.v1beta1" | jq .
$

$ zcat  must-gather.local.2085467479204652461/quay-io-openshift-release-dev-ocp-v4-0-art-dev-sha256-53cc66fe93fcee37285748f191975eccc56ac244f225613432e1a5d50c67d940/audit_logs/kube-apiserver/ip-10-0-1* | grep -i "openshift-controller-manager-sa" | grep -i "networking.k8s.io" | jq .
                            
    {
      "kind": "Event",
      "apiVersion": "audit.k8s.io/v1",
      "level": "Metadata",
      "auditID": "f80bd200-0d9e-403a-b1d3-96498e1739ed",
      "stage": "ResponseStarted",
      "requestURI": "/apis/networking.k8s.io/v1beta1/ingresses?allowWatchBookmarks=true&resourceVersion=14761&timeout=6m26s&timeoutSeconds=386&watch=true",
      "verb": "watch",
      "user": {
        "username": "system:serviceaccount:openshift-controller-manager:openshift-controller-manager-sa",
        "uid": "1f41728b-2a60-4919-96b8-282a93011290",
        "groups": [
          "system:serviceaccounts",
          "system:serviceaccounts:openshift-controller-manager",
          "system:authenticated"
        ]
      },
      "sourceIPs": [
        "10.0.147.111"
      ],
      "userAgent": "openshift-controller-manager/v0.0.0 (linux/amd64) kubernetes/$Format",
      "objectRef": {
        "resource": "ingresses",
        "apiGroup": "networking.k8s.io",
        "apiVersion": "v1beta1"
      },
      "responseStatus": {
        "metadata": {},
        "status": "Success",
        "message": "Connection closed early",
        "code": 200
      },
      "requestReceivedTimestamp": "2020-05-07T06:00:43.629867Z",
      "stageTimestamp": "2020-05-07T06:00:43.660092Z",
      "annotations": {
        "authorization.k8s.io/decision": "allow",
        "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"system:openshift:openshift-controller-manager\" of ClusterRole \"system:openshift:openshift-controller-manager\" to ServiceAccount \"openshift-controller-manager-sa/openshift-controller-manager\""
      }
    }
    {
      "kind": "Event",
      "apiVersion": "audit.k8s.io/v1",
      "level": "Metadata",
      "auditID": "f80bd200-0d9e-403a-b1d3-96498e1739ed",
      "stage": "ResponseComplete",
      "requestURI": "/apis/networking.k8s.io/v1beta1/ingresses?allowWatchBookmarks=true&resourceVersion=14761&timeout=6m26s&timeoutSeconds=386&watch=true",
      "verb": "watch",
      "user": {
        "username": "system:serviceaccount:openshift-controller-manager:openshift-controller-manager-sa",
        "uid": "1f41728b-2a60-4919-96b8-282a93011290",
        "groups": [
          "system:serviceaccounts",
          "system:serviceaccounts:openshift-controller-manager",
          "system:authenticated"
        ]
      },
      "sourceIPs": [
        "10.0.147.111"
      ],
      "userAgent": "openshift-controller-manager/v0.0.0 (linux/amd64) kubernetes/$Format",
      "objectRef": {
        "resource": "ingresses",
        "apiGroup": "networking.k8s.io",
        "apiVersion": "v1beta1"
      },
      "responseStatus": {
        "metadata": {},
        "status": "Success",
        "message": "Connection closed early",
        "code": 200
      },
      "requestReceivedTimestamp": "2020-05-07T06:00:43.629867Z",
      "stageTimestamp": "2020-05-07T06:00:43.660157Z",
      "annotations": {
        "authorization.k8s.io/decision": "allow",
        "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"system:openshift:openshift-controller-manager\" of ClusterRole \"system:openshift:openshift-controller-manager\" to ServiceAccount \"openshift-controller-manager-sa/openshift-controller-manager\""
      }
    }
-------

Comment 6 errata-xmlrpc 2020-07-13 17:14:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409