Bug 1843949 - stop adding service-ca to token secret in 4.5
Summary: stop adding service-ca to token secret in 4.5
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Samples
Version: 4.6
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 4.6.0
Assignee: Gabe Montero
QA Contact: XiuJuan Wang
URL:
Whiteboard:
Depends On:
Blocks: 1843953
TreeView+ depends on / blocked
 
Reported: 2020-06-04 13:38 UTC by Gabe Montero
Modified: 2020-10-27 16:05 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Removed functionality
Doc Text:
The service-serving CA is no longer available in pods at /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt. This file has been deprecated since 4.1. Pods that currently consume the service-serving CA bundle from /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt must migrate to obtaining the CA bundle from a configMap annotated with service.beta.openshift.io/inject-cabundle=true. The change removes the OCP templates identified as using this removed functionality.
Clone Of: 1813894
: 1843953 (view as bug list)
Environment:
Last Closed: 2020-10-27 16:04:47 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 16:05:14 UTC

Description Gabe Montero 2020-06-04 13:38:59 UTC
+++ This bug was initially created as a clone of Bug #1813894 +++

with the goal of removing https://github.com/openshift/kubernetes/pull/116/commits/66d4751e4f866a9e51386eaac93bbdb3537f4813 in 4.6

1. find the initial deprecation notice in docs
2. have the value be off by default, with some ugly wiring (probably env var) to turn back on.
2.5. write a controller in the operator that removes the service-ca from all secrets.
3. create a new field in kcm.operator.openshift.io named `enableDeprecatedAndRemovedServiceCAKeyUntilNextRelease_ThisMakesClusterImpossibleToUpgrade`.  The name is abusive and clear.  People who set it should be very aware and not call us.
4. if the value is set, set the env var and mark the cluster upgradeable==false

In 4.6, we can remove the code entirely because no one can be relying on it.

--- Additional comment from Maru Newby on 2020-03-24 02:30:59 UTC ---

Corrected commit targeted for removal: https://github.com/openshift/kubernetes/commit/46562f3b5e34287b6ef79b92e54d9bee78ab735d

--- Additional comment from Maru Newby on 2020-03-24 06:00:15 UTC ---

Initial deprecation notice: https://github.com/openshift/openshift-docs/blob/enterprise-4.1/release_notes/ocp-4-1-release-notes.adoc#service-ca-bundle-changes

--- Additional comment from Stefan Schimanski on 2020-05-06 11:03:47 UTC ---



--- Additional comment from Maru Newby on 2020-05-07 22:46:46 UTC ---

PR to remove the code has been posted, and its merge should be deferred until 4.6: 

https://github.com/openshift/origin/pull/24393

--- Additional comment from Maru Newby on 2020-05-20 14:14:02 UTC ---

Still waiting on the updates to the following operators:

- openshift/cluster-kube-controller-manager-operator (submitted but blocked by persistent and unrelated test flake)
- openshift/cluster-samples-operator (coordinating with maintainers of jboss-container-images to get required upstream changes merged)

--- Additional comment from Maru Newby on 2020-05-29 21:22:06 UTC ---

The cluster-samples-operator fix is still in-progress, but testing the change is now possible.

The change can be tested by creating a pod in a 4.5 cluster and verifying the absence of the service serving CA in the pod filesystem:


/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt

--- Additional comment from Maru Newby on 2020-06-02 14:50:55 UTC ---

The openshift/library PR has merged: 

https://github.com/openshift/library/pull/219

Tomorrow (June 3rd), once a nightly job has made the necessary updates to the branch, cluster-samples-operator will be able to vendor the change. This vendoring change will need to merge to master and then backported to 4.5.

Comment 3 Gabe Montero 2020-06-04 21:41:39 UTC
for verification, simply instantiate one of the SSO templates Maru changed and confirm service-ca is not in the token secret

visit the changed files from the PR associated with this bz for that list

Comment 4 XiuJuan Wang 2020-06-10 09:27:12 UTC
Instantiate one of the SSO template

$oc new-project test
$oc new-app sso73-ocp4-x509-https

$for i in `oc get secret | awk '{print $1}'`; do oc get secret $i  -o json | jq '.data["service-ca.crt"]' -r  ; done
Error from server (NotFound): secrets "NAME" not found
null
null
null
null
null
null
null
null
null
null
null

$oc get secret
NAME                       TYPE                                  DATA   AGE
builder-dockercfg-s9cjw    kubernetes.io/dockercfg               1      21m
builder-token-6px6k        kubernetes.io/service-account-token   3      21m
builder-token-dj9pc        kubernetes.io/service-account-token   3      21m
default-dockercfg-gsbpq    kubernetes.io/dockercfg               1      21m
default-token-g7r9b        kubernetes.io/service-account-token   3      21m
default-token-tf4hs        kubernetes.io/service-account-token   3      21m
deployer-dockercfg-4xghh   kubernetes.io/dockercfg               1      21m
deployer-token-cz94g       kubernetes.io/service-account-token   3      21m
deployer-token-nq245       kubernetes.io/service-account-token   3      21m
sso-x509-https-secret      kubernetes.io/tls                     2      21m
sso-x509-jgroups-secret    kubernetes.io/tls                     2      21m
[wxj@dhcp-140-124 kubeconfig]$ oc get cm 
NAME             DATA   AGE
sso-service-ca   1      21m
[wxj@dhcp-140-124 kubeconfig]$ oc get cm  -o yaml 
apiVersion: v1
items:
- apiVersion: v1
  data:
    service-ca.crt: |
      -----BEGIN CERTIFICATE-----
      MIIDUTCCAjmgAwIBAgIIGB7oiru9erkwDQYJKoZIhvcNAQELBQAwNjE0MDIGA1UE
      Awwrb3BlbnNoaWZ0LXNlcnZpY2Utc2VydmluZy1zaWduZXJAMTU5MTc1NzI3NDAe
      Fw0yMDA2MTAwMjQ3NTNaFw0yMjA4MDkwMjQ3NTRaMDYxNDAyBgNVBAMMK29wZW5z
      aGlmdC1zZXJ2aWNlLXNlcnZpbmctc2lnbmVyQDE1OTE3NTcyNzQwggEiMA0GCSqG
      SIb3DQEBAQUAA4IBDwAwggEKAoIBAQDu0RQJaJWEZr0IZwaUIBTBxG4LGbD0WddY
      Ykon3fITh34kSeQzarz5V4VqQdxDjkMWepcDgaDNr4geOdCzKr0u9GNO/QGrGDs2
      b51ETg98ql4l+t2+pz8ZwQ+bCWv1oQTzouNbitB/UjT8dBN1px+K345xMRlqgqd1
      SBmmn+wQV7cXDqQH1SmIqSUMWz50QbfhYadSK2gQLVFr0KaQ4/CX5I2pO3baE2WN
      l8PPPk99YXZNLqHNprADHIq7LSkMb28r6upwhxQm7CgYpzWPS7XF/mW0KEiR9LbK
      LmD0aUXd8m/kyePAB7rDK3FZu6Cpn/luOJifyzp0nVIncZiuIwmhAgMBAAGjYzBh
      MA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRoNX0f
      wv3C3BYMlHyG41QT9rT4LDAfBgNVHSMEGDAWgBRoNX0fwv3C3BYMlHyG41QT9rT4
      LDANBgkqhkiG9w0BAQsFAAOCAQEA4sD89atblKn600RhUvx0VgoSA1o48Ve8wUU1
      uFVDot4kBOJ2fpUO33J+ZnBVsURWht/37WqwaaX0Lmj3NwEct24lEiM9YTCTbwIW
      Q/Cic2Zezv4ioos7B6TxlmtoB17liWPbiRd3Pk6mditxPboPBjiS33cW3HVQNnrT
      dTD8Zocf26l2Neq+f2devx/RPUgertB7a3HsD0VOiq2ZpY0tz115U21P+h4A/ehx
      rmccbTuCr9kRl/3ptoED0eUMYBHJriXSjkhVvo4NwmBWd157utPk3zLfMfTcikO7
      DdesAngJHf98j+0CG817SU3i9Keia5qm1zfm4b7sseDmzqsgRQ==
      -----END CERTIFICATE-----
  kind: ConfigMap
  metadata:
    annotations:
      description: ConfigMap providing service ca bundle.
      openshift.io/generated-by: OpenShiftNewApp
      service.beta.openshift.io/inject-cabundle: "true"
    creationTimestamp: "2020-06-10T09:01:47Z"
    labels:
      app: sso73-ocp4-x509-https
      app.kubernetes.io/component: sso73-ocp4-x509-https
      app.kubernetes.io/instance: sso73-ocp4-x509-https
      application: sso
      rhsso: 7.3.8.GA
      template: sso73-ocp4-x509-https
    managedFields:
    - apiVersion: v1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:description: {}
            f:openshift.io/generated-by: {}
            f:service.beta.openshift.io/inject-cabundle: {}
          f:labels:
            .: {}
            f:app: {}
            f:app.kubernetes.io/component: {}
            f:app.kubernetes.io/instance: {}
            f:application: {}
            f:rhsso: {}
            f:template: {}
      manager: oc
      operation: Update
      time: "2020-06-10T09:01:47Z"
    - apiVersion: v1
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          .: {}
          f:service-ca.crt: {}
      manager: service-ca-operator
      operation: Update
      time: "2020-06-10T09:01:47Z"
    name: sso-service-ca
    namespace: bug
    resourceVersion: "143484"
    selfLink: /api/v1/namespaces/bug/configmaps/sso-service-ca
    uid: 866cefb4-ef33-4774-8c4b-aba0aaddadd6
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

oc get pods
 NAME           READY   STATUS      RESTARTS   AGE
sso-1-deploy   0/1     Completed   0          23m
sso-1-rw6zg    1/1     Running     0          23m
[wxj@dhcp-140-124 kubeconfig]$  oc rsh sso-1-rw6zg
sh-4.2$ ls /var/run/secrets/kubernetes.io/serviceaccount/
ca.crt	namespace  token

Mark this bug verified in 4.6.0-0.nightly-2020-06-09-234748

Comment 6 errata-xmlrpc 2020-10-27 16:04:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196


Note You need to log in before you can comment on or make changes to this bug.