Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1699321

Summary: controller-manager/scheduler secure ports served by temporary self signed certificates
Product: OpenShift Container Platform Reporter: Sergiusz Urbaniak <surbania>
Component: MasterAssignee: Stefan Schimanski <sttts>
Status: CLOSED ERRATA QA Contact: zhou ying <yinzhou>
Severity: urgent Docs Contact:
Priority: high    
Version: 4.1.0CC: aos-bugs, jokerman, mfojtik, mmccomas, xxia
Target Milestone: ---Keywords: BetaBlocker, NeedsTestCase
Target Release: 4.1.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:47:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sergiusz Urbaniak 2019-04-12 12:16:22 UTC
Currently, monitoring must skip TLS verification to scrape controller manager/scheduler secure endpoints (10257/10259) because they are served using temporary self signed installer certificates.

Example certs served from the kube scheduler endpoint:

$ kubectl \
  -n openshift-kube-scheduler \
  port-forward \
  pod/openshift-kube-scheduler-ip-10-0-135-193.ec2.internal 10259

$ openssl crl2pkcs7 -nocrl -certfile \
  <(echo 'QUIT' | openssl s_client -showcerts -connect localhost:10259) \
  | openssl pkcs7 -print_certs -text -noout

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 2 (0x2)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=localhost-ca@1555061666
        Validity
            Not Before: Apr 12 08:34:26 2019 GMT
            Not After : Apr 11 08:34:26 2020 GMT
        Subject: CN=localhost@1555061667
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Alternative Name: 
                DNS:localhost, DNS:localhost, IP Address:127.0.0.1
    Signature Algorithm: sha256WithRSAEncryption
...

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=localhost-ca@1555061666
        Validity
            Not Before: Apr 12 08:34:26 2019 GMT
            Not After : Apr 11 08:34:26 2020 GMT
        Subject: CN=localhost-ca@1555061666
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment, Certificate Sign
            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
...

Comment 2 Sergiusz Urbaniak 2019-04-12 13:43:17 UTC
@michael: what about the scheduler operator?

Comment 3 Michal Fojtik 2019-04-12 13:48:45 UTC
(In reply to Sergiusz Urbaniak from comment #2)
> @michael: what about the scheduler operator?

Stefan is going to port it to scheduler now.

Comment 4 Stefan Schimanski 2019-04-15 09:36:29 UTC
This also depends on https://github.com/openshift/installer/pull/1576/ to work with metrics.

Comment 5 Stefan Schimanski 2019-04-15 09:37:11 UTC
https://github.com/openshift/cluster-kube-scheduler-operator/pull/97 is the scheduler PR to wire the right serving certs.

Comment 6 Michal Fojtik 2019-04-16 09:10:26 UTC
Both PR's merged and we should be now serving securely.

As a follow up, monitoring team should switch their scraping to use the secure ports, but that should be tracked outside this BZ.

Comment 8 zhou ying 2019-04-19 05:11:20 UTC
Confirmed with latest ocp 4.1 , the issue has fixed:
Payload: 4.1.0-0.nightly-2019-04-18-210657

kube-controller-manager certinfo:
[zhouying@dhcp-140-138 tmp]$ openssl x509 -in contls2.crt --text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 6244008379750077698 (0x56a72c8a0990d502)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = openshift-service-serving-signer@1555643835
        Validity
            Not Before: Apr 19 03:17:38 2019 GMT
            Not After : Apr 18 03:17:39 2021 GMT
        Subject: CN = kube-controller-manager.openshift-kube-controller-manager.svc
....

kube-scheduler cert info:
[zhouying@dhcp-140-138 tmp]$ openssl x509 -in schedulerctls.crt --text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 3938385304973640660 (0x36a7f491839283d4)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = openshift-service-serving-signer@1555643835
        Validity
            Not Before: Apr 19 03:17:37 2019 GMT
            Not After : Apr 18 03:17:38 2021 GMT
        Subject: CN = scheduler.openshift-kube-scheduler.svc

Comment 10 errata-xmlrpc 2019-06-04 10:47:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758