Bug 1857025 - Creating faulty(bad formatted cert&key) route makes other existing routes inaccessible
Summary: Creating faulty(bad formatted cert&key) route makes other existing routes ina...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.11.0
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
: 3.11.z
Assignee: Miciah Dashiel Butler Masters
QA Contact: Arvind iyengar
URL:
Whiteboard:
Depends On: 1843856
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-15 00:45 UTC by Miciah Dashiel Butler Masters
Modified: 2022-08-04 22:30 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1857021
Environment:
Last Closed: 2020-09-16 07:46:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift origin pull 25281 0 None closed Bug 1857025: Sanitize TLS config that has key bundled with cert 2021-02-15 22:03:00 UTC
Red Hat Product Errata RHBA-2020:3695 0 None None None 2020-09-16 07:47:04 UTC

Comment 1 Miciah Dashiel Butler Masters 2020-07-30 08:27:53 UTC
We'll track this 3.11 backport in the upcoming sprint.

Comment 2 Daneyon Hansen 2020-08-13 17:00:05 UTC
https://github.com/openshift/origin/pull/25281 needs a /approve.

Comment 3 Miciah Dashiel Butler Masters 2020-08-21 05:08:07 UTC
The proposed fix is merged but not yet verified; we'll continue tracking it this upcoming sprint.

Comment 4 Arvind iyengar 2020-08-26 08:53:28 UTC
Tested this in "v3.11.272" which is the latest available build version. It is noted that unlike the 4.x environment, the router still seem to hit the same error when a badly formatted certificate/pem file is loaded via the edge route creation: 
-------
# oc version
oc v3.11.268
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://ip-172-18-5-96.ec2.internal:8443
openshift v3.11.272
kubernetes v1.11.0+d4cacc0

# oc create route edge myroute --port=http --service=service-unsecure --cert=badcerts/badcert.pem

# oc get route myroute -o yaml
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  annotations:
    openshift.io/host.generated: "true"
  creationTimestamp: "2020-08-26T07:28:29Z"
  labels:
    name: service-unsecure
  name: myroute
  namespace: test-1
spec:
  host: myroute-test-1.apps.0826-ckh.qe.rhcloud.com
  port:
    targetPort: http
  tls:
    certificate: |+
      -----BEGIN CERTIFICATE-----

      MIIGnTCCBYWgAwIBAgITMAABqrxgky5s36Pm5QAAAAGqvDANBgkqhkiG9w0BAQsFADBcMQswCQYD

      VQQGEwJCRTERMA8GA1UEChMIUHJveGltdXMxFTATBgNVBAsTDHByb3hpbXVzLmNvbTEjMCEGA1UE
....
      /u6vZUeHkxS3NTsBckuFBi0eAO5Ipi2LwhGAgYDtu0H6R4plTvpsvkgCuNPdTRj+8dm3M+qjTY+f

      nK4M1zl3MdCXc4k0/ZjXEDuDIKWVfiG/RplQ4CtaXTh1ZXiVZDn75X/7jfs=

      -----END CERTIFICATE-----

      -----BEGIN RSA PRIVATE KEY-----

      MIIEowIBAAKCAQEA1zRtfzz4YVQ9lineLvM85bJUEBgzrMECZKr4GyVnZJs/dwYm

      qMNRxC1iNbaRBvuZ4WinTQGG1elbAhVrg23abMlbIScqyrkGGUYKpk2EbJks8mkn
...
      IbzMGlK4fSSLl4OblFP+7cstzlAM2sKsOMDPXo1vAT+x10oik+dO

      -----END RSA PRIVATE KEY-----

    termination: edge


# oc -n default describe  pod/router-1-5pm4l  | grep -i validation 
      EXTENDED_VALIDATION:                    true

# oc -n default logs router-1-5pm4l --tail 5
[ALERT] 238/080326 (412) : Fatal errors found in configuration.
E0826 08:03:32.811967       1 limiter.go:137] error reloading router: exit status 1
[ALERT] 238/080332 (415) : parsing [/var/lib/haproxy/conf/haproxy.config:116] : 'bind 127.0.0.1:10444' : 'crt-list' : error processing line 1 in file '/var/lib/haproxy/conf/cert_config.map' : unable to load SSL certificate file '/var/lib/haproxy/router/certs/test-1:myroute.pem' file does not exist.
[ALERT] 238/080332 (415) : Error(s) found in configuration file : /var/lib/haproxy/conf/haproxy.config
[ALERT] 238/080332 (415) : Fatal errors found in configuration.
-------

Comment 5 Arvind iyengar 2020-08-26 09:15:21 UTC
It appears that the latest build does have the patches. Going by the build time and the patch merge they last build is dated 18/08 which pre-dates the merge timeframe. Moving to "ON_QA" state for not to retest again in the  next build when it will be available.

Comment 6 Arvind iyengar 2020-08-31 04:52:26 UTC
Retested in the latest "v3.11.273" payload which has the patches merges. It is noted that in this version, introducing edge route with badly formatted certs/perms, the router continues to function without any issues and the certs gets injected into the router with right format:
------
# oc version
oc v3.11.273
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://ip-172-18-10-80.ec2.internal:8443
openshift v3.11.273
kubernetes v1.11.0+d4cacc0


# oc create route edge myroute --port=http --service=service-unsecure --cert=badcert.pem 
route.route.openshift.io/myroute created

# oc get route myroute -o yaml
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  annotations:
    openshift.io/host.generated: "true"
  creationTimestamp: "2020-08-31T04:13:49Z"
  labels:
    name: service-unsecure
  name: myroute
  namespace: test1
spec:
  host: myroute-test1.apps.0831-qv7.qe.rhcloud.com
  port:
    targetPort: http
  tls:
    certificate: |
      -----BEGIN CERTIFICATE-----

      MIIGnTCCBYWgAwIBAgITMAABqrxgky5s36Pm5QAAAAGqvDANBgkqhkiG9w0BAQsFADBcMQswCQYD

      VQQGEwJCRTERMA8GA1UEChMIUHJveGltdXMxFTATBgNVBAsTDHByb3hpbXVzLmNvbTEjMCEGA1UE
...
..
.
      /u6vZUeHkxS3NTsBckuFBi0eAO5Ipi2LwhGAgYDtu0H6R4plTvpsvkgCuNPdTRj+8dm3M+qjTY+f

      nK4M1zl3MdCXc4k0/ZjXEDuDIKWVfiG/RplQ4CtaXTh1ZXiVZDn75X/7jfs=

      -----END CERTIFICATE-----

      -----BEGIN RSA PRIVATE KEY-----

      MIIEowIBAAKCAQEA1zRtfzz4YVQ9lineLvM85bJUEBgzrMECZKr4GyVnZJs/dwYm

      qMNRxC1iNbaRBvuZ4WinTQGG1elbAhVrg23abMlbIScqyrkGGUYKpk2EbJks8mkn
...
..
.
      ZAzCWKsg7fgC+wBMreoQzYRimc4qUwhjpcWmAKwEO9Xgo3I0gRFLATgZPvgBtX0W

      IbzMGlK4fSSLl4OblFP+7cstzlAM2sKsOMDPXo1vAT+x10oik+dO

      -----END RSA PRIVATE KEY-----


# oc -n default logs router-1-m4kvw --tail 5
 - Checking http://localhost:80 ...
 - Health check ok : 0 retry attempt(s).
E0831 04:13:49.398728       1 limiter.go:137] error reloading router: wait: no child processes
 - Checking http://localhost:80 ...
 - Health check ok : 0 retry attempt(s).

# oc -n default rsh router-1-m4kvw 
sh-4.2$ ls -ltrh /var/lib/haproxy/router/certs/                   
total 4.0K
-rw-r--r--. 1 1000000000 root 4.0K Aug 31 04:13 test1:myroute.pem
------

Comment 9 errata-xmlrpc 2020-09-16 07:46:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 3.11.286 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3695


Note You need to log in before you can comment on or make changes to this bug.