Bug 1857022

Summary: Creating faulty(bad formatted cert&key) route makes other existing routes inaccessible
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: NetworkingAssignee: Miciah Dashiel Butler Masters <mmasters>
Networking sub component: router QA Contact: Arvind iyengar <aiyengar>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: aiyengar, amcdermo, aos-bugs, dhansen, hongli
Version: 3.11.0   
Target Milestone: ---   
Target Release: 4.4.z   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-01 19:41:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1857021    
Bug Blocks: 1857023    

Comment 1 Miciah Dashiel Butler Masters 2020-07-30 08:26:14 UTC
We'll track this 4.4 backport in the upcoming sprint.

Comment 5 Arvind iyengar 2020-08-24 08:10:09 UTC
The merge made into "4.4.0-0.nightly-2020-08-20-031859" release version. With the patched version it is noted that improperly formatted PEM/CERT file containing the crt and the key in one file does not disrupt the router operation with certificate loading errors. The route gets admitted and the specific tls file gets add in the router:
-----
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.4.0-0.nightly-2020-08-20-031859   True        False         3h9m    Cluster version is 4.4.0-0.nightly-2020-08-20-031859

$ oc create route edge myroute --port=http --service=service-unsecure --hostname=service-unsecure-marty.internalapps.oc44-1857022-patched.qe.devcluster.openshift.com --cert=bad.pem
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  creationTimestamp: "2020-08-24T07:45:35Z"
  labels:
    name: service-unsecure
  name: myroute
  namespace: marty
  resourceVersion: "72108"
  selfLink: /apis/route.openshift.io/v1/namespaces/marty/routes/myroute
  uid: 53756b37-5b4e-48c6-8709-92f7ad07bf58
spec:
  host: service-unsecure-marty.internalapps.oc44-1857022-patched.qe.devcluster.openshift.com
  port:
    targetPort: http
  tls:
    certificate: |+
      -----BEGIN CERTIFICATE-----

      MIIGnTCCBYWgAwIBAgITMAABqrxgky5s36Pm5QAAAAGqvDANBgkqhkiG9w0BAQsFADBcMQswCQYD

      VQQGEwJCRTERMA8GA1UEChMIUHJveGltdXMxFTATBgNVBAsTDHByb3hpbXVzLmNvbTEjMCEGA1UE
....
      -----END CERTIFICATE-----
      -----BEGIN RSA PRIVATE KEY-----

      MIIEowIBAAKCAQEA1zRtfzz4YVQ9lineLvM85bJUEBgzrMECZKr4GyVnZJs/dwYm

      qMNRxC1iNbaRBvuZ4WinTQGG1elbAhVrg23abMlbIScqyrkGGUYKpk2EbJks8mkn

      bg7DK3Hzxv+3tkmrPoK9CDi8D4IaA1Z4Bt9QAOEh1gQcs8eYD72CY9Y/W3JNXpBT
      -----END RSA PRIVATE KEY-----

    termination: edge

$ oc -n openshift-ingress logs deployments/router-default --tail 5
Found 2 pods, using pod/router-default-56c8f74654-zw9bl
I0824 07:34:46.957017       1 router.go:559] template "level"=0 "msg"="router reloaded"  "output"=" - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"
I0824 07:34:51.939560       1 router.go:559] template "level"=0 "msg"="router reloaded"  "output"=" - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"
I0824 07:35:31.625057       1 router.go:559] template "level"=0 "msg"="router reloaded"  "output"=" - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"
I0824 07:36:07.431921       1 router.go:559] template "level"=0 "msg"="router reloaded"  "output"=" - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"
I0824 07:45:35.059890       1 router.go:559] template "level"=0 "msg"="router reloaded"  "output"=" - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"

$ oc rsh router-<name>
$ ls /var/lib/haproxy/router/certs/marty\:myroute.pem 
/var/lib/haproxy/router/certs/marty:myroute.pem <---
-----

* The same steps on unpatched version leads to following error and cease of router operation:
----
$ oc -n openshift-ingress logs router-default-5884cffcb5-wgb25 --tail 5
I0824 08:00:03.423699       1 router.go:559] template "level"=0 "msg"="router reloaded"  "output"=" - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"
E0824 08:00:33.537929       1 limiter.go:165] error reloading router: exit status 1
[ALERT] 236/080033 (679) : parsing [/var/lib/haproxy/conf/haproxy.config:119] : 'bind 127.0.0.1:10444' : 'crt-list' : error processing line 1 in file '/var/lib/haproxy/conf/cert_config.map' : unable to load SSL certificate file '/var/lib/haproxy/router/certs/bob:myroute.pem' file does not exist.
[ALERT] 236/080033 (679) : Error(s) found in configuration file : /var/lib/haproxy/conf/haproxy.config
[ALERT] 236/080033 (679) : Fatal errors found in configuration.
----

Comment 7 errata-xmlrpc 2020-09-01 19:41:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.4.19 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3514