Bug 1511732
| Summary: | [3.5] extended route validation does not catch newlines in certificate | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Steven Walter <stwalter> |
| Component: | Networking | Assignee: | Jacob Tanenbaum <jtanenba> |
| Networking sub component: | router | QA Contact: | zhaozhanqi <zzhao> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | urgent | ||
| Priority: | urgent | CC: | aos-bugs, bbennett, bmeng, dmoessne, dsafford, fweimer, haowang, jtanenba, nbhatt, rhowe, sgaikwad, weliang |
| Version: | 3.5.1 | ||
| Target Milestone: | --- | ||
| Target Release: | 3.5.z | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: Invalid PEM data is left in the route config file during extended validation.
Consequence: The router crashes
Fix: Sanitize PEM data from route configs
Result: Properly catch malformed certificates in extended validation.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-12-14 21:02:32 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
From Clayton: Extended validation in 3.5 had gaps. Most were fixed in 3.6. You can run the 3.6 diagnostic command against a 3.5 cluster to check whether any routes would fail EV - if they can recreate in a test cluster please have them rerun. Most of the things fixed were invalid PEM blocks and incomplete content. To reproduce this I removed one "-" from the -----END CERTIFICATE---- Was only able to reproduce in 3.5 removing a dash from -----END CERTIFICATE---- and only END CERTIFICATE section hit this issue. |+ |- and | all work fine with 3.5 routes. Fatal errors only happened after editing -----END CERTIFICATE---- In 3.6 the router corrected the pem file adding the - back to the route when the pem was added to /var/lib/haproxy/router/certs/, the route object itself still showed the missing -. This did not cause 3.6 router to fail. PR/926 fixed the problem. Testing passed QE did the testing on 3.5.5.31.47 1. As comment 7 said: |+ |- and | all work fine 2. if I removed one "-" from the -----END CERTIFICATE----, the route will be marked as 'ExtendedValidationFailed' This situation is now very critical. I just set the Customer Escalation flag and the am requesting a hotfix (so I flipped that flag too). Hello Jacob/Rashid I wrote yesterday in ref Bug 1511732 which is still in Status (Modified) and the following question below from the Action Plan: 1. Are we able to meet customers expectations on being able to deliver an errata fixing this by the 15th of December? If not 2. When is the cut off date on knowing if this expectation cant be achieved? 3. How long would the hotifix take? and when or who can deliver this. For more info, contact Aaron Ship (Critsit Manager) aship in irc verified this bug according to comment 15 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3438 |
Description of problem: In the route definition we saw empty lines: ... certificate: | -----BEGIN CERTIFICATE----- ... key: | -----BEGIN PRIVATE KEY----- ... It is hardly noticeable in YAML format. On the OpenShift master console GUI it can be seen better. Normally it should be |+ instead of just a | character. Then the haproxy concatenates the certificates into a PEM file where the empty lines appear, and we think it causes those error messages. Version-Release number of selected component (if applicable): image: ose-haproxy-router:v3.5.5.31 OpenShift version: v3.5.5.31.24 How reproducible: Unconfirmed Steps to Reproduce: 1. Created route in web console Actual results: 2017-11-09T09:20:32.957490000Z [ALERT] 312/092032 (127789) : parsing [/var/lib/haproxy/conf/haproxy.config:142] : 'bind 127.0.0.1:10444' : 'crt-list' : error processing line 150 in file '/var/lib/haproxy/conf/cert_config.map' : unable to load SSL certificate from PEM file '/var/lib/haproxy/router/certs/example.pem'. 2017-11-09T09:20:32.957682000Z [ALERT] 312/092032 (127789) : Error(s) found in configuration file : /var/lib/haproxy/conf/haproxy.config 2017-11-09T09:20:32.957873000Z [ALERT] 312/092032 (127789) : Fatal errors found in configuration. Expected results: Non-fatal warning, not accepting the route Additional info: Verified oc set env dc/router EXTENDED_VALIDATION=true Deleting route causes error to go away