Proposed title of this feature request SSL/TLS client certificate validation for OpenShift router Description: Request a feature to enable mTLS (mutual TLS) authentication with the OpenShift Router. With options to set ACL used to match the certificate common name, rules to delete the three headers, and headers added for transparency. Example diff of features added: --- examples/haproxy-config.template 2017-06-29 12:54:18.131120000 -0400 +++ custom/haproxy-config.template 2017-07-25 16:45:31.829700000 -0400 @@ -186,12 +186,45 @@ frontend fe_sni # terminate ssl on edge - bind 127.0.0.1:{{env "ROUTER_SERVICE_SNI_PORT" "10444"}} ssl no-sslv3 {{ if (len .DefaultCertificate) gt 0 }}crt {{.DefaultCertificate}}{{ else }}crt /var/lib/haproxy/conf/default_pub_keys.pem{{ end }} crt-list /var/lib/haproxy/conf/cert_config.map accept-proxy + bind 127.0.0.1:{{env "ROUTER_SERVICE_SNI_PORT" "10444"}} ssl no-sslv3 {{ if (len .DefaultCertificate) gt 0 }}crt {{.DefaultCertificate}}{{ else }}crt /var/lib/haproxy/conf/default_pub_keys.pem{{ end }} crt-list /var/lib/haproxy/conf/cert_config.map accept-proxy ca-file /var/lib/haproxy/conf/custom/client-chain.pem verify optional mode http # Remove port from Host header http-request replace-header Host (.*):.* \1 + # Check the CN of the client certificate to see if it is in the list + # An example of checking the CN in OpenShift is in the OpenShift docs + # https://docs.openshift.com/container-platform/3.4/install_config/router/customized_haproxy_router.html#using-annotations + acl match ssl_c_s_dn(CN) -m sub CustomName\ TAM\ Web\ Client + + # Default X-Headers-Sanitized to false, set to true if test fails + http-request set-header X-Headers-Sanitized false + + # Debugging information. Add headers to indicate front-end and status. + http-request set-header X-HAProxy-Frontend fe_sni + http-request set-header X-Headers-Sanitized true unless { ssl_c_verify 0 } match + + # Debugging information. Store a copy of iv- headers in X-Sanitized + http-request set-header X-Sanitized-iv-user %[req.hdr(iv-user)] unless { ssl_c_verify 0 } match + http-request set-header X-Sanitized-iv-creds %[req.hdr(iv-creds)] unless { ssl_c_verify 0 } match + http-request set-header X-Sanitized-iv-groups %[req.hdr(iv-groups)] unless { ssl_c_verify 0 } match + + # Sanitize iv- headers unless verification succeeded and CN matches + http-request del-header iv-user unless { ssl_c_verify 0 } match + http-request del-header iv-groups unless { ssl_c_verify 0 } match + http-request del-header iv-creds unless { ssl_c_verify 0 } match + + # Pass the certificate information to the client (informational) + # https://raymii.org/s/tutorials/haproxy_client_side_ssl_certificates.html + http-request set-header X-SSL %[ssl_fc] + http-request set-header X-SSL-Client-Verify %[ssl_c_verify] + http-request set-header X-SSL-Client-SHA1 %{+Q}[ssl_c_sha1] + http-request set-header X-SSL-Client-DN %{+Q}[ssl_c_s_dn] + http-request set-header X-SSL-Client-CN %{+Q}[ssl_c_s_dn(cn)] + http-request set-header X-SSL-Issuer %{+Q}[ssl_c_i_dn] + http-request set-header X-SSL-Client-Not-Before %{+Q}[ssl_c_notbefore] + http-request set-header X-SSL-Client-Not-After %{+Q}[ssl_c_notafter] + # check re-encrypt backends first - from most specific to general path. acl reencrypt base,map_beg(/var/lib/haproxy/conf/os_reencrypt.map) -m found @@ -237,12 +270,43 @@ frontend fe_no_sni # terminate ssl on edge - bind 127.0.0.1:{{env "ROUTER_SERVICE_NO_SNI_PORT" "10443"}} ssl no-sslv3 {{ if (len .DefaultCertificate) gt 0 }}crt {{.DefaultCertificate}}{{ else }}crt /var/lib/haproxy/conf/default_pub_keys.pem{{ end }} accept-proxy + bind 127.0.0.1:{{env "ROUTER_SERVICE_NO_SNI_PORT" "10443"}} ssl no-sslv3 {{ if (len .DefaultCertificate) gt 0 }}crt {{.DefaultCertificate}}{{ else }}crt /var/lib/haproxy/conf/default_pub_keys.pem{{ end }} accept-proxy ca-file /var/lib/haproxy/conf/custom/client-chain.pem verify optional mode http # Remove port from Host header http-request replace-header Host (.*):.* \1 + # Check the CN of the client certificate to see if it is in the list + acl match ssl_c_s_dn(CN) -m sub CustomName \ TAM\ Web\ Client + + # Default X-Headers-Sanitized to false, set to true if test fails + http-request set-header X-Headers-Sanitized false + + # Debugging information. Add headers to indicate front-end and status. + http-request set-header X-HAProxy-Frontend fe_no_sni + http-request set-header X-Headers-Sanitized true unless { ssl_c_verify 0 } match + + # Debugging information. Store a copy of iv- headers in X-Sanitized + http-request set-header X-Sanitized-iv-user %[req.hdr(iv-user)] unless { ssl_c_verify 0 } match + http-request set-header X-Sanitized-iv-creds %[req.hdr(iv-creds)] unless { ssl_c_verify 0 } match + http-request set-header X-Sanitized-iv-groups %[req.hdr(iv-groups)] unless { ssl_c_verify 0 } match + + # Sanitize iv- headers unless verification succeeded and CN matches + http-request del-header iv-user unless { ssl_c_verify 0 } match + http-request del-header iv-groups unless { ssl_c_verify 0 } match + http-request del-header iv-creds unless { ssl_c_verify 0 } match + + # Pass the certificate information to the client (informational) + # https://raymii.org/s/tutorials/haproxy_client_side_ssl_certificates.html + http-request set-header X-SSL %[ssl_fc] + http-request set-header X-SSL-Client-Verify %[ssl_c_verify] + http-request set-header X-SSL-Client-SHA1 %{+Q}[ssl_c_sha1] + http-request set-header X-SSL-Client-DN %{+Q}[ssl_c_s_dn] + http-request set-header X-SSL-Client-CN %{+Q}[ssl_c_s_dn(cn)] + http-request set-header X-SSL-Issuer %{+Q}[ssl_c_i_dn] + http-request set-header X-SSL-Client-Not-Before %{+Q}[ssl_c_notbefore] + http-request set-header X-SSL-Client-Not-After %{+Q}[ssl_c_notafter] + # check re-encrypt backends first - path or host based. acl reencrypt base,map_beg(/var/lib/haproxy/conf/os_reencrypt.map) -m found
The big question that we need to work out is what control there needs to be on a per-route basis. i.e. there are a lot of options for this, and if you need per-route control of most of them this becomes harder. What can be defined router-wide? - The ca that are trusted for the client certs? - The http headers that are set for the requests? Obviously, things like the CNs to trust or whether to allow or drop connections that were not verified will need to vary per route, but that can be done with an annotation. Is it reasonable to mandate that all routes handled by a particular router would be validated with the same CA cert (if validation was requested by a per-route annotation)? Then, if validation was requested, we would set some X-SSL headers as this does: https://raymii.org/s/tutorials/haproxy_client_side_ssl_certificates.html Then on each route you would have an annotation that would specify whether client verification was required, or optional. If required, it would drop connections that didn't validate. If optional, or required with a valid connection, the http headers would be set. There would also be an optional annotation on the routes that would let you provide a CN to match against. However, we would not support adding arbitrary headers, or having alternate backends depending on marches. Is that sufficient?
Associated PR: https://github.com/openshift/origin/pull/19891 While generalizing this, there's a few things of note here: 1. The default config uses the CN name as an additional restriction mechanism for clients. This can be optionally turned on if needed. For the specific customer case here, they can just define a custom template that sets/unsets the custom headers rather than deny the http requests. 2. The names of some of the headers e.g. X-SSL-Client-Not{Before,After} are slightly different for consistency. 3. There's some additional headers for the certificate client version and serial number: X-SSL-Client-Serial and X-SSL-Client-Version 4. The X-SSL-Client-SHA1 header value is converted to hex format (rather than sending it down as binary).
Looking and the associated PR, it seems that the decision was made to have the mutual TLS authentication configured at the router level, versus the other option which would have been the route level. Can we explain why? Maybe it is acceptable that all the client be validated by the same CA bundle, but at least one would expect that the list of allowed CNs would be different per route...
(In reply to raffaele spazzoli from comment #14) Correct. Below are some of the reasons: 1. All edge and reencrypt routes share the same frontend. 2. Certificates for trust must be provided by cluster-admin 3. Non-SNI traffic must be handled by a single frontend. When passthrough is not used, especially in the use-case for legacy clients where SNI is not supported, the SNI and non-SNI frontends must match to prevent a backdoor. Because certificate validation occurs on the frontend and non-SNI traffic cannot have more than a single frontend, it would be difficult to support on a per-route basis. Providing the certificate trust chain is a potential issue as haproxy uses a file that a project user cannot provide to the router without the help of an administrator. While multiple frontends can potentially be templated for SNI traffic, the non-SNI use case this was intended for is a limiting factor.
Red Hat is moving OpenShift feature requests to a new JIRA RFE system. This bz (RFE) has been identified as a feature request which is still being evaluated and has been moved. As the new Jira RFE system is not yet public, Red Hat Support can help answer your questions about your RFEs via the same support case system. https://.jira.coreos.com/browse/RFE-158