Bug 1651034 (CVE-2018-16886) - CVE-2018-16886 etcd: Improper Authentication in auth/store.go:AuthInfoFromTLS() via gRPC-gateway
Summary: CVE-2018-16886 etcd: Improper Authentication in auth/store.go:AuthInfoFromTLS...
Status: CLOSED ERRATA
Alias: CVE-2018-16886
Product: Security Response
Classification: Other
Component: vulnerability
Version: unspecified
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Red Hat Product Security
QA Contact:
URL:
Whiteboard: impact=moderate,public=20190111:2205,...
Keywords: Security
Depends On: 1666159 1665782 1665810 1666154 1666155 1666156 1666157 1685702
Blocks: 1651035
TreeView+ depends on / blocked
 
Reported: 2018-11-19 01:27 UTC by Sam Fowler
Modified: 2019-06-11 11:13 UTC (History)
29 users (show)

(edit)
Etcd, versions 3.2.0 through 3.2.25 and 3.3.0 through 3.3.10, are vulnerable to an improper authentication issue when role-based access control (RBAC) is used and client-cert-auth is enabled. If an etcd client server's TLS certificate contains a Common Name (CN) which matches a valid RBAC username, a remote attacker may authenticate as that user with any valid (trusted) client certificate in a REST API request to the gRPC-gateway.
Clone Of:
(edit)
Last Closed: 2019-06-10 10:42:46 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2019:0237 None None None 2019-01-31 19:23 UTC
Red Hat Product Errata RHSA-2019:1352 None None None 2019-06-04 19:10 UTC

Internal Trackers: 1676902

Description Sam Fowler 2018-11-19 01:27:46 UTC
Etcd, versions 3.2.0 through 3.2.25 and 3.3.0 through 3.3.10, are vulnerable to an improper authentication issue when role-based access control (RBAC) is used and client-cert-auth is enabled. If an etcd client server's TLS certificate contains a Common Name (CN) which matches a valid RBAC username, a remote attacker may authenticate as that user with any valid (trusted) client certificate in a REST API request to the gRPC-gateway.

Introduced in commit:
https://github.com/etcd-io/etcd/commit/0191509637546621d6f2e18e074e955ab8ef374d

Upstream issue:
https://github.com/etcd-io/etcd/pull/10366

Upstream patch:
https://github.com/etcd-io/etcd/commit/bf9d0d8291dc71ecbfb2690612954e1a298154b2
https://github.com/etcd-io/etcd/commit/a9a9466fb8ba11ad7bb6a44d7446fbd072d59887
https://github.com/etcd-io/etcd/commit/99704e2a97e8710da942bdc737417fc9c9a2c03f
https://github.com/etcd-io/etcd/commit/83c051b701d33261eef91a719e4421c81b000ba4

Comment 3 Sam Batschelet 2018-11-27 13:11:40 UTC
From my review, this issue is not reproducible. Those curious in my testing can review the following.

https://docs.google.com/document/d/1ph7LOfUsdKo2qdgMvFahVSd7T6661c8yRr_mR2m3Gms/edit?usp=sharing

My feeling is given the data we have this embargo should be lifted.

Comment 5 Sam Fowler 2019-01-04 01:17:56 UTC
Acknowledgments:

Name: Matt Wheeler (Osirium)

Comment 6 Sam Batschelet 2019-01-04 02:29:24 UTC
Point release with fix scheduled for Tuesday January 8th. v3.3.11

https://github.com/etcd-io/etcd/pull/10366

Comment 7 Jan Chaloupka 2019-01-08 11:53:31 UTC
Hi Sam,

#10366 is not merged yet and the latest release available is v3.3.10. Is the patch ready to be merged and just waiting for finishing upstream review. Or, is there anything else left to be done before the PR can be accepted?

Comment 8 Sam Batschelet 2019-01-08 12:52:05 UTC
Jan,
The PR should be merged today and a new release cut shortly. As far as scope this also affects all etcd 3.2.x and 3.3.x that is using --client-cert-auth. So we are backporting fix to 3.2 as well.

Comment 17 Paul Harvey 2019-01-11 04:35:09 UTC
Setting CVSSv3 PR:L because an attacker still needs to obtain a valid TLS certificate that can be mutually authenticated by the server.

Comment 18 Paul Harvey 2019-01-11 04:43:09 UTC
Updated title to convey the scope of the vulnerability more concisely

Comment 19 Paul Harvey 2019-01-11 04:59:32 UTC
Added doctext, mitigation and external references.

Comment 22 Paul Harvey 2019-01-11 07:57:57 UTC
Quick list of upstream versions affected:
  $ git tag --contains 0191509637546621d6f2e18e074e955ab8ef374d | tr '\n' ' '
  v3.2.0 v3.2.0-rc.0 v3.2.0-rc.1 v3.2.1 v3.2.10 v3.2.11 v3.2.12 v3.2.13 v3.2.14 v3.2.15 v3.2.16 v3.2.17 v3.2.18 v3.2.19 v3.2.2 v3.2.20 v3.2.21 v3.2.22 v3.2.23 v3.2.24 v3.2.25 v3.2.3 v3.2.4 v3.2.5 v3.2.6 v3.2.7 v3.2.8 v3.2.9 v3.3.0 v3.3.0-rc.0 v3.3.0-rc.1 v3.3.0-rc.2 v3.3.0-rc.3 v3.3.0-rc.4 v3.3.1 v3.3.10 v3.3.2 v3.3.3 v3.3.4 v3.3.5 v3.3.6 v3.3.7 v3.3.8 v3.3.9

Comment 23 Sam Batschelet 2019-01-11 14:15:45 UTC
(In reply to Paul Harvey from comment #21)
> Mitigation:
> 
> If a gRPC proxy is used, remove it from the affected deployment such that
> clients connect directly to etcd, or use an authentication method other than
> client-cert-auth.

FTR this issue does not require grpc-proxy.

If etcd is using RBAC and `--client-cert-auth` is enabled. Make sure that the client server certificate `--cert-file` does not include a common name. IF a common name is part of this certificate consider removing. If removal is not possible understand that a RBAC user with username = to CN can be exploited by any REST request to the gRPC-gateway. Essentially the username in the CA will always be used during authentication. 

openssl x509 -noout -subject -in client.pem | grep -o 'CN.*'

example curl --key $key --cert $cert --cacert $ca -L https://localhost:2379/v3beta/kv/put \
  -X POST -d '{"key": "Zm9v", "value": "YmFy"}'

Comment 24 Jan Chaloupka 2019-01-11 15:41:05 UTC
Sam,

checking the etcd release [1] I still don't see any new update. Can we wait until 3.3.11 is out or do we need to downstream the PR ourselves?

[1] https://github.com/etcd-io/etcd/releases

Comment 25 Jan Chaloupka 2019-01-11 15:41:23 UTC
By update I mean release.

Comment 26 Sam Batschelet 2019-01-11 15:49:31 UTC
Hi Jan from above.

> Yeah we would like to cut 3.3.11 with 3.2.26 but the maintainer from Google that does 3.2 releases has been delayed returning from holiday and should return Friday.

I should know more today, I am sorry this process takes so long but we (etcd) currently are dependent on 2 specific maintainers to cut an etcd release. I am trying to get involved in this process to improve this turn around.

Comment 28 Sam Batschelet 2019-01-11 22:06:06 UTC
https://github.com/etcd-io/etcd/releases

v3.3.11 and v3.3.26 are now live.

Comment 30 Paul Harvey 2019-01-14 00:27:59 UTC
Re-worded flaw description according to common 23, thanks Sam. Could you please sanity-check my updated wording of the DocText and Mitigation section?

Comment 34 Paul Harvey 2019-01-14 00:40:24 UTC
Mitigation:

Ensure that the client server TLS certificate (specified in --cert-file argument or ETCD_CERT_FILE environment variable) does not include a CN (Common Name) field. If a Common Name field is part of this certificate, replace it with one which omits it.

To check the CN field of a certificate:
  openssl x509 -noout -subject -in /path/to/client.crt | grep -o 'CN.*'

To check if there is a username matching the CN field in the TLS client certificate:
  etcdctl user get <TLS client certificate CN>

For more information on TLS authentication features including how client-cert-auth is enabled, refer to the etcd transport security model documentation: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md
For more information on Role-based access control including how it is enabled, refer to the etcd role-based access control documentation: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/authentication.md

Comment 37 Paul Harvey 2019-01-14 01:55:01 UTC
Created etcd tracking bugs for this issue:

Affects: fedora-all [bug 1665782]

Comment 38 Paul Harvey 2019-01-14 03:36:40 UTC
This isn't really an auth bypass; the mutual TLS authentication is still working, it's more that our REST endpoint is makeing an auth decision based on irrelevant data. Changed to CWE-287, Improper Authentication

Comment 45 Paul Harvey 2019-01-25 06:07:07 UTC
openshift-enterprise-3.11: metrics-server component was incorrectly marked as affected; it is actually notaffected because the vendored etcd code in the upstream project was at version 3.1.10, which is not part of the vulnerable version range for this flaw.

Comment 46 errata-xmlrpc 2019-01-31 19:23:08 UTC
This issue has been addressed in the following products:

  Red Hat Enterprise Linux 7 Extras

Via RHSA-2019:0237 https://access.redhat.com/errata/RHSA-2019:0237

Comment 52 Jason Shepherd 2019-04-04 02:18:07 UTC
Statement:

OpenShift Container Platform 3.x, and 4.1 versions do not use etcd Role-based access control so they are not affected.

Comment 55 errata-xmlrpc 2019-06-04 19:10:01 UTC
This issue has been addressed in the following products:

  Red Hat Enterprise Linux 7 Extras

Via RHSA-2019:1352 https://access.redhat.com/errata/RHSA-2019:1352


Note You need to log in before you can comment on or make changes to this bug.