Bug 1471707

Summary:	exposing docker-registry with a non tls-passthrough route does not work
Product:	OpenShift Container Platform	Reporter:	Alexander Koksharov <akokshar>
Component:	Image Registry	Assignee:	Michal Minar <miminar>
Status:	CLOSED ERRATA	QA Contact:	ge liu <geliu>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	3.4.0	CC:	aos-bugs, dyan, haowang, mfojtik, obulatov, peasters, pweil, sjr
Target Milestone:	---
Target Release:	3.7.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: The registry used to append forwarded target port to redirected location urls. Registry client gets confused by the received location containing superfluous port and cannot match it against the original host. This happened when exposed with tls-termination other than passthrough. Consequence: Client's new request to the target location lacks credentials. As a consequence, image push fails due to authorization error. Fix: Registry was rebased to newer version which fixes forwarding processing logic. Result: Registry now doesn't confuse its clients. Clients can push images successfully to the exposed registry using arbitrary tls-termination.	Story Points:	---
Clone Of:
Clones:	1489039 1489042 (view as bug list)		Environment:
Last Closed:	2017-11-28 22:04:10 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1489039, 1489042

Description Alexander Koksharov 2017-07-17 09:42:35 UTC

Description of problem:

Docker registry is exposed with tls passthrough route. All is working fine.
However, client is getting openshift's self-signed certificate when connected to the service.

Attempts to change route to anything else other then tls-passthrough brick authentication. Even login is successfull, push fails. Here is an example:
# docker login -u test -p 5ah1OnexCWZA-OVi1I1aqP3QGRwurfdodx6qZYmfD4A docker-registry-default.apps.lex.lab
Login Succeeded
# docker push docker-registry-default.apps.lex.lab/test1/alpine
The push refers to a repository [docker-registry-default.apps.lex.lab/test1/alpine]
5bef08742407: Pushing [==================================================>] 3.962 MB/3.962 MB
unauthorized: authentication required


Version-Release number of selected component (if applicable):
3.4 
3.5

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Michal Minar 2017-07-17 13:47:41 UTC

I smell a promising bugfix candidate: https://github.com/openshift/origin/pull/14866

I'll confirm this soon.

Comment 4 Michal Minar 2017-08-01 11:14:46 UTC

Unfortunately, https://github.com/openshift/origin/pull/14866 doesn't fix the issue. I'm debugging further.

Comment 8 Oleg Bulatov 2017-09-04 12:21:01 UTC

Fixed in upstream, rebase [1] merged into 3.7.

[1]: https://github.com/openshift/origin/pull/15694

Comment 9 Michal Minar 2017-09-06 10:03:17 UTC

Added doc text.

Comment 11 Michal Minar 2017-09-08 14:25:27 UTC

This needs to be double-checked.

@tomckay found out that with the fix in question, :443 suffix added to registry names causes timeouts.

We need to make sure that our registry can be addressed both with&without the :443 suffix because many customers added it to their external registries as a work-around for the broken port forwarding.

This needs to be further investigated.

Comment 12 Michal Minar 2017-09-27 16:29:21 UTC

I've successfully pushed with&without the :443 to the recent docker registry with the fix applied.

Therefore, I'm switching this to QA for confirmation. And I'll start with the back-porting effort.

Comment 13 Dongbo Yan 2017-09-29 07:58:33 UTC

Verified
openshift v3.7.0-0.127.0
kubernetes v1.7.0+80709908fd
etcd 3.2.1

Comment 16 errata-xmlrpc 2017-11-28 22:04:10 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188