Bug 1333978

Summary:	docker push fails at each layer and finally with unauth error
Product:	OpenShift Container Platform	Reporter:	Steve Speicher <sspeiche>
Component:	Image Registry	Assignee:	Michal Minar <miminar>
Status:	CLOSED DUPLICATE	QA Contact:	Wei Sun <wsun>
Severity:	low	Docs Contact:
Priority:	medium
Version:	3.2.0	CC:	aos-bugs, dakini, pweil, sspeiche, vdinh
Target Milestone:	---	Flags:	sspeiche: needinfo+
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-08-15 13:35:42 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Steve Speicher 2016-05-06 21:24:58 UTC

Description of problem:
I'm able to successfully push an image but not others, see error message below.

Version-Release number of selected component (if applicable):
dev-preview-int
OSE 3.2.0.41

How reproducible:

Steps to Reproduce:
1. Login to dev-preview-int
2. docker login -u sspeiche -e sspeiche -p `oc whoami -t` https://registry.dev-preview-int.openshift.com
3. (pull images into local registry, tag)
4. docker push (see below)

Actual results:
** FAILURE **

$ docker push registry.dev-preview-int.openshift.com/sspeiche-microservices/nodejs-4-rhel7
The push refers to a repository [registry.dev-preview-int.openshift.com/sspeiche-microservices/nodejs-4-rhel7]
a1bee891fb36: Image push failed 
c57a1a457290: Image push failed 
7ce91b62fd38: Image push failed 
unauthorized: authentication required

Expected results:

** SUCCESS **

Comment 1 Steve Speicher 2016-05-06 21:26:53 UTC

I was able to successfully push after this, as:

docker push registry.dev-preview-int.openshift.com/sspeiche-jenkins/oc
The push refers to a repository [registry.dev-preview-int.openshift.com/sspeiche-jenkins/oc]
5f70bf18a086: Pushed 
1b2ae4a2ae16: Pushed 
89eaed6841a8: Pushed 
00ad69006606: Pushed 
6f94656375ca: Pushed 
6ad012af89d8: Pushed 
latest: digest: sha256:669f27dbc122f82fdc06c1bd674b12200b071703ce8aa97b662e0d5b426d2cff size: 6279

Comment 2 Vu Dinh 2016-05-09 15:26:29 UTC

Hi Steve,

I was able to push an image from Dockerhub to OpenShift Dev-Preview this morning on the first try. I wonder if you still experience the push failure.

Vu

Comment 3 Steve Speicher 2016-05-09 16:35:49 UTC

I just pull locally jankins:2.0 image from dockerhub. I then tagged it and pushed it to registry.dev-preview-int.openshift.com and after waiting a long time, I got a 504 error.

$ docker pull jenkins:2.0
$ docker tag jenkins:2.0 registry.dev-preview-int.openshift.com/sspeiche-jenkins/jenkins
$ docker push registry.dev-preview-int.openshift.com/sspeiche-jenkins/jenkins
The push refers to a repository [registry.dev-preview-int.openshift.com/sspeiche-jenkins/jenkins]
8b267d6498d7: Pushed 
5f70bf18a086: Pushed 
76e949818998: Pushed 
bc5d64543995: Pushed 
afe9dda354c0: Pushed 
72c095b58395: Pushed 
d4eb89664ea9: Pushed 
ec7adcf6c1db: Pushed 
1abd30ec1bf3: Pushed 
88148dd5f97a: Pushed 
aaab6c7855ac: Pushed 
6ef60a97b376: Pushed 
dd3463e7917c: Pushed 
9dab486f5005: Pushed 
f07790056444: Pushed 
8befcf623ce4: Pushed 
3d5a262d6929: Pushed 
6eb35183d3b8: Pushed 
Received unexpected HTTP status: 504 Gateway Time-out

Comment 4 Steve Speicher 2016-05-09 16:37:17 UTC

2nd attempt succeeds, trying to reproduce the unauth error still:

docker push registry.dev-preview-int.openshift.com/sspeiche-jenkins/jenkins
The push refers to a repository [registry.dev-preview-int.openshift.com/sspeiche-jenkins/jenkins]
8b267d6498d7: Layer already exists 
5f70bf18a086: Layer already exists 
76e949818998: Layer already exists 
bc5d64543995: Layer already exists 
afe9dda354c0: Layer already exists 
72c095b58395: Layer already exists 
d4eb89664ea9: Layer already exists 
ec7adcf6c1db: Layer already exists 
1abd30ec1bf3: Layer already exists 
88148dd5f97a: Layer already exists 
aaab6c7855ac: Layer already exists 
6ef60a97b376: Layer already exists 
dd3463e7917c: Layer already exists 
9dab486f5005: Layer already exists 
f07790056444: Layer already exists 
8befcf623ce4: Layer already exists 
3d5a262d6929: Layer already exists 
6eb35183d3b8: Layer already exists 
latest: digest: sha256:2c1d0325822d86d19ea396b77bc8e6b024e9243bfd54a4986f0f87a66aa5867d size: 22923

Comment 5 Steve Speicher 2016-05-09 16:44:37 UTC

This is user error (but confusing error message).

I have the wrong namespace provided for my image (I put sspeiche-microservices but I should put sspeiche-jenkins). The only error I get is "unauthorized: authentication required".

So it is really a usability / error-message bug. I need to open a separate bug about the 504 errors.

Comment 6 Vu Dinh 2016-05-09 16:46:19 UTC

Hi Steve,

The timeout issue is due to the registry time-out restriction that ops has set up for the Online Dev-Preview registry. They already increased it once but it's not enough. I'm currently in talk with ops to increase it again to avoid this issue.

Vu

Comment 7 Stefanie Forrester 2016-05-09 16:52:13 UTC

Here are the new settings. The internal API is now using the maximum allowed timeout to prevent disconnects. 
https://github.com/openshift/online/issues/131#issuecomment-217921293

Comment 8 Steve Speicher 2016-05-11 13:16:50 UTC

For me, it is a usability issue if you fat finger the name or not logged in. So I'd say it is medium or low sev/pri.

Comment 9 Michal Minar 2016-08-15 13:35:42 UTC


*** This bug has been marked as a duplicate of bug 1335279 ***