Bug 1294115

Summary:	Get error status 404 when trying to push result image to internal docker registry if registry is temporarily down and resumed immediately
Product:	OpenShift Container Platform	Reporter:	Xia Zhao <xiazhao>
Component:	Build	Assignee:	Ben Parees <bparees>
Status:	CLOSED ERRATA	QA Contact:	Wenjing Zheng <wzheng>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	3.1.0	CC:	aos-bugs, bleanhar, bparees, jokerman, pruan
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-01-26 19:20:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Xia Zhao 2015-12-25 02:38:54 UTC

Description of problem:
Get error status 404 when trying to push result image to internal docker registry if registry is temporarily down and resumed immediately

Version-Release number of selected component (if applicable):
oc v3.1.1.0
kubernetes v1.1.0-origin-1107-g4c8e6f4

How reproducible:
always

Steps to Reproduce:
1. Make sure the docker registry pod is running normally 
$oc get pods -n default
2. Create a new project
3. Create resources
$oc process -f https://raw.githubusercontent.com/openshift/origin/master/examples/sample-app/application-template-stibuild.json | oc create -f -
4. Check build log, during pushing result image to registry, scale down registry pod to 0
$oc build-logs <build_name>
$oc scale rc --replicas=0 docker-registry-1 -n default
5. Check the registry pod like step #1
$oc get pods -n default
6. Scale up registry pod and make sure the docker registry is up again
$oc scale rc --replicas=1 docker-registry-1 -n default
$oc get pods -n default
7. Watch build status and log

Actual results:
Failed in pushing result image to docker registry when registry is temporarily down:

$oc build-logs <build-name>
I1223 05:49:35.619426       1 sti.go:214] Using provided push secret for pushing 172.30.235.193:5000/xiazhao/origin-ruby-sample:latest image
I1223 05:49:35.619456       1 sti.go:218] Pushing 172.30.235.193:5000/xiazhao/origin-ruby-sample:latest image ...
I1223 05:50:10.665277       1 sti.go:223] Registry server Address:
I1223 05:50:10.665303       1 sti.go:224] Registry server User Name: serviceaccount
I1223 05:50:10.665313       1 sti.go:225] Registry server Email: serviceaccount
I1223 05:50:10.665321       1 sti.go:230] Registry server Password: <<non-empty>>
F1223 05:50:10.665333       1 builder.go:185] Error: build error: Failed to push image. Response from registry is: Error: Status 404 trying to push repository xiazhao/origin-ruby-sample: "404 page not found\n"

Expected results:
Result image should be pushed successfully to registry even if registry is temporarily down

Additional info:
The test environment has multiple node

Comment 1 Xia Zhao 2015-12-25 02:40:12 UTC

Issue did not repro on origin: devenv-rhel7_3032

Comment 2 Ben Parees 2016-01-04 15:13:08 UTC

We're not going to retry a 404 error from the registry, so i think this is working as designed.

Comment 3 Xia Zhao 2016-01-05 03:41:24 UTC

Verified on all-in-one OSE environment in order to avoid 404 error, build succeeded. Please resolve this bug as by design if OSE is designed not to treat with 404 error in this scenario. I'll then close it.

Thanks,
Xia

Comment 5 errata-xmlrpc 2016-01-26 19:20:41 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2016:0070