Bug 1332285

Summary: image push after build taking 30+ minutes
Product: OpenShift Online Reporter: Joel Diaz <jdiaz>
Component: Image RegistryAssignee: Michal Minar <miminar>
Status: CLOSED DUPLICATE QA Contact: Wei Sun <wsun>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.xCC: aos-bugs, jokerman, mmccomas, vdinh
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-06 21:08:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1314381, 1382795    
Bug Blocks:    

Description Joel Diaz 2016-05-02 19:17:09 UTC 
Description of problem:
A number of times, we've seen our OpenShift cluster get into a state where a container push to the image registry takes an extraordinary amount of time.

Restarting the docker-registry pod gets things up to a normal speed.

Version-Release number of selected component (if applicable):
[root@ops-master-f58e0 ~]# rpm -qa | grep atomic-openshift
atomic-openshift-sdn-ovs-3.1.0.4-1.git.15.5e061c3.el7aos.x86_64
atomic-openshift-utils-3.0.13-1.git.0.5e8c5c7.el7aos.noarch
atomic-openshift-clients-3.1.0.4-1.git.15.5e061c3.el7aos.x86_64
atomic-openshift-master-3.1.0.4-1.git.15.5e061c3.el7aos.x86_64
atomic-openshift-3.1.0.4-1.git.15.5e061c3.el7aos.x86_64
tuned-profiles-atomic-openshift-node-3.1.0.4-1.git.15.5e061c3.el7aos.x86_64
atomic-openshift-node-3.1.0.4-1.git.15.5e061c3.el7aos.x86_64


How reproducible:
Unknown

Steps to Reproduce:
1. Let OpenShift run uninterrupted for days
2. Attempt a build/push of a new image
3.

Actual results:
Step 19 : LABEL "io.openshift.build.commit.author" "openshift_ops_ci \u003clibra-ops\u003e" "io.openshift.build.commit.date" "Fri Apr 22 17:09:59 2016 -0400" "io.openshift.build.commit.id" "8b8bddde67ca1fcbad29470f9cce27188609b84c" "io.openshift.build.commit.ref" "stg" "io.openshift.build.commit.message" "Prod into stage by openshift-ops-bot." "io.openshift.build.source-location" "https://github.com/openshift/openshift-tools" "io.openshift.build.source-context-dir" "docker/oso-zabbix-server/rhel7"
 ---> Running in a224cbac5515
 ---> c6ecf636ebea
Removing intermediate container a224cbac5515
Successfully built c6ecf636ebea
I0425 10:54:04.881328       1 docker.go:86] Pushing image 172.30.157.196:5000/monitoring/oso-rhel7-zabbix-server:latest ...
I0425 11:31:35.922826       1 docker.go:90] Push successful



Expected results (after a fresh restart of the docker-registry pod):
Step 19 : LABEL "io.openshift.build.commit.id" "8b8bddde67ca1fcbad29470f9cce27188609b84c" "io.openshift.build.commit.ref" "stg" "io.openshift.build.commit.message" "Prod into stage by openshift-ops-bot." "io.openshift.build.source-location" "https://github.com/openshift/openshift-tools" "io.openshift.build.source-context-dir" "docker/oso-zabbix-server/rhel7" "io.openshift.build.commit.author" "openshift_ops_ci \u003clibra-ops\u003e" "io.openshift.build.commit.date" "Fri Apr 22 17:09:59 2016 -0400"
 ---> Running in be3d4214638e
 ---> ca928bf2122f
Removing intermediate container be3d4214638e
Successfully built ca928bf2122f
I0425 11:47:16.955878       1 docker.go:86] Pushing image 172.30.157.196:5000/monitoring/oso-rhel7-zabbix-server:latest ...
I0425 11:48:41.232239       1 docker.go:90] Push successful


Additional info:
Comment 1 Vu Dinh 2016-05-05 13:46:23 UTC 
I think we already have a bug for this issue:

https://bugzilla.redhat.com/show_bug.cgi?id=1314381

Vu
Comment 2 Joel Diaz 2016-05-05 16:24:02 UTC 
(In reply to Vu Dinh from comment #1)
> I think we already have a bug for this issue:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1314381

Yep, that does sound like what I was seeing.
Comment 3 Dan McPherson 2016-05-06 21:08:56 UTC 

*** This bug has been marked as a duplicate of bug 1314381 ***