Bug 1579675

Summary: [free-int] Failed to connect to docker registry
Product: OpenShift Online Reporter: XiaochuanWang <xiaocwan>
Component: WebsiteAssignee: Abhishek Gupta <abhgupta>
Status: VERIFIED --- QA Contact: wangyu <yuwan>
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: aos-bugs, yasun, yufchang
Target Milestone: ---Keywords: OnlineStarter
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
scalegroup nodes are unable to pull image none

Description XiaochuanWang 2018-05-18 06:35:43 UTC 
Description of problem:
Some templates failed to push images

Version-Release number of selected component (if applicable):
OpenShift Master:  v3.10.0-0.47.0 (online version 3.6.0.83)
Kubernetes Master: v1.10.0+b81c8f8
OpenShift Web Console: v3.10.0-0.47.0

How reproducible:
Always

Steps to Reproduce:
1. $ oc new-app --template=laravel-mysql-persistent 
2. $ oc logs -f laravel-mysql-persistent-1-build
3.

Actual results:
"""
laravel/framework suggests installing symfony/psr-http-message-bridge (Required to psr7 bridging features (0.2.*).)
sebastian/global-state suggests installing ext-uopz (*)
phpunit/php-code-coverage suggests installing ext-xdebug (>=2.2.1)
phpunit/phpunit suggests installing phpunit/php-invoker (~1.1)
Generating optimized autoload files
> Illuminate\Foundation\ComposerScripts::postInstall
> php artisan optimize
Generating optimized class loader
=> sourcing 20-copy-config.sh ...
---> 06:17:36     Processing additional arbitrary httpd configuration provided by s2i ...
=> sourcing 00-documentroot.conf ...
=> sourcing 50-mpm-tuning.conf ...
=> sourcing 40-ssl-certs.sh ...

Pushing image 172.30.215.46:5000/xiaocwan-inta2/laravel-mysql-persistent:latest ...
Warning: Push failed, retrying in 5s ...
Warning: Push failed, retrying in 5s ...
Registry server Address: 
Registry server User Name: serviceaccount
Registry server Email: serviceaccount
Registry server Password: <<non-empty>>
error: build error: Failed to push image: After retrying 6 times, Push image still failed due to error: Get https://172.30.215.46:5000/v1/_ping: dial tcp 172.30.215.46:5000: getsockopt: connection refused

"""

Expected results:
Should push successfully.

Additional info:
Seem also even failed on docker login and docker pull.
$ docker login registry.free-int.openshift.com -u $(oc whoami) -p $(oc whoami -t)
Warning: failed to get default registry endpoint from daemon (Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.26/info: dial unix /var/run/docker.sock: connect: permission denied). Using system default: https://index.docker.io/v1/
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.26/auth: dial unix /var/run/docker.sock: connect: permission denied
Comment 2 Stefanie Forrester 2018-07-23 18:29:04 UTC 
Created attachment 1470025 [details]
scalegroup nodes are unable to pull image

I checked over this in free-int today. My current theory is that scalegroup nodes are missing the docker registry pull credentials, and are unable to pull the builder image. See attached log for details.
Comment 4 yasun 2018-07-24 02:49:24 UTC 
On QEs' side, pulling/pushing image is successful. But we have no authority to test all the nodes. @Stefanie, could you mind rerun your script to get a perfect testing result?
Comment 5 yasun 2018-07-24 05:56:58 UTC 
I have tested all the compute nodes, pulling/pushing image is successful.
Comment 6 Stefanie Forrester 2018-07-24 15:09:50 UTC 
All nodes are able to pull now.