Bug 1373843 - OSE helloworld example app not working: getsockopt: connection refused
Summary: OSE helloworld example app not working: getsockopt: connection refused
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Quickstart Cloud Installer
Classification: Red Hat
Component: Installation - OpenShift
Version: 1.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 1.1
Assignee: Jason Montleon
QA Contact: James Olin Oden
Derek
URL:
Whiteboard:
: 1365992 1367173 1384103 1395584 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-07 09:06 UTC by Antonin Pagac
Modified: 2017-02-28 01:39 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-02-28 01:39:24 UTC
Target Upstream Version:


Attachments (Terms of Use)
output from 'oc logs build/php-helloworld-1' (9.09 KB, text/plain)
2016-09-07 09:06 UTC, Antonin Pagac
no flags Details
'oc describe pod php-helloworld-1-build' (2.74 KB, text/plain)
2016-09-07 09:09 UTC, Antonin Pagac
no flags Details
Output of OpenShift commands (2.62 KB, application/x-gzip)
2016-11-23 06:33 UTC, Landon LaSmith
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1365992 0 unspecified CLOSED Installation summary link for OSE example application is broken. 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1374903 0 unspecified CLOSED OSE failed at 95%, ansible: "unable to load template file" 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHEA-2017:0335 0 normal SHIPPED_LIVE Red Hat Quickstart Installer 1.1 2017-02-28 06:36:13 UTC

Internal Links: 1365992 1374903

Description Antonin Pagac 2016-09-07 09:06:53 UTC
Created attachment 1198605 [details]
output from 'oc logs build/php-helloworld-1'

Description of problem:
'curl hello-openshift.ose.example.com' gives me:

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>


From OSE master node, 'oc get pods':

NAME                     READY     STATUS    RESTARTS   AGE
php-helloworld-1-build   0/1       Error     0          13h


When logged as admin, 'oc get pods':

NAME                         READY     STATUS             RESTARTS   AGE
docker-registry-1-deploy     0/1       DeadlineExceeded   0          14h


Doing 'oc project helloworld' and then 'oc status':

http://hello-openshift.ose.example.com (svc/php-helloworld)
  dc/php-helloworld deploys istag/php-helloworld:latest <-
    bc/php-helloworld builds http://www.github.com/dymurray/php-helloworld.git with openshift/php:5.6 
      build #1 failed 14 hours ago - 914c47a: Changed outputted text (Dylan Murray <dymurray>)
    deployment #1 waiting on image or update

Errors:
  * build/php-helloworld-1 has failed.


Output from 'oc status -v':

In project helloworld on server https://rhv-ose-ose-master1.example.com:8443

http://hello-openshift.ose.example.com (svc/php-helloworld)
  dc/php-helloworld deploys istag/php-helloworld:latest <-
    bc/php-helloworld builds http://www.github.com/dymurray/php-helloworld.git with openshift/php:5.6 
      build #1 failed 14 hours ago - 914c47a: Changed outputted text (Dylan Murray <dymurray>)
    deployment #1 waiting on image or update

Errors:
  * build/php-helloworld-1 has failed.
    try: Inspect the build failure with 'oc logs build/php-helloworld-1'
Warnings:
  * The image trigger for dc/php-helloworld will have no effect until istag/php-helloworld:latest is imported or created by a build.
  * dc/php-helloworld has no readiness probe to verify pods are ready to accept traffic or ensure deployment is successful.
    try: oc set probe dc/php-helloworld --readiness ...

View details with 'oc describe <resource>/<name>' or list everything with 'oc get all'.


Please find complete output of 'oc logs build/php-helloworld-1' in attachment. Last line of that log is:

F0906 13:35:40.000834       1 builder.go:204] Error: build error: Failed to push image. Response from registry is: Put http://172.30.182.34:5000/v1/repositories/helloworld/php-helloworld/: dial tcp 172.30.182.34:5000: getsockopt: connection refused


I don't know why there is refused connection, I don't see into OSE networking yet.

Version-Release number of selected component (if applicable):
QCI-1.0-RHEL-7-20160902.5
QCIOOO-8.0-RHEL-7-20160902.1

How reproducible:
Sample app fail seen on pre-GA builds, investigated only today; unsure how reproducible

Steps to Reproduce:
1. Deploy OSE on RHV
2. While sshed to Satellite, run 'curl <URL of hello world app>'
3. 503 error returned.

Actual results:
503 error

Expected results:
200 OK, 'Hello OpenShift'

Additional info:

Comment 1 Antonin Pagac 2016-09-07 09:09:24 UTC
Created attachment 1198606 [details]
'oc describe pod php-helloworld-1-build'

Comment 2 Dylan Murray 2016-09-07 15:16:08 UTC
Spoke with Antonin and we are waiting to see if this can be reproduced. If it can, the log we need would be from the command 'oc logs bc/docker-registry'. Normally this error is due to OpenShift docker registry not running, but in this instance the docker-registry deployer failed before the registry even was created.

Comment 4 Dylan Murray 2016-09-08 19:59:52 UTC
Couldn't reproduce with the latest ISO on 9/8. I verified after deployment the registry was running and the sample application was accessible.

Comment 5 Antonin Pagac 2016-09-09 06:58:51 UTC
I finally managed to reproduce the issue with 20160902.5. The error message is the same.
Yesterday, when manually started, sample app was working. There is a suspicion for a timing issue. I'll try to reproduce using 9/8 ISO.

Comment 6 Antonin Pagac 2016-09-09 13:42:53 UTC
Reproduced also with QCI-1.0-RHEL-7-20160902.5.

Comment 7 Antonin Pagac 2016-09-09 13:44:26 UTC
Sorry, I meant QCI-1.0-RHEL-7-20160908.1

Comment 8 Antonin Pagac 2016-09-11 06:17:55 UTC
Upon manually starting the failing sample app, I get this error:

"Could not resolve host: www.github.com; Unknown error"

I then did these steps:

"[root@5-ose-master1 ~]# ping github.com
ping: unknown host github.com
[root@5-ose-master1 ~]# getent hosts github.com
192.30.253.113  github.com
[root@5-ose-master1 ~]# ping github.com
PING github.com (192.30.253.113) 56(84) bytes of data.
64 bytes from 192.30.253.113: icmp_seq=1 ttl=51 time=15.7 ms
...
[root@5-ose-master1 ~]# oc start-build php-helloworld"

The example app is now working and available. It seems for some reason the DNS is taking too long to respond on my bare metal setup.

Comment 10 Landon LaSmith 2016-11-23 06:31:53 UTC
Same issue seen on the QCI-1.1-RHEL-7-20161121.t.0 iso.  I have seen successes with the sample app on this iso version but this deployment took ~6hrs to deploy.

Comment 11 Landon LaSmith 2016-11-23 06:33:01 UTC
Created attachment 1222996 [details]
Output of OpenShift commands

Comment 12 Dylan Murray 2016-12-06 16:04:26 UTC
*** Bug 1365992 has been marked as a duplicate of this bug. ***

Comment 13 Dylan Murray 2016-12-06 16:05:25 UTC
*** Bug 1367173 has been marked as a duplicate of this bug. ***

Comment 14 Dylan Murray 2016-12-07 14:36:59 UTC
With sample application working as of 12/7 I am moving this to ON_QA in correlation with BZ 1384103.

Comment 15 James Olin Oden 2016-12-08 16:40:51 UTC
Still seems to be broken.  I'm getting a "503 Service Unavailable" when I click the link.

The compose is QCI-1.1-RHEL-7-20161206.t.0

Comment 16 Dylan Murray 2016-12-08 18:25:03 UTC
After investigating James' deployment, the master host is unable to mount the NFS share from the Satellite host without getting an "access denied". I am able to mount his share from my desktop which tells me there is some firewall issue or perhaps selinux issue blocking the master host from mounting.

Comment 17 James Olin Oden 2016-12-08 18:29:01 UTC
*** Bug 1384103 has been marked as a duplicate of this bug. ***

Comment 18 James Olin Oden 2016-12-08 18:44:13 UTC
*** Bug 1395584 has been marked as a duplicate of this bug. ***

Comment 20 James Olin Oden 2016-12-15 13:58:40 UTC
Verified in QCI-1.1-RHEL-7-20161212.t.2-QCI-x86_64-dvd1.iso

Comment 23 errata-xmlrpc 2017-02-28 01:39:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:0335


Note You need to log in before you can comment on or make changes to this bug.