Bug 1458652

Summary: kibana container can't be started because readiness probe failed
Product: OpenShift Container Platform Reporter: Xia Zhao <xiazhao>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED DUPLICATE QA Contact: Xia Zhao <xiazhao>
Severity: urgent Docs Contact:
Priority: high    
Version: 3.6.0CC: aos-bugs, mifiedle, wsun, xtian
Target Milestone: ---Keywords: TestBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-06-14 08:59:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Xia Zhao 2017-06-05 06:15:37 UTC 
Description of problem:
Deploy logging 3.6.0 on OCP, after other components are all started up, the kibana container can't be started because readiness probe failed:

# oc get event | grep kibana
...
Readiness probe failed: rpc error: code = 13 desc = invalid header field value "oci runtime error: exec failed: container_linux.go:247: starting container process caused \"exec: \\\"/usr/share/kibana/probe/readiness.sh\\\": stat /usr/share/kibana/probe/readiness.sh: no such file or directory\"\n"

# oc get po
NAME                                      READY     STATUS    RESTARTS   AGE
logging-curator-1-vs9sj                   1/1       Running   0          6m
logging-es-data-master-lfw1lt94-1-8k1vg   1/1       Running   0          6m
logging-fluentd-lkv5r                     1/1       Running   0          6m
logging-fluentd-zkfxz                     1/1       Running   0          6m
logging-kibana-1-7bdgz                    1/2       Running   0          6m
logging-kibana-1-deploy                   1/1       Running   0          6m

Version-Release number of selected component (if applicable):
openshift3/logging-kibana             v3.6                b2ee235a5512        2 days ago          342.4 MB

openshift-ansible-playbooks-3.6.94-1.git.0.fff177b.el7.noarch

# openshift version
openshift v3.6.85
kubernetes v1.6.1+5115d708d7
etcd 3.1.0

How reproducible:
Always

Steps to Reproduce:
1.Deploy logging 3.6.0 on OCP
2.After other components are all started up, check kibana pod status


Actual results:
2.the kibana container can't be started because readiness probe failed

Expected results:
2.kibana should be started up successfully

Additional info:
Comment 1 Xia Zhao 2017-06-05 08:16:28 UTC 
This issue blocked logging UI tests.
Comment 2 Mike Fiedler 2017-06-05 12:21:18 UTC 
Likely the same as https://bugzilla.redhat.com/show_bug.cgi?id=1452807
Comment 3 Xia Zhao 2017-06-06 02:50:13 UTC 
(In reply to Mike Fiedler from comment #2)
> Likely the same as https://bugzilla.redhat.com/show_bug.cgi?id=1452807

Yes, I think so.
Comment 4 Mike Fiedler 2017-06-13 11:46:52 UTC 
Marking this urgent since logging UI testing is still blocked for 3.6