Bug 1474715

Summary: Failed to start Kibana pod, permission denied to run run.sh
Product: OpenShift Container Platform Reporter: Junqi Zhao <juzhao>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED ERRATA QA Contact: Xia Zhao <xiazhao>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.6.0CC: aos-bugs, chuyu, jcantril, jwozniak, pweil, rmeggins, wsun
Target Milestone: ---Keywords: Regression, TestBlocker
Target Release: 3.7.0   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: The execute bit in the downstream repo was not set for run.sh Consequence: Fix: Result:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-28 22:05:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Junqi Zhao 2017-07-25 09:00:28 UTC
Description of problem:
Failed to start Kibana pod

# oc  get po
NAME                                      READY     STATUS             RESTARTS   AGE
logging-curator-1-9c8fh                   1/1       Running            0          10h
logging-es-data-master-j5cf6q7g-1-pq9wk   1/1       Running            0          10h
logging-fluentd-08xjz                     1/1       Running            0          10h
logging-fluentd-8dvzx                     1/1       Running            0          10h
logging-kibana-1-0mh0z                    1/2       CrashLoopBackOff   70         10h

# oc logs logging-kibana-1-0mh0z -c kibana
container_linux.go:247: starting container process caused "exec: \"./run.sh\": permission denied"

Version-Release number of selected component (if applicable):
# openshift version
openshift v3.6.169
kubernetes v1.6.1+5115d708d7
etcd 3.2.1

Images from brew registry
logging-kibana          v3.6.170-1          15078dfaeef7        2 hours ago         342.4 MB

How reproducible:

Steps to Reproduce:
1. Deploy logging 3.6.0

Actual results:
Failed to start Kibana pod

Expected results:
All pods should be healthy

Additional info:

Comment 1 Junqi Zhao 2017-07-25 09:14:47 UTC
Cases related to kibana are blocked

Comment 2 Jan Wozniak 2017-07-25 13:38:54 UTC
A 3.6 merge origin with downstream causes run.sh to have file permissions as 0644 (without executable bit) [1]

First parent of mentioned merge has ./run.sh permissions set to 0644 and entrypoint is CMD ["sh", "/run.sh"] [2] while second parent to 0755 and entrypoint is CMD ["./run.sh"] [3]. Merge became an unfortunate combination of these two approaches, the only non-runnable.

Someone with a write access to downstream repos could fix by simply 'chmod +x run.sh' and commit.

[1] http://pkgs.devel.redhat.com/cgit/rpms/logging-kibana-docker/commit/?h=rhaos-3.6-rhel-7&id=c2eeba2f087fdf280f9c4f0b17c14680fbb44aba

[2] http://pkgs.devel.redhat.com/cgit/rpms/logging-kibana-docker/tree/?h=rhaos-3.6-rhel-7&id=ce5c66327427d0e6af17fb76f4ad0a115f5a61af

[3] http://pkgs.devel.redhat.com/cgit/rpms/logging-kibana-docker/tree/?h=rhaos-3.6-rhel-7&id=5b709bf2f8b5a0319d63a52f303c30f831f47175

Comment 3 Rich Megginson 2017-07-25 13:48:42 UTC
Jan - you should have permission to do this.

Have you used the rhpkg command line tool?  If not, https://mojo.redhat.com/docs/DOC-140221

Once you have that installed, I can show you how to do this.

Comment 5 Xia Zhao 2017-07-28 06:57:10 UTC
It's fixed, verified with this latest image:
logging-kibana          v3.6                429262be200e        6 hours ago

logging UI is back with log entries inside. Set to verified.

Test env:
# openshift version
openshift v3.6.171
kubernetes v1.6.1+5115d708d7
etcd 3.2.1

Comment 9 errata-xmlrpc 2017-11-28 22:05:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.