Bug 1469918 - The searchguard plugin script is missing in the latest elasticsearch image
The searchguard plugin script is missing in the latest elasticsearch image
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging (Show other bugs)
Unspecified Unspecified
high Severity high
: ---
: 3.7.0
Assigned To: Jeff Cantrill
Xia Zhao
: Regression, TestBlocker
Depends On:
  Show dependency treegraph
Reported: 2017-07-12 01:39 EDT by Xia Zhao
Modified: 2017-11-28 17:00 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-11-28 17:00:46 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
es_log (944.91 KB, text/plain)
2017-07-12 01:39 EDT, Xia Zhao
no flags Details

  None (edit)
Description Xia Zhao 2017-07-12 01:39:14 EDT
Created attachment 1296734 [details]

Description of problem:
elasticsearch not able to start up since searchguard not yet initialized :
$ oc logs logging-es-data-master-bru9a1e0-1-9rp80
[2017-07-12 02:43:13,002][INFO ][container.run            ] Seeding the searchguard ACL index.  Will wait up to 604800 seconds.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IOError: [Errno 2] No such file or directory: '/usr/share/elasticsearch/config/elasticsearch.yml'
/usr/local/bin/es_seed_acl: line 16: /usr/share/elasticsearch/plugins/openshift-elasticsearch/sgadmin.sh: No such file or directory
[2017-07-12 02:43:13,116][WARN ][container.run            ] Error seeding the searchguard ACL index... retrying in 10 seconds - 0 retries so far
[2017-07-12 02:43:13,117][WARN ][container.run            ] Seeding will continue to fail until the cluster status is YELLOW
[2017-07-12 02:43:13,227][INFO ][container.run            ] Remaining red indices: 0
/usr/local/bin/es_seed_acl: line 16: /usr/share/elasticsearch/plugins/openshift-elasticsearch/sgadmin.sh: No such file or directory
[2017-07-12 05:17:24,754][ERROR][com.floragunn.searchguard.auth.BackendRegistry] Not yet initialized (you may need to run sgadmin)
/usr/local/bin/es_seed_acl: line 16: /usr/share/elasticsearch/plugins/openshift-elasticsearch/sgadmin.sh: No such file or directory

Went into container, found that directory "/usr/share/elasticsearch/plugins/" is missing:
# oc rsh logging-es-data-master-bru9a1e0-1-9rp80
sh-4.2$ cd /usr/share/elasticsearch/    
sh-4.2$ ls -al
total 4
drwxr-xr-x.  6 root root  110 Jul 11 17:19 .
drwxr-xr-x. 74 root root 4096 Jul 11 17:19 ..
lrwxrwxrwx.  1 root root   29 Jul 11 17:19 elasticsearch -> /usr/share/java/elasticsearch
drwxr-xr-x.  2 root root   58 Jul 11 17:17 index_patterns
drwxr-xr-x.  2 root root  164 Jul 11 17:17 index_templates
drwxr-xr-x.  2 root root   86 Jul 11 17:17 kibana_ui_objects
drwxr-xr-x.  2 root root   26 Jul 11 17:17 probe

Version-Release number of selected component (if applicable):
logging-elasticsearch   v3.6                bcacb3cf0505        12 hours ago        404.2 MB

# openshift version
openshift v3.6.140
kubernetes v1.6.1+5115d708d7
etcd 3.2.1

How reproducible:

Steps to Reproduce:
1.Deploy logging 3.6.0
2.Check efk status

Actual results:
elasticsearch not able to start up since searchguard not yet initialized:
$ oc get po
NAME                                      READY     STATUS    RESTARTS   AGE
logging-curator-1-j8qwf                   1/1       Running   29         2h
logging-es-data-master-bru9a1e0-1-9rp80   1/1       Running   0          2h
logging-fluentd-dngrh                     1/1       Running   0          2h
logging-fluentd-jbhb4                     1/1       Running   1          2h
logging-kibana-1-4mdqf                    2/2       Running   0          2h

Expected results:
es should work well

Additional info:
Full es log attached
Comment 1 Xia Zhao 2017-07-12 01:47:35 EDT
Blocks all logging 3.6.0 tests.
Comment 4 Xia Zhao 2017-07-12 22:32:37 EDT
It's fixed and logging system are back working fine. Set to verified:

Images verified with:
logging-elasticsearch   v3.6                ca1c9074bf99        6 hours ago         404.7 MB
Comment 9 errata-xmlrpc 2017-11-28 17:00:46 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.