Bug 1323122

Summary:	[Intservice_public_196] The ES copy settings can not be recovered when logging is deployed without OPS clusters
Product:	OKD	Reporter:	Xia Zhao <xiazhao>
Component:	Logging	Assignee:	Rich Megginson <rmeggins>
Status:	CLOSED CURRENTRELEASE	QA Contact:	chunchen <chunchen>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	3.x	CC:	aos-bugs, jcantril, lmeyer, wsun
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-05-12 17:10:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Xia Zhao 2016-04-01 10:30:36 UTC

Description of problem:
The ES copy test script run failed without OPS clusters, full logs: http://pastebin.test.redhat.com/361648

It passed when OPS clusters are deployed in logging project. Passed log here: http://pastebin.test.redhat.com/361714

Version-Release number of selected component (if applicable):
openshift v1.1.5-66-g9b04353
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5

How reproducible:
Always

Steps to Reproduce:
1. Deploy CEFK stacks without OPS cluster, after all logging pods are running, check that fluentd pod logs did not contain errors
2. Prepare frontend service in -n test
$ oc new-project test
$ oc new-app https://raw.githubusercontent.com/openshift/origin/master/examples/sample-app/application-template-stibuild.json
Wait for the 2 frontend pods running:
$ oc get po | grep frontend
frontend-1-s4sqz            1/1       Running     0          3m
frontend-1-t7kv5            1/1       Running     0          3m
3. Specify --config argument in env :
$ export KUBECONFIG=/etc/origin/master/admin.kubeconfig
4. Add the oc client user to cluster admin
$ oadm policy add-cluster-role-to-user cluster-admin xiazhao
5. Go to openshift master node, oc login to openshift with user xiazhao,  and switched to the logging project, then run the test script:
wget https://raw.githubusercontent.com/openshift/origin-aggregated-logging/master/hack/testing/test-es-copy.sh
chmod 755 test-es-copy.sh
VERBOSE=1 ./test-es-copy.sh false

Actual results:
The ES copy settings was not recovered, test script run failed without OPS clusters, full logs: http://pastebin.test.redhat.com/361648

Expected results:
The ES copy settings should recover

Additional info:

Comment 1 Luke Meyer 2016-04-05 17:14:42 UTC

Rich is looking into this today.

Comment 2 Rich Megginson 2016-04-06 22:51:41 UTC

Fix at https://github.com/openshift/origin-aggregated-logging/pull/112

Comment 3 Xia Zhao 2016-04-08 09:55:36 UTC

Tested with the new script, it got passed. Please change this issue to ON_QA, and I will accept the Trello card after it got tc-approved.

Comment 4 Xia Zhao 2016-04-11 01:56:51 UTC

Verified the issue with https://raw.githubusercontent.com/richm/origin-aggregated-logging/es-copy-improve/hack/testing/test-es-copy.sh, test script got passed. Set this to verified.

Thanks,
Xia