Bug 1276235

Summary: [AEP]Got deployer pod Error when deploy logging on AEP
Product: OKD Reporter: wyue
Component: LoggingAssignee: Luke Meyer <lmeyer>
Status: CLOSED CURRENTRELEASE QA Contact: chunchen <chunchen>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.xCC: aos-bugs, wsun, xtian
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-23 21:17:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
result of 'oc logs'
none
result of 'oc describe' none

Description wyue 2015-10-29 08:01:00 UTC 
Description of problem:
Pod 'logging-deployer-spi6f' gets into Error status when deploying on AEP according to docs:
https://github.com/openshift/origin-aggregated-logging/tree/master/deployment


Version-Release number of selected component (if applicable):
oc v3.0.2.903-114-g2849767
openshift v3.0.2.903-114-g2849767
kubernetes v1.2.0-alpha.1-1107-g4c8e6f4
etcd 2.1.2


How reproducible:
Always

Steps to Reproduce:
1.oc new-project logging
2.oc secrets new logging-deployer nothing=/dev/null
3.oc create -f - <<API
apiVersion: v1
kind: ServiceAccount
metadata:
  name: logging-deployer
secrets:
- name: logging-deployer
API

oc policy add-role-to-user edit \
          system:serviceaccount:logging:logging-deployer
4. oc process -f deployer.yaml -v IMAGE_PREFIX=rcm-img-docker01.build.eng.bos.redhat.com:5001/openshift3/,KIBANA_HOSTNAME=kibana.example.com,PUBLIC_MASTER_URL=https://xxxx:8443,ES_INSTANCE_RAM=1024M,ES_CLUSTER_SIZE=1 | oc create -f -

Actual results:
[root@dhcp-128-16 sample-app]# oc get pods
NAME                     READY     STATUS    RESTARTS   AGE
logging-deployer-spi6f   0/1       Error     0          27m


Expected results:
deployer pod should complete successfully

Additional info:
attachments are results of 'oc logs' and 'oc describe pods'
Comment 1 wyue 2015-10-29 08:02:18 UTC 
Created attachment 1087398 [details]
result of 'oc logs'
Comment 2 wyue 2015-10-29 08:02:48 UTC 
Created attachment 1087399 [details]
result of 'oc describe'
Comment 3 Luke Meyer 2015-10-29 11:52:33 UTC 
From pod log:

+ oc delete all --selector logging-infra=kibana
Unable to find "buildconfigs" that match the selector "logging-infra=kibana": the server could not find the requested resource
Unable to find "builds" that match the selector "logging-infra=kibana": the server could not find the requested resource

I would consider this an AEP bug as evidently the "all" alias still includes buildconfigs and builds which are removed from the resource mapper.

We can, however, work around this in the deployer by being more explicit about what we're deleting.
Comment 4 Luke Meyer 2015-10-29 12:33:08 UTC 
Actually the problem is using the OSE client against AEP. Apparently the client is what translates "all" to its individual components. The workaround stands.
Comment 6 chunchen 2015-11-02 05:44:22 UTC 
It's fixed, checked against latest AEP env, so mark it as verified.

Please refer to the below messages:

[root@openshift-161 ~]# oc version
oc v3.0.2.905
kubernetes v1.2.0-alpha.1-1107-g4c8e6f4


[chunchen@F17-CCY daily]$ oc logs logging-deployer-c18r5
<------------snip------------>
+ oc create -f -
+ oc process logging-support-pre-template
serviceaccount "aggregated-logging-kibana" created
serviceaccount "aggregated-logging-elasticsearch" created
serviceaccount "aggregated-logging-fluentd" created
service "logging-es" created
service "logging-es-cluster" created
service "logging-es-ops" created
service "logging-es-ops-cluster" created
service "logging-kibana" created
service "logging-kibana-ops" created
+ oc delete dc,rc,pod --selector logging-infra=kibana
No resources found
+ oc delete dc,rc,pod --selector logging-infra=fluentd
No resources found
+ oc delete dc,rc,pod --selector logging-infra=elasticsearch
No resources found
+ (( n=0 ))
+ (( n<1 ))
+ oc process logging-es-template
+ oc create -f -
deploymentconfig "logging-es-kla3hsdv" created
+ (( n++ ))
+ (( n<1 ))
+ oc process logging-fluentd-template
+ oc create -f -
deploymentconfig "logging-fluentd" created
+ oc process logging-kibana-template
+ oc create -f -
deploymentconfig "logging-kibana" created
Success!
+ '[' false == true ']'
+ set +x

=================================