Description of problem: The install fails when trying to install logging on a 3.11 OCP cluster. Version-Release number of selected component (if applicable): oc v3.11.0-0.14.0 kubernetes v1.11.0+d4cacc0 features: Basic-Auth GSSAPI Kerberos SPNEGO openshift v3.11.0-0.14.0 kubernetes v1.11.0+d4cacc0 How reproducible: Always Steps to Reproduce: 1. Install logging using openshift-ansible playbook. ansible-playbook -i inventory openshift-ansible/playbooks/openshift-logging/config.yml -- inventory file [OSEv3:vars] deployment_type=openshift-enterprise openshift_deployment_type=openshift-enterprise openshift_release=v3.11 openshift_docker_additional_registries=registry.reg-aws.openshift.com oreg_url=registry.reg-aws.openshift.com:443/openshift3/ose-${component}:${version} openshift_logging_image_version=v3.11 openshift_logging_install_logging=true openshift_logging_es_cluster_size=1 openshift_logging_es_pvc_dynamic=true openshift_logging_es_pvc_size=50Gi openshift_logging_es_pvc_storage_class_name=gp2 openshift_logging_fluentd_read_from_head=false openshift_logging_curator_nodeselector={"node-role.kubernetes.io/infra": "true"} openshift_logging_kibana_nodeselector={"node-role.kubernetes.io/infra": "true"} openshift_logging_es_nodeselector={"node-role.kubernetes.io/infra": "true"} openshift_logging_es_memory_limit=8Gi openshift_logging_install_eventrouter=true Actual results: The install fails at task TASK [Evaluate oo_elasticsearch_nodes] ************************************************************************************************************************************** skipping: [ec2-34-217-107-181.us-west-2.compute.amazonaws.com] => (item=ec2-34-217-107-181.us-west-2.compute.amazonaws.com) fatal: [ec2-34-217-107-181.us-west-2.compute.amazonaws.com]: FAILED! => {"msg": "The conditional check 'hostvars[item]['openshift']['common']['ip'] in openshift_logging_elas ticsearch_hosts' failed. The error was: error while evaluating conditional (hostvars[item]['openshift']['common']['ip'] in openshift_logging_elasticsearch_hosts): 'ansible.v ars.hostvars.HostVarsVars object' has no attribute 'openshift'\n\nThe error appears to have been in '/root/openshift-ansible/playbooks/openshift-logging/private/config.yml': line 82, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n # files for specific nodes based on <node>.status.addresses[@.type==InternalIP].address\n - name: Evaluate oo_elasticsearch_nodes\n ^ here\n"} Description of problem: Version-Release number of the following components: rpm -q openshift-ansible rpm -q ansible ansible --version How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag
Eric - any ideas?
Can you provide the full output of running the logging playbook with -vvv?
Created attachment 1476156 [details] The ansible log with -vvv
The fact collection looks to have those structures in them... Siva, What does your inventory group section look like? ([masters], [nodes], etc)?
Created attachment 1476167 [details] inventory file
I think that task is failing when its evaluating the etcd group since it is not a node. If so then the fix would be to only iterate over the list of nodes, not groups['OSEv3']. Will try to recreate this locally.
Eric, Yes that looks like the case. I made the etcd group empty and re-ran the playbook. it finished successfully.
Siva, that's what I observed locally as well. The plan is to update the loop to only iterate over the list of node hosts; I will open up a PR with the fix afterwards.
https://github.com/openshift/openshift-ansible/pull/9606
Verified this bug on openshift v3.11.0-0.20.0 oc v3.11.0-0.19.0 kubernetes v1.11.0+d4cacc0 features: Basic-Auth GSSAPI Kerberos SPNEGO openshift-ansible git log # cd openshift-ansible # git log commit d003bdf30c23a47f485f9f1f394a3d247c88da06 Merge: ac09fc2 921df2b Author: OpenShift Merge Robot <openshift-merge-robot.github.com> Date: Thu Aug 23 08:56:20 2018 -0700 The test was performed on a OCP cluster which had split etcd and master. And etcd was not a node. THe installation completed successfully. # oc project Using project "openshift-logging" on server # oc get pods NAME READY STATUS RESTARTS AGE logging-es-data-master-lzw28btf-1-5xpt7 2/2 Running 0 6m logging-fluentd-4lz76 1/1 Running 0 6m logging-fluentd-gl8hm 1/1 Running 0 6m logging-fluentd-n4dfw 1/1 Running 0 6m logging-fluentd-qqn2j 1/1 Running 0 6m logging-kibana-1-vc9v7 2/2 Running 0 7m
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:2652