Description of problem: Install glusterfs on an existing gluster. The playbook failed. Version-Release number of the following components: rpm -q openshift-ansible [fedora@ip-172-31-33-174 openshift-ansible]$ git log --oneline -1 cddde04a8 (HEAD -> master, origin/master, origin/HEAD) Merge pull request #5735 from ingvagabund/default-oo_new_etcd_to_config-to-empty-list rpm -q ansible ansible-2.3.2.0-1.fc26.noarch ansible --version ansible 2.3.2.0 config file = /home/fedora/openshift-ansible/ansible.cfg configured module search path = Default w/o overrides python version = 2.7.13 (default, Sep 5 2017, 08:53:59) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)] How reproducible: 1 master, 1 infra, 2 compute nodes Steps to Reproduce: 1. openshift-ansible/playbooks/byo/config.yml to install openshift cluster without glusterfs 2. add glusterfs relevant parts into inv. file 3. openshift-ansible/playbooks/byo/openshift-glusterfs/config.yml to install glusterfs Actual results: 2017-10-13 19:18:39,722 p=14236 u=fedora | TASK [openshift_storage_glusterfs : Deploy deploy-heketi pod] ***************************************************************** 2017-10-13 19:18:39,722 p=14236 u=fedora | task path: /home/fedora/openshift-ansible/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part1.yml:18 2017-10-13 19:18:39,758 p=14236 u=fedora | Using module file /home/fedora/openshift-ansible/roles/lib_openshift/library/oc_process.py 2017-10-13 19:18:40,571 p=14236 u=fedora | fatal: [ec2-54-213-229-145.us-west-2.compute.amazonaws.com]: FAILED! => { "changed": false, "failed": true, "module_stderr": "Shared connection to ec2-54-213-229-145.us-west-2.compute.amazonaws.com closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1635, in <module>\r\n main()\r\n File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1628, in main\r\n rval = OCProcess.run_ansible(module.params, module.check_mode)\r\n File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1584, in run_ansible\r\n all_results = ocprocess.needs_update()\r\n File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1503, in needs_update\r\n for obj in self.template:\r\n File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1449, in template\r\n raise OpenShiftCLIError('Error processing template [%s]: %s' %(self.name, results))\r\n__main__.OpenShiftCLIError: Error processing template [deploy-heketi]: {'cmd': '/usr/bin/oc process deploy-heketi -v HEKETI_USER_KEY=/trpuz0MMFHWuPZqq0YX2sUL1xya9ybbLaraKpLWyMU= IMAGE_VERSION=latest HEKETI_KUBE_NAMESPACE=glusterfs HEKETI_ADMIN_KEY=2zTf7a3qmLFy8/1pqRKT5JrQ8I7HK9dP+bBgksNkmXs= IMAGE_NAME=rhgs3/rhgs-volmanager-rhel7 CLUSTER_NAME=storage HEKETI_ROUTE=heketi-storage HEKETI_EXECUTOR=kubernetes -n glusterfs', 'returncode': 1, 'results': {}, 'stderr': u'Flag --value has been deprecated, Use -p, --param instead.\\nerror: unable to process template\\n Required value: template.parameters[8]: parameter TOPOLOGY_PATH is required and must be specified\\n', 'stdout': u''}\r\n", "msg": "MODULE FAILURE", "rc": 0 } Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag
PR is upstream: https://github.com/openshift/openshift-ansible/pull/5667
Verified with: [fedora@ip-172-31-33-174 openshift-ansible]$ git log --oneline -1 b8879b5c6 (HEAD -> glusterfs) GlusterFS: remove topology reference from deploy-heketi root@ip-172-31-15-103: ~ # oc version oc v3.7.0-0.153.0 It passed the tasked of creating heketi pod in this bug, but got error later on TASK [openshift_storage_glusterfs : Create heketi DB volume]. I will attach the log later.
PR for latest bug is upstream: https://github.com/openshift/openshift-ansible/pull/5769
Verified: Thanks for the quick fix. [fedora@ip-172-31-33-174 openshift-ansible]$ git log --oneline -2 3d3853836 (HEAD -> glusterfs2) GlusterFS: Remove image option from heketi command 27321df00 (origin/master, origin/HEAD, master) Merge pull request #5667 from jarrpa/deploy-heketi-fix-3.7 root@ip-172-31-11-69: ~ # oc version oc v3.7.0-0.153.0 [fedora@ip-172-31-33-174 ~]$ ansible-playbook -i /tmp/2.file openshift-ansible/playbooks/byo/openshift-glusterfs/config.yml PLAY RECAP ******************************************************************************************************************** ec2-54-186-162-10.us-west-2.compute.amazonaws.com : ok=47 changed=4 unreachable=0 failed=0 ec2-54-187-203-117.us-west-2.compute.amazonaws.com : ok=47 changed=4 unreachable=0 failed=0 ec2-54-200-62-253.us-west-2.compute.amazonaws.com : ok=47 changed=4 unreachable=0 failed=0 ec2-54-202-151-185.us-west-2.compute.amazonaws.com : ok=107 changed=38 unreachable=0 failed=0 ec2-54-213-59-28.us-west-2.compute.amazonaws.com : ok=44 changed=2 unreachable=0 failed=0 localhost : ok=12 changed=0 unreachable=0 failed=0 INSTALLER STATUS ************************************************************************************************************** Initialization : Complete etcd Install : Not Started NFS Install : Not Started Load balancer Install : Not Started Master Install : Not Started Master Additional Install : Not Started Node Install : Not Started GlusterFS Install : Complete Hosted Install : Not Started Metrics Install : Not Started Logging Install : Not Started Service Catalog Install : Not Started Management Install : Not Started
Jose A. Rivera: another thought. I noticed that "GlusterFS Install" is before "Hosted Install" in the playbook summary (see Comment 7). Should it be the other way around like Metrics or Logging install? I did verification of the fix on an existing gluster. During the tests on a new gluster last Friday, glusterfs failed on very early stage. My guess would be that it depends on tasks in "Hosted Install". That was why I stepped back and started to install on an existing cluster. Let me know what you think about the ordering of the playbook.
If I remember correctly, "Hosted Install" in this case is for the hosted registry. In this case, GlusterFS MUST come first if it is to provide storage for the hosted registry, and indeed for Logging and Metrics as well.
I see. Thanks for the explanation.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:3188