Bug 1502054 - glusterfs installation failed: parameter TOPOLOGY_PATH is required and must be specified
Summary: glusterfs installation failed: parameter TOPOLOGY_PATH is required and must b...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 3.7.0
Assignee: Jose A. Rivera
QA Contact: Hongkai Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-10-13 20:07 UTC by Hongkai Liu
Modified: 2017-11-28 22:17 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-11-28 22:17:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:3188 0 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update 2017-11-29 02:34:54 UTC

Description Hongkai Liu 2017-10-13 20:07:10 UTC
Description of problem:
Install glusterfs on an existing gluster. The playbook failed.

Version-Release number of the following components:
rpm -q openshift-ansible
[fedora@ip-172-31-33-174 openshift-ansible]$ git log --oneline -1
cddde04a8 (HEAD -> master, origin/master, origin/HEAD) Merge pull request #5735 from ingvagabund/default-oo_new_etcd_to_config-to-empty-list

rpm -q ansible
ansible-2.3.2.0-1.fc26.noarch

ansible --version
ansible 2.3.2.0
  config file = /home/fedora/openshift-ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Sep  5 2017, 08:53:59) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]


How reproducible:
1 master, 1 infra, 2 compute nodes


Steps to Reproduce:
1. openshift-ansible/playbooks/byo/config.yml to install openshift cluster without glusterfs
2. add glusterfs relevant parts into inv. file
3. openshift-ansible/playbooks/byo/openshift-glusterfs/config.yml to install glusterfs

Actual results:
2017-10-13 19:18:39,722 p=14236 u=fedora |  TASK [openshift_storage_glusterfs : Deploy deploy-heketi pod] *****************************************************************
2017-10-13 19:18:39,722 p=14236 u=fedora |  task path: /home/fedora/openshift-ansible/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part1.yml:18
2017-10-13 19:18:39,758 p=14236 u=fedora |  Using module file /home/fedora/openshift-ansible/roles/lib_openshift/library/oc_process.py
2017-10-13 19:18:40,571 p=14236 u=fedora |  fatal: [ec2-54-213-229-145.us-west-2.compute.amazonaws.com]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "Shared connection to ec2-54-213-229-145.us-west-2.compute.amazonaws.com closed.\r\n", 
    "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1635, in <module>\r\n    main()\r\n  File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1628, in main\r\n    rval = OCProcess.run_ansible(module.params, module.check_mode)\r\n  File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1584, in run_ansible\r\n    all_results = ocprocess.needs_update()\r\n  File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1503, in needs_update\r\n    for obj in self.template:\r\n  File \"/tmp/ansible_jK4C_k/ansible_module_oc_process.py\", line 1449, in template\r\n    raise OpenShiftCLIError('Error processing template [%s]: %s' %(self.name, results))\r\n__main__.OpenShiftCLIError: Error processing template [deploy-heketi]: {'cmd': '/usr/bin/oc process deploy-heketi -v HEKETI_USER_KEY=/trpuz0MMFHWuPZqq0YX2sUL1xya9ybbLaraKpLWyMU= IMAGE_VERSION=latest HEKETI_KUBE_NAMESPACE=glusterfs HEKETI_ADMIN_KEY=2zTf7a3qmLFy8/1pqRKT5JrQ8I7HK9dP+bBgksNkmXs= IMAGE_NAME=rhgs3/rhgs-volmanager-rhel7 CLUSTER_NAME=storage HEKETI_ROUTE=heketi-storage HEKETI_EXECUTOR=kubernetes -n glusterfs', 'returncode': 1, 'results': {}, 'stderr': u'Flag --value has been deprecated, Use -p, --param instead.\\nerror: unable to process template\\n  Required value: template.parameters[8]: parameter TOPOLOGY_PATH is required and must be specified\\n', 'stdout': u''}\r\n", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}


Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 3 Jose A. Rivera 2017-10-13 20:28:00 UTC
PR is upstream: https://github.com/openshift/openshift-ansible/pull/5667

Comment 4 Hongkai Liu 2017-10-16 13:52:46 UTC
Verified with:
[fedora@ip-172-31-33-174 openshift-ansible]$ git log --oneline -1
b8879b5c6 (HEAD -> glusterfs) GlusterFS: remove topology reference from deploy-heketi

root@ip-172-31-15-103: ~ # oc version
oc v3.7.0-0.153.0

It passed the tasked of creating heketi pod in this bug, but got error later on
TASK [openshift_storage_glusterfs : Create heketi DB volume].

I will attach the log later.

Comment 6 Jose A. Rivera 2017-10-16 14:07:58 UTC
PR for latest bug is upstream: https://github.com/openshift/openshift-ansible/pull/5769

Comment 7 Hongkai Liu 2017-10-16 15:34:01 UTC
Verified: Thanks for the quick fix.


[fedora@ip-172-31-33-174 openshift-ansible]$ git log --oneline -2
3d3853836 (HEAD -> glusterfs2) GlusterFS: Remove image option from heketi command
27321df00 (origin/master, origin/HEAD, master) Merge pull request #5667 from jarrpa/deploy-heketi-fix-3.7
root@ip-172-31-11-69: ~ # oc version
oc v3.7.0-0.153.0

[fedora@ip-172-31-33-174 ~]$ ansible-playbook -i /tmp/2.file openshift-ansible/playbooks/byo/openshift-glusterfs/config.yml
PLAY RECAP ********************************************************************************************************************
ec2-54-186-162-10.us-west-2.compute.amazonaws.com : ok=47   changed=4    unreachable=0    failed=0   
ec2-54-187-203-117.us-west-2.compute.amazonaws.com : ok=47   changed=4    unreachable=0    failed=0   
ec2-54-200-62-253.us-west-2.compute.amazonaws.com : ok=47   changed=4    unreachable=0    failed=0   
ec2-54-202-151-185.us-west-2.compute.amazonaws.com : ok=107  changed=38   unreachable=0    failed=0   
ec2-54-213-59-28.us-west-2.compute.amazonaws.com : ok=44   changed=2    unreachable=0    failed=0   
localhost                  : ok=12   changed=0    unreachable=0    failed=0   


INSTALLER STATUS **************************************************************************************************************
Initialization             : Complete
etcd Install               : Not Started
NFS Install                : Not Started
Load balancer Install      : Not Started
Master Install             : Not Started
Master Additional Install  : Not Started
Node Install               : Not Started
GlusterFS Install          : Complete
Hosted Install             : Not Started
Metrics Install            : Not Started
Logging Install            : Not Started
Service Catalog Install    : Not Started
Management Install         : Not Started

Comment 8 Hongkai Liu 2017-10-16 15:43:58 UTC
Jose A. Rivera: another thought.

I noticed that "GlusterFS Install" is before "Hosted Install" in the playbook summary (see Comment 7).

Should it be the other way around like Metrics or Logging install?

I did verification of the fix on an existing gluster.

During the tests on a new gluster last Friday, glusterfs failed on very early stage. My guess would be that it depends on tasks in "Hosted Install".

That was why I stepped back and started to install on an existing cluster.

Let me know what you think about the ordering of the playbook.

Comment 9 Jose A. Rivera 2017-10-16 15:46:04 UTC
If I remember correctly, "Hosted Install" in this case is for the hosted registry. In this case, GlusterFS MUST come first if it is to provide storage for the hosted registry, and indeed for Logging and Metrics as well.

Comment 10 Hongkai Liu 2017-10-16 17:31:52 UTC
I see.
Thanks for the explanation.

Comment 13 errata-xmlrpc 2017-11-28 22:17:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188


Note You need to log in before you can comment on or make changes to this bug.