Red Hat Bugzilla – Bug 1520174
Enabling logging in install inventory causes install to fail
Last modified: 2018-04-05 05:33:53 EDT
Created attachment 1362368 [details] Install log file Description of problem: I am installing OpenShift in AWS, if I enable the installation of logging via the openshift_logging_install_logging=true variable the installation fails. If I remove this variable it works as expected. I am using my playbook at the following URL if you want to try to reproduce easily: https://github.com/gnunn1/openshift-aws-setup Version-Release number of selected component (if applicable): openshift-ansible-3.7.9-1.git.7.eedd332.el7.noarch How reproducible: Every time Steps to Reproduce: 1. 2. 3. Actual results: TASK [openshift_hosted_facts : Set hosted facts] ******************************* ok: [ip-10-0-1-58.us-west-2.compute.internal] => (item=hosted) failed: [ip-10-0-1-58.us-west-2.compute.internal] (item=logging) => {"changed": false, "failed": true, "item": "logging", "module_stderr": "Shared connection to ip-10-0-1-58.us-west-2.compute.internal closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ansible_f580vY/ansible_module_openshift_facts.py\", line 2470, in <module>\r\n main()\r\n File \"/tmp/ansible_f580vY/ansible_module_openshift_facts.py\", line 2457, in main\r\n protected_facts_to_overwrite)\r\n File \"/tmp/ansible_f580vY/ansible_module_openshift_facts.py\", line 1830, in __init__\r\n protected_facts_to_overwrite)\r\n File \"/tmp/ansible_f580vY/ansible_module_openshift_facts.py\", line 1879, in generate_facts\r\n facts = set_selectors(facts)\r\n File \"/tmp/ansible_f580vY/ansible_module_openshift_facts.py\", line 496, in set_selectors\r\n facts['logging']['selector'] = None\r\nTypeError: 'str' object does not support item assignment\r\n", "msg": "MODULE FAILURE", "rc": 0} failed: [ip-10-0-1-58.us-west-2.compute.internal] (item=loggingops) => {"changed": false, "failed": true, "item": "loggingops", "module_stderr": "Shared connection to ip-10-0-1-58.us-west-2.compute.internal closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ansible_af_GOx/ansible_module_openshift_facts.py\", line 2470, in <module>\r\n main()\r\n File \"/tmp/ansible_af_GOx/ansible_module_openshift_facts.py\", line 2457, in main\r\n protected_facts_to_overwrite)\r\n File \"/tmp/ansible_af_GOx/ansible_module_openshift_facts.py\", line 1830, in __init__\r\n protected_facts_to_overwrite)\r\n File \"/tmp/ansible_af_GOx/ansible_module_openshift_facts.py\", line 1879, in generate_facts\r\n facts = set_selectors(facts)\r\n File \"/tmp/ansible_af_GOx/ansible_module_openshift_facts.py\", line 496, in set_selectors\r\n facts['logging']['selector'] = None\r\nTypeError: 'unicode' object does not support item assignment\r\n", "msg": "MODULE FAILURE", "rc": 0} failed: [ip-10-0-1-58.us-west-2.compute.internal] (item=metrics) => {"changed": false, "failed": true, "item": "metrics", "module_stderr": "Shared connection to ip-10-0-1-58.us-west-2.compute.internal closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ansible_o0C_b0/ansible_module_openshift_facts.py\", line 2470, in <module>\r\n main()\r\n File \"/tmp/ansible_o0C_b0/ansible_module_openshift_facts.py\", line 2457, in main\r\n protected_facts_to_overwrite)\r\n File \"/tmp/ansible_o0C_b0/ansible_module_openshift_facts.py\", line 1830, in __init__\r\n protected_facts_to_overwrite)\r\n File \"/tmp/ansible_o0C_b0/ansible_module_openshift_facts.py\", line 1879, in generate_facts\r\n facts = set_selectors(facts)\r\n File \"/tmp/ansible_o0C_b0/ansible_module_openshift_facts.py\", line 496, in set_selectors\r\n facts['logging']['selector'] = None\r\nTypeError: 'unicode' object does not support item assignment\r\n", "msg": "MODULE FAILURE", "rc": 0} failed: [ip-10-0-1-58.us-west-2.compute.internal] (item=prometheus) => {"changed": false, "failed": true, "item": "prometheus", "module_stderr": "Shared connection to ip-10-0-1-58.us-west-2.compute.internal closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ansible_RMmUeg/ansible_module_openshift_facts.py\", line 2470, in <module>\r\n main()\r\n File \"/tmp/ansible_RMmUeg/ansible_module_openshift_facts.py\", line 2457, in main\r\n protected_facts_to_overwrite)\r\n File \"/tmp/ansible_RMmUeg/ansible_module_openshift_facts.py\", line 1830, in __init__\r\n protected_facts_to_overwrite)\r\n File \"/tmp/ansible_RMmUeg/ansible_module_openshift_facts.py\", line 1879, in generate_facts\r\n facts = set_selectors(facts)\r\n File \"/tmp/ansible_RMmUeg/ansible_module_openshift_facts.py\", line 496, in set_selectors\r\n facts['logging']['selector'] = None\r\nTypeError: 'unicode' object does not support item assignment\r\n", "msg": "MODULE FAILURE", "rc": 0} NO MORE HOSTS LEFT ************************************************************* [WARNING]: Could not create retry file '/usr/share/ansible/openshift- ansible/playbooks/byo/config.retry'. [Errno 13] Permission denied: u'/usr/share/ansible/openshift-ansible/playbooks/byo/config.retry' PLAY RECAP ********************************************************************* ip-10-0-1-24.us-west-2.compute.internal : ok=48 changed=5 unreachable=0 failed=0 ip-10-0-1-58.us-west-2.compute.internal : ok=236 changed=71 unreachable=0 failed=1 ip-10-0-1-9.us-west-2.compute.internal : ok=48 changed=5 unreachable=0 failed=0 localhost : ok=11 changed=0 unreachable=0 failed=0 INSTALLER STATUS *************************************************************** Initialization : Complete Health Check : Complete etcd Install : Complete Master Install : In Progress This phase can be restarted by running: playbooks/byo/openshift-master/config.yml Expected results: OpenShift installs with logging Additional info: Description of problem: Version-Release number of the following components: rpm -q openshift-ansible rpm -q ansible ansible --version How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag
Created attachment 1362369 [details] inventory
Gerald, You can repeatedly and consistently deploy a cluster but fail when installing logging? There is no chance this is intermittent?
@Scott, Is there someone who works on the fact gathering code that may address this. We have not even executed the logging playbooks when this error occurs
It fails everytime when install logging is true so yes it is consistent. With it set to false the cluster installs fine without issue.
Jeff, Yeah, I'll see if Andrew can take a stab at it.
https://github.com/openshift/openshift-ansible/pull/6306
@scott, logging can be deployed using openshift-ansible-3.7.26, I had modified the target version to v3.7.z.
@jialiu, Logging can be deployed during installing OpenShift with openshift_logging_storage_kind=dynamic. the pull/6306 inlcude many other fix in filter and openshift_hosted. please investigate if we need more test coverage.
(In reply to Anping Li from comment #8) > @scott, logging can be deployed using openshift-ansible-3.7.26, I had > modified the target version to v3.7.z. Ok, thanks!
Logging can be installed alongside using openshift-ansible-3.7.2. so move bug to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0636