Description of problem: The redeploy-certificates.yml failed if the /etc/origin/logging is deleted on masters Version-Release number of selected component (if applicable): ose-ansible:v3.11.141 How reproducible: always 1. Deploy logging 2. rm -rf /etc/origin/logging 3. redeploy certificates using playbook ansible-playbook playbooks/openshift-logging/redeploy-certificates.yml Actual results: TASK [openshift_logging_kibana : Generate oauth secret] ************************ Friday 30 August 2019 02:52:56 +0000 (0:00:01.162) 0:01:09.801 ********* changed: [ec2-54-224-41-112.compute-1.amazonaws.com] TASK [include_role : openshift_logging_kibana] ********************************* Friday 30 August 2019 02:52:57 +0000 (0:00:01.144) 0:01:10.946 ********* TASK [openshift_logging_kibana : Retrieving the cert to use when generating secrets for the logging components] *** Friday 30 August 2019 02:52:57 +0000 (0:00:00.273) 0:01:11.220 ********* ok: [ec2-54-224-41-112.compute-1.amazonaws.com] => (item={u'name': u'ca_file', u'file': u'ca.crt'}) ok: [ec2-54-224-41-112.compute-1.amazonaws.com] => (item={u'name': u'kibana_internal_key', u'file': u'kibana-internal.key'}) ok: [ec2-54-224-41-112.compute-1.amazonaws.com] => (item={u'name': u'kibana_internal_cert', u'file': u'kibana-internal.crt'}) ok: [ec2-54-224-41-112.compute-1.amazonaws.com] => (item={u'name': u'server_tls', u'file': u'server-tls.json'}) ok: [ec2-54-224-41-112.compute-1.amazonaws.com] => (item={u'name': u'session_secret', u'file': u'session_secret'}) ok: [ec2-54-224-41-112.compute-1.amazonaws.com] => (item={u'name': u'oauth_secret', u'file': u'oauth_secret'}) TASK [include_role : {{logging_role}}] ***************************************** Friday 30 August 2019 02:52:58 +0000 (0:00:01.548) 0:01:12.769 ********* TASK [openshift_logging_elasticsearch : Set ES secret] ************************* Friday 30 August 2019 02:52:59 +0000 (0:00:00.397) 0:01:13.166 ********* fatal: [ec2-54-224-41-112.compute-1.amazonaws.com]: FAILED! => {"changed": false, "msg": {"cmd": "/usr/bin/oc -ojson secrets new logging-elasticsearch key=/etc/origin/logging/logging-es.jks truststore=/etc/origin/logging/truststore.jks searchguard.key=/etc/origin/logging/elasticsearch.jks searchguard.truststore=/etc/origin/logging/truststore.jks admin-key=/etc/origin/logging/system.admin.key admin-cert=/etc/origin/logging/system.admin.crt admin-ca=/etc/origin/logging/ca.crt admin.jks=/etc/origin/logging/system.admin.jks passwd.yml=/etc/origin/logging/passwd.yml -n openshift-logging", "results": {}, "returncode": 1, "stderr": "Command \"new\" is deprecated, use oc create secret\nerror: error reading /etc/origin/logging/passwd.yml: no such file or directory\n", "stdout": ""}} PLAY RECAP ********************************************************************* ec2-3-85-242-224.compute-1.amazonaws.com : ok=0 changed=0 unreachable=0 failed=0 ec2-3-89-32-175.compute-1.amazonaws.com : ok=0 changed=0 unreachable=0 failed=0 ec2-34-229-54-18.compute-1.amazonaws.com : ok=0 changed=0 unreachable=0 failed=0 ec2-35-172-116-251.compute-1.amazonaws.com : ok=0 changed=0 unreachable=0 failed=0 ec2-54-224-41-112.compute-1.amazonaws.com : ok=122 changed=29 unreachable=0 failed=1 localhost : ok=11 changed=0 unreachable=0 failed=0 INSTALLER STATUS *************************************************************** Initialization : Complete (0:00:14) Logging Cert Redeploy : In Progress (0:00:59) Actual results: The certificates can be regenerated.
Workaround: preserve the file /etc/origin/logging/passwd.yml when you delete files under /etc/origin/logging
TASK [openshift_logging_kibana : Generating Kibana route template] ************* Saturday 12 October 2019 05:40:20 +0000 (0:00:00.094) 0:01:42.038 ****** fatal: [ci-vm-10-0-148-139.hosted.upshift.rdu2.redhat.com]: FAILED! => {"msg": "The field 'vars' has an invalid value, which includes an undefined variable. The error was: 'openshift_logging_kibana_ops_hostname' is undefined\n\nThe error appears to have been in '/usr/share/ansible/openshift-ansible/roles/openshift_logging_kibana/tasks/generate_route.yaml': line 27, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Generating Kibana route template\n ^ here\n"} PLAY RECAP ********************************************************************* ci-vm-10-0-148-139.hosted.upshift.rdu2.redhat.com : ok=147 changed=38 unreachable=0 failed=1 ci-vm-10-0-149-248.hosted.upshift.rdu2.redhat.com : ok=0 changed=0 unreachable=0 failed=0 ci-vm-10-0-150-201.hosted.upshift.rdu2.redhat.com : ok=0 changed=0 unreachable=0 failed=0 ci-vm-10-0-150-223.hosted.upshift.rdu2.redhat.com : ok=0 changed=0 unreachable=0 failed=0 ci-vm-10-0-151-110.hosted.upshift.rdu2.redhat.com : ok=0 changed=0 unreachable=0 failed=0 ci-vm-10-0-151-111.hosted.upshift.rdu2.redhat.com : ok=0 changed=0 unreachable=0 failed=0 localhost : ok=11 changed=0 unreachable=0 failed=0
verify using non-ops cluster. Trace the bug in comment 3 in BZ1747307
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3139