Description of problem: Upgrade logging from 3.3.1 to 3.5.0, failed at TASK [openshift_logging : Applying /tmp/openshift-logging-ansible-1PQ3bY/templates/logging-curator-dc.yaml]: TASK [openshift_logging : Applying /tmp/openshift-logging-ansible-1PQ3bY/templates/logging-curator-dc.yaml] *** task path: /root/openshift-ansible/roles/openshift_logging/tasks/oc_apply.yaml:13 Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/commands/command.py <$master> ESTABLISH SSH CONNECTION FOR USER: root <$master> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/libra.pem"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r $master '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"'' fatal: [$master]: FAILED! => { "changed": false, "cmd": [ "oc", "--config=/tmp/openshift-logging-ansible-1PQ3bY/admin.kubeconfig", "apply", "-f", "/tmp/openshift-logging-ansible-1PQ3bY/templates/logging-curator-dc.yaml", "-n", "logging" ], "delta": "0:00:00.125620", "end": "2017-02-20 03:48:33.265045", "failed": true, "failed_when_result": true, "invocation": { "module_args": { "_raw_params": "oc --config=/tmp/openshift-logging-ansible-1PQ3bY/admin.kubeconfig apply -f /tmp/openshift-logging-ansible-1PQ3bY/templates/logging-curator-dc.yaml -n logging", "_uses_shell": false, "chdir": null, "creates": null, "executable": null, "removes": null, "warn": true }, "module_name": "command" }, "rc": 1, "start": "2017-02-20 03:48:33.139425", "warnings": [] } STDERR: error: unable to decode "/tmp/openshift-logging-ansible-1PQ3bY/templates/logging-curator-dc.yaml": [pos 1595]: json: expect char '"' but got char 't' PLAY RECAP ********************************************************************* $master : ok=149 changed=3 unreachable=0 failed=1 Version-Release number of selected component (if applicable): https://github.com/openshift/openshift-ansible -b master How reproducible: Always Steps to Reproduce: 1.Deploy logging 3.3.1 stacks (on OCP 3.5.0) with journald log driver enabled and node selectors defined in configmap: "use-journal": "true" "curator-nodeselector": "logging-infra-fluentd=true" "es-nodeselector": "logging-infra-fluentd=true" "kibana-nodeselector": "logging-infra-fluentd=true" Also bound es with hostPV storage on es node, wait until log entries shown on kibana UI. 2.Upgrade to logging 3.5.0 stacks by using ansible, specifying these parameters in inventory file (as in the attachment): openshift_logging_fluentd_use_journal=true openshift_logging_es_nodeselector={'logging-infra-fluentd':'true'} openshift_logging_kibana_nodeselector={'logging-infra-fluentd':'true'} openshift_logging_curator_nodeselector={'logging-infra-fluentd':'true'} openshift_logging_fluentd_nodeselector={'logging-infra-fluentd':'true'} 3.Check upgrade result Actual results: Upgrade failed Expected results: Upgrade should be successful Additional info: Ansible upgrade log attached inventory file for the upgrade attached Repro env attached
Created attachment 1255622 [details] ansible upgrade log
Created attachment 1255623 [details] the inventory file used for logging upgrade
Verified on OCP 3.5 with the latest playbooks from openshift-ansible, this error does not happen now. Set it to verified and close it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0903