Description of problem: After deploying logging 3.4.1 successfully by using deployer pod and then upgrade to logging 3.5.0 via ansible, the upgrade is failed due to take the wrong namespace. In this defect, my namespace is juzhao, but ansible take "/tmp/openshift-logging-ansible-x00dCB/admin.kubeconfig" as namespace Version-Release number of selected component (if applicable): # openshift version openshift v3.5.0.18+9a5d1aa kubernetes v1.5.2+43a9be4 etcd 3.1.0 Image id: openshift3/logging-elasticsearch d715f4d34ad4 openshift3/logging-kibana e0ab09c2cbeb openshift3/logging-fluentd 47057624ecab openshift3/logging-auth-proxy 139f7943475e openshift3/logging-curator 7f034fdf7702 How reproducible: Always Steps to Reproduce: 1. Deploy logging 3.4.1 first, make sure all the pods are running and kibana UI can be accessed. 2. prepare the inventory file [oo_first_master] $master-public-dns ansible_user=root ansible_ssh_user=root ansible_ssh_private_key_file="~/cfile/libra.pem" openshift_public_hostname=$master-public-dns [oo_first_master:vars] deployment_type=openshift-enterprise openshift_release=v3.5.0 openshift_logging_install_logging=false openshift_logging_upgrade_logging=true openshift_logging_kibana_hostname=kibana.$sub-domain public_master_url=https://$master-public-dns:8443 openshift_logging_fluentd_hosts=$node openshift_logging_image_prefix=registry.ops.openshift.com/openshift3/ openshift_logging_image_version=3.5.0 openshift_logging_namespace=juzhao openshift_logging_use_ops=true openshift_logging_es_cluster_size=1 3. Running the playbook from a control machine (my laptop) which is not oo_master: git clone https://github.com/openshift/openshift-ansible ansible-playbook -vvv -i ~/inventory playbooks/common/openshift-cluster/openshift_logging.yml Actual results: TASK [openshift_logging : start elasticsearch] ********************************* task path: /home/fedora/openshift-ansible/roles/openshift_logging/tasks/upgrade_logging.yaml:16 Using module file /home/fedora/openshift-ansible/roles/lib_openshift/library/oc_scale.py <ec2-54-86-165-237.compute-1.amazonaws.com> ESTABLISH SSH CONNECTION FOR USER: root <ec2-54-86-165-237.compute-1.amazonaws.com> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/fedora/Downloads/libra-new.pem"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r ec2-54-86-165-237.compute-1.amazonaws.com '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"'' failed: [ec2-54-86-165-237.compute-1.amazonaws.com] (item=deploymentconfig/logging-es-56xhpu2b) => { "failed": true, "invocation": { "module_args": { "debug": false, "kind": "dc", "kubeconfig": "/etc/origin/master/admin.kubeconfig", "name": "logging-es-56xhpu2b", "namespace": "/tmp/openshift-logging-ansible-x00dCB/admin.kubeconfig", "replicas": 1, "state": "present" }, "module_name": "oc_scale" }, "object": "deploymentconfig/logging-es-56xhpu2b" } MSG: {u'cmd': u'oc -n /tmp/openshift-logging-ansible-x00dCB/admin.kubeconfig get dc logging-es-56xhpu2b -o json', u'returncode': 1, u'results': [{}], u'stderr': u'error: invalid namespace "/tmp/openshift-logging-ansible-x00dCB/admin.kubeconfig": [may not contain \'/\']\n', u'stdout': u''} PLAY RECAP ********************************************************************* ec2-54-86-165-237.compute-1.amazonaws.com : ok=253 changed=35 unreachable=0 failed=1 Expected results: Upgrade can be successful. Additional info:
fixed in: https://github.com/openshift/openshift-ansible/pull/3310
Commits pushed to master at https://github.com/openshift/openshift-ansible https://github.com/openshift/openshift-ansible/commit/fa1b4494a40218e173a54a10f3269fb0c7c39baa bug 1420684. On logging upgrade use the correct value for namespace https://github.com/openshift/openshift-ansible/commit/8ee9a7bfe9901db69e5d2ee1d118ed0f7ce81005 Merge pull request #3310 from jcantrill/bz_1420684_fix_namespace_on_logging_upgrade bug 1420684. On logging upgrade use the correct value for namespace
blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1418911
(In reply to Junqi Zhao from comment #3) > blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1418911 correct: blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1421011
although "503 Service Unavailable" shows in Kibana UI after upgrade, but ansible can use the right namespace now. # openshift version openshift v3.5.0.19+199197c kubernetes v1.5.2+43a9be4 etcd 3.1.0 Image id: logging-elasticsearch d715f4d34ad4 logging-kibana e0ab09c2cbeb logging-fluentd 47057624ecab logging-auth-proxy 139f7943475e logging-curator 7f034fdf7702 set it to VERIFIED and close it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3438