Hide Forgot
Created attachment 1413938 [details] oc apply error Description of problem: During an upgrade from 3.7 to 3.9, the following error was reported. Retries encounter the same issue. See attached listing. Version-Release number of selected component (if applicable): v3.9.14
Which playbook was executed to invoke this role? openshift-ansible doesn't include this in any playbooks other than gce provisioning.
Correction, this is called via the control plane upgrade. https://github.com/openshift/openshift-ansible/blob/master/playbooks/openshift-master/private/upgrade.yml#L83-L85 The problem likely happened because the oc command doesn't utilize the admin kubeconfig and on that particular host root's kubeconfig was in another project at the time. We need to sweep the codebase again for this problem.
We're also likely running this code on each master where as I believe it only needs to be run once as it's cluster scoped.
master: https://github.com/openshift/openshift-ansible/pull/7713
release-3.9: https://github.com/openshift/openshift-ansible/pull/7758
Commit is in build openshift-ansible-3.9.20-1.git.0.f99fb43.el7
Couldn't reproduce this issue with openshift-ansible-3.9.14-1.git.0.ca2cfc3.el7.noarch.rpm when upgrading 3.7 to 3.9. openshift-ansible wouldn't run the step unless we have openshift_master_bootstrap_enabled=true set in ansible inventory file, tried an upgrade from 3.7 to 3.9 using openshift-ansible-3.9.24-1.git.0.d0289ea.el7.noarch with openshift_master_bootstrap_enabled=true set, no such error happened. [root@gpei-preserved ~]# grep -A 2 "Create auto-approver on cluster" logs/0423_upgrade TASK [openshift_bootstrap_autoapprover : Create auto-approver on cluster] *************************************************************************************************** changed: [qe-gpei-37master-1.0423-t8x.qe.rhcloud.com] => {"changed": true, "cmd": ["oc", "apply", "-f", "/tmp/openshift-approver/", "--config=/etc/origin/master/admin.kubeconfig"], "delta": "0:00:00.323191", "end": "2018-04-23 04:27:42.663713", "failed": false, "rc": 0, "start": "2018-04-23 04:27:42.340522", "stderr": "", "stderr_lines": [], "stdout": "clusterrolebinding \"bootstrap-autoapprover\" created\nclusterrole \"system:node-bootstrap-autoapprover\" created\nserviceaccount \"bootstrap-autoapprover\" created\nstatefulset \"bootstrap-autoapprover\" created", "stdout_lines": ["clusterrolebinding \"bootstrap-autoapprover\" created", "clusterrole \"system:node-bootstrap-autoapprover\" created", "serviceaccount \"bootstrap-autoapprover\" created", "statefulset \"bootstrap-autoapprover\" created"]} -- TASK [openshift_bootstrap_autoapprover : Create auto-approver on cluster] *************************************************************************************************** changed: [qe-gpei-37master-1.0423-t8x.qe.rhcloud.com] => {"changed": true, "cmd": ["oc", "apply", "-f", "/tmp/openshift-approver/", "--config=/etc/origin/master/admin.kubeconfig"], "delta": "0:00:00.386440", "end": "2018-04-23 04:38:44.751348", "failed": false, "rc": 0, "start": "2018-04-23 04:38:44.364908", "stderr": "", "stderr_lines": [], "stdout": "clusterrolebinding \"bootstrap-autoapprover\" configured\nclusterrole \"system:node-bootstrap-autoapprover\" configured\nserviceaccount \"bootstrap-autoapprover\" unchanged\nstatefulset \"bootstrap-autoapprover\" configured", "stdout_lines": ["clusterrolebinding \"bootstrap-autoapprover\" configured", "clusterrole \"system:node-bootstrap-autoapprover\" configured", "serviceaccount \"bootstrap-autoapprover\" unchanged", "statefulset \"bootstrap-autoapprover\" configured"]} Move this bug to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1566