Hide Forgot
Description of problem: If there are files under /etc/openshift/generated-configs in HA master environment, the upgrade will fail at "TASK: [openshift_master_certificates | file ]". Version-Release number of selected component (if applicable): atomic-openshift-utils-3.0.29-1 How reproducible: always Steps to Reproduce: 1. Set 3 masters (ose 3.0.2.0 ) Pacemaker HA environment. 2. make sure there is /etc/openshift/generated-configs on master1. This directory was generated by ansible. In my HA Environment which installed by the previous ansible playbook, there is this directory on master1. but there aren't such directory on master2 , master3 and nodes. 3. ansible-playbook -i config/host3020clusterup /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_0_to_v3_1/upgrade.yml Actual Result: The upgrade failed. You can see message 'msg: Cannot link, file exists at destination' For more information, please refer to the attached file Expected Result: Upgrade succeed Addional: There isn't such issue on single master environment. If delete the /etc/openshift/generated-configs, the upgrade succeed
Created attachment 1114756 [details] Upgrade output
When certificates are created as part of the upgrade, we hard link those into the generated-configs dir for all masters. In this case, we created master.proxy-client.{crt,key} on the first master and those files must have already existed in /etc/openshift/generated-configs/<master-hostname>/. Another possibility is that something else caused the installer to think certificates were missing but, since the log fails on master.proxy-client.key, I think we generated them during this run. Was this upgrade ran against a fresh 3.0.2.0 install that hadn't been upgraded previously?
(In reply to Andrew Butcher from comment #2) > When certificates are created as part of the upgrade, we hard link those > into the generated-configs dir for all masters. In this case, we created > master.proxy-client.{crt,key} on the first master and those files must have > already existed in /etc/openshift/generated-configs/<master-hostname>/. > > Another possibility is that something else caused the installer to think > certificates were missing but, since the log fails on > master.proxy-client.key, I think we generated them during this run. > > Was this upgrade ran against a fresh 3.0.2.0 install that hadn't been > upgraded previously? Yes, It is a fresh 3.0.2.0 installed by prior ansible playbook. Which only generated-configs on the master master. If we installed 3.0.2.0 with latest ansible playbook. generated-configs will be created on all masters/nodes.
Proposed fix: https://github.com/openshift/openshift-ansible/pull/1245 Would you mind testing this from a checkout of the master branch of openshift-ansible?
Verified and pass