Description of problem: The etcd datastore migration playbook was run after upgrading from OCP 3.5 to 3.6. The datastore was been updated to v3; existing secrets can be seen to be encrypted when using th3 v3 API to query etcd. However, querying the etcd database using the v2 API shows the old, unencrypted (just base64 encoded) secret. Description of problem: Version-Release number of the following components: rpm -q openshift-ansible rpm -q ansible ansible --version How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag
Ok, understanding this a bit more the process was upgrade from 3.5 to 3.6. Migrate from v2 to v3. Enable etcd encryption at rest per https://docs.openshift.com/container-platform/3.6/admin_guide/encrypting_data.html Yes, the v2 keys are not removed during the etcd v2 to v3 migration. They can be removed by running the following on your master: etcdctl2 rm -r /kubernetes.io etcdctl2 rm -r /openshift.io Please note that etcd backups are also taken during the upgrade and migration processes. If you truly want to eliminate all traces of unencrypted data you'll also want to carefully remove /var/lib/etcd/openshift-backup* making sure not to touch /var/lib/etcd/member which is the live database. Example directory listing # ls -la /var/lib/etcd total 4 drwxr-xr-x. 7 etcd etcd 218 Feb 2 14:48 . drwxr-xr-x. 44 root root 4096 Feb 2 14:51 .. drwx------. 4 etcd etcd 29 Feb 2 13:42 member drwx------. 3 root root 20 Feb 2 14:36 openshift-backup-post-3.0-20180202143640 drwx------. 3 root root 20 Feb 2 14:48 openshift-backup-post-3.0-20180202144816 drwx------. 3 root root 20 Feb 2 14:36 openshift-backup-pre-upgrade-20180202143626 drwx------. 3 root root 20 Feb 2 14:48 openshift-backup-pre-upgrade-20180202144759
*** This bug has been marked as a duplicate of bug 1514487 ***