Bug 1808068 - openshift-ca.crt replaced during 3.11.104 to 3.11.157 upgrade
Summary: openshift-ca.crt replaced during 3.11.104 to 3.11.157 upgrade
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 3.11.z
Assignee: Russell Teague
QA Contact: Gaoyun Pei
URL:
Whiteboard:
Depends On: 1826448
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-27 18:13 UTC by dtarabor
Modified: 2023-10-06 19:17 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Missing mounts on the sync pod to the pki directory Consequence: The openshift-ca.crt found was not accessible and was recreated. Fix: Add the missing mounts and volumes to the sync pod. Result: The openshift-ca.crt file is accessible in the sync pod.
Clone Of:
Environment:
Last Closed: 2020-05-28 05:44:13 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift openshift-ansible pull 12119 0 None closed Bug 1808068: Add /usr/share/pki to sync pod 2020-09-24 21:08:07 UTC
Red Hat Product Errata RHBA-2020:2215 0 None None None 2020-05-28 05:44:30 UTC

Description dtarabor 2020-02-27 18:13:24 UTC
Description of problem:

Version-Release number of the following components:
ansible-2.6.20-1.el7ae.noarch
openshift-ansible-3.11.157-1.git.0.10b76ed.el7.noarch
ansible 2.6.20

How reproducible:

reproduced repeatedly on a test environment.

Steps to Reproduce:
1. Upgrade from 3.11.104 to 3.11.157

Actual results:
https://github.com/openshift/openshift-ansible/blob/release-3.11/roles/openshift_node_group/files/sync.yaml#L127-L133

[/usr/share/ansible/openshift-ansible/roles/openshift_node_group/tasks/sync.yml:]

the ca.crt is replaced, causing issues curling any endpoints that initially worked (in this case, an artifactory repo that was seemingly unaltered up until this play in the upgrade playbook)

Expected results:

the ability to curl an artifactory repo successfully.

Additional info:

see sf case # 02578970 comment #50 for:

curl attempts (curl_before*)
/etc/pki* (folder contents)
ansible logs (upgrade*.logs*

curl_before.log
pki_before.tgz
upgrade1.logs

curl_after_fail.log
pki_after_fail.tgz

pki_after_update_trust.tgz
curl_after_update_trust.log

upgrade2.logs
curl_after_2nd_upgrade.log
pki_after_2nd_upgrade.tgz

Comment 1 Chad Scribner 2020-03-05 14:39:06 UTC
This is the patch that was applied to rectify the issue.

--- /usr/share/ansible/openshift-ansible/roles/openshift_node_group/files/sync.yaml    2019-12-02 09:21:20.000000000 +0100
+++ /usr/share/ansible/openshift-ansible/roles/openshift_node_group/files/sync.yaml.fix    2020-02-21 17:03:00.000000000 +0100
@@ -226,6 +226,8 @@
           readOnly: true
         - mountPath: /etc/pki
           name: host-pki
+        - mountPath: /usr/share/pki
+          name: host-pki-usr

       volumes:
       # In bootstrap mode, the host config contains information not easily available
@@ -246,6 +248,10 @@
           path: /etc/pki
           type: ""
         name: host-pki
+      - hostPath:
+          path: /usr/share/pki
+          type: ""
+        name: host-pki-usr
       # Sync daemonset should tolerate all taints to make sure it runs on all nodes
       tolerations:
       - operator: "Exists"

Comment 5 Russell Teague 2020-05-07 17:15:41 UTC
We've had some issues with 3.11 CI that have been resolved and the PR merged.  The change will be picked up by QE and tested before shipping in the next release.

Comment 10 Gaoyun Pei 2020-05-19 13:45:01 UTC
Thanks for the info.

With using openshift-ansible-3.11.218-1.git.0.6f55149.el7.noarch, either for fresh install or upgrade from previous 3.11, the /usr/share/pki is also mounted to node sync pod.

[root@gpei-311bmaster-etcd-nfs-1 ~]# ls /usr/share/pki/ca-trust-source/anchors
test.crt

[root@gpei-311bmaster-etcd-nfs-1 ~]# oc describe -n openshift-node pod sync-ngmpq
...
Volumes:
...
  host-pki:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/pki
    HostPathType:  
  host-pki-usr:
    Type:          HostPath (bare host directory volume)
    Path:          /usr/share/pki
    HostPathType:  

[root@gpei-311bmaster-etcd-nfs-1 ~]# oc -n openshift-node rsh sync-ngmpq
sh-4.2# ls /usr/share/pki/ca-trust-source/anchors
test.crt

So it should be OK to move this bug to verified.

Comment 12 errata-xmlrpc 2020-05-28 05:44:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2215


Note You need to log in before you can comment on or make changes to this bug.