Bug 1591632 - master controllers edits the secret after upgrade to 3.9
Summary: master controllers edits the secret after upgrade to 3.9
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Master
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ---
: 3.9.z
Assignee: Michal Fojtik
QA Contact: ge liu
URL:
Whiteboard:
: 1596333 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-06-15 07:20 UTC by Vladislav Walek
Modified: 2021-09-09 14:36 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-07-18 09:18:59 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:2213 0 None None None 2018-07-18 09:19:11 UTC

Description Vladislav Walek 2018-06-15 07:20:28 UTC
Description of problem:

On a 3.9.30 cluster, when restarting the "atomic-openshift-master-controllers" service, all secrets are overwritten. This is a big issue, since this makes the cluster practically not usable. in attachment what we see, first "oc get secret", than restart the services, again "oc get secret", you can see the value changed. In the logs of the controller I see this:

Jun 14 18:17:11 master-1 atomic-openshift-master-controllers[72597]: I0614 18:17:11.439085   72597 start_master.go:652] Starting "openshift.io/serviceaccount-pull-secrets"
Jun 14 18:17:11 master-1 atomic-openshift-master-controllers[72597]: I0614 18:17:11.455637   72597 start_master.go:662] Started "openshift.io/serviceaccount-pull-secrets"

all secrets containing docker credentials for private docker repo's where changed. instead of the url to the private docker repo, they now contained the IP for the integrated registry. This means no images couldn't be pulled from the registries anymore.

and with the loglevel set to 6, and 1 secret modified to its correct value again, you see the secret gets overwritten.
Jun 14 18:37:20 master-1 atomic-openshift-master-controllers[79076]: I0614 18:37:18.237210   79076 round_trippers.go:436] PUT https://master-1:8443/api/v1/namespaces/<namespace>/secrets/registrysecret-k8s 200 OK in 22 milliseconds

Version-Release number of selected component (if applicable):
OpenShift Container Platform 3.9.30



Additional info:
will attach all the logs

Comment 4 Michal Fojtik 2018-06-15 11:47:53 UTC
The fix was merged and should be available in next 3.9.z update.

Comment 6 ge liu 2018-06-19 09:28:51 UTC
@vwalek, is there any key difference between our recreate steps? thanks in advance!

Comment 11 Ryan Howe 2018-06-28 18:10:51 UTC
*** Bug 1596333 has been marked as a duplicate of this bug. ***

Comment 26 errata-xmlrpc 2018-07-18 09:18:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2213


Note You need to log in before you can comment on or make changes to this bug.