Bug 1811144 - Simultaneous changes to image.config.openshift.io/cluster results in multiple machineconfigs
Summary: Simultaneous changes to image.config.openshift.io/cluster results in multiple...
Keywords:
Status: CLOSED DUPLICATE of bug 1809007
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: 4.5.0
Assignee: Urvashi Mohnani
QA Contact: Sunil Choudhary
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-06 17:30 UTC by Caden Marchese
Modified: 2024-01-06 04:28 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-20 18:15:18 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Caden Marchese 2020-03-06 17:30:49 UTC
Description of problem:
Following the document at https://docs.openshift.com/container-platform/4.3/builds/setting-up-trusted-ca.html, and then https://docs.openshift.com/container-platform/4.3/openshift_images/image-configuration.html#images-configuration-insecure_image-configuration before the operator picks up the change occasionally results in multiple machineconfigs that conflict with each other.

Version-Release number of selected component (if applicable):
Openshift 4.3, AWS UPI, Private

How reproducible:
Somewhat

Steps to Reproduce:
1. Apply an update to add a trusted CA for a registry (https://docs.openshift.com/container-platform/4.3/builds/setting-up-trusted-ca.html)
2. Use 'oc patch' to apply the change 
3. Quickly also add a blocked registry or configuration change (https://docs.openshift.com/container-platform/4.3/openshift_images/image-configuration.html#images-configuration-insecure_image-configuration

Actual results:
Instability as a result of two conflicting configurations that overwrite each other. Note the timestamps in the master machine config pool:

NAME                                                        GENERATEDBYCONTROLLER                      IGNITIONVERSION   CREATED
00-master                                                   25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
00-worker                                                   25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
01-master-container-runtime                                 25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
01-master-kubelet                                           25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
01-worker-container-runtime                                 25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
01-worker-kubelet                                           25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
99-master-f5d6922e-4244-4f3c-8fdb-8c97438d94ca-registries   25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
99-master-ssh                                                                                          2.2.0             13d
99-worker-5bb1b481-477b-43fa-887c-86723aea3a2a-registries   25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
99-worker-ssh                                                                                          2.2.0             13d
rendered-master-46a54ecdaeb9d1591ec7dfd736a66acc            25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             24m  <--- new
rendered-master-9e4c66aa9ed5f8986d052321957cfe4d            25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d  <--- old
rendered-master-a809ec58cc89d31d7fdb92eb05e24338            25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             57m  <--- new
rendered-master-d0b850ca7fd04c9c96e61ef65ce5df1d            25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d  <--- old
rendered-worker-3972ca77d7852f1899bf4ed3fde14ade            25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
rendered-worker-97f9cea03aa9f88392ca97df618ced5d            25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             13d
rendered-worker-d64a8fcb5b4cf3a6d23acd5537c6fea1            25bb6aeb58135c38a667e849edf5244871be4992   2.2.0             57m

We have a master group MCP that is flipping between the old resource and new resource (see timestamps)

Expected results:
The machineconfig would normally be updated, rather than duplicated and replaced continually.

Additional info:
must-gather from customer's cluster as well as diff results between the two machineconfigs is available on request.

The workaround is to roll back to the machineconfig that predates the change to image.config.openshift.io/cluster.

Comment 5 Red Hat Bugzilla 2024-01-06 04:28:22 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.