Bug 2070930 - Machine config daemon daemonset keeps proxy environment variables after proxy settings removal
Summary: Machine config daemon daemonset keeps proxy environment variables after proxy...
Keywords:
Status: CLOSED DUPLICATE of bug 1981549
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Machine Config Operator
Version: 4.7
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: MCO Team
QA Contact: Rio Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-04-01 11:09 UTC by Pablo Alonso Rodriguez
Modified: 2022-04-04 14:32 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-04-04 14:32:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Pablo Alonso Rodriguez 2022-04-01 11:09:14 UTC
Description of problem:

When a proxy is removed from cluster-wide proxy settings, the corresponding environment variables are not removed from the machine-config-daemon daemonset.

Version-Release number of MCO (Machine Config Operator) (if applicable):

4.8.17

Platform (AWS, VSphere, Metal, etc.): 

Irrelevant (reproducible in many platforms)

Are you certain that the root cause of the issue being reported is the MCO (Machine Config Operator)?
(Y/N/Not sure):

Of course

How reproducible:

Always

Did you catch this issue by running a Jenkins job? If yes, please list:

Reproduced manually

Steps to Reproduce:
1. Install a cluster with proxy or configure a proxy in the cluster
2. Ensure that the proxy settings are applied and machine-config-daemon (the variables are added correctly if needed, that's not part of the problem)
3. Remove the proxy from cluster-wide proxy settings (so that the cluster no longer uses a proxy)
4. Wait until all the components are re-configured, including MCO

Actual results:

Proxy settings still in machine-config-daemon environment variables

Expected results:

machine-config-daemon daemonset properly re-configured

Additional info:

I'll privately attach an example must-gather and audit logs, but you should be able to easily reproduce the issue by following the "steps to reproduce" in any cluster.

Some relevant observations:
- The `proxy:` field at the `controllerconfig` object IS properly set to `null` as one would expect.
- Other MCO components seem to be properly re-configured (or, at least, don't contain the proxy after removed)
- However, audit logs confirm that machine-config-operator posts updates to the machine-config-daemon daemonset object that include the proxy environment variables even after proxy removal and after proxy field on controllerconfig is already set to null

So this looks like a problem in the logic used to merge observed and desired state for the machine-config-daemon daemonset, such that it does add the proxy variables when needed but doesn't remove them when needed.

A valid and tested workaround is to delete the daemonset, because MCO then re-creates it correctly.

Comment 2 Yu Qi Zhang 2022-04-04 13:35:31 UTC
This is https://bugzilla.redhat.com/show_bug.cgi?id=1981549, which was fixed as part of 4.10 via https://github.com/openshift/machine-config-operator/pull/2800

It looks like we didn't opt to backport this. Is there a version you would like this backported to?

Comment 3 Pablo Alonso Rodriguez 2022-04-04 14:23:48 UTC
4.8.z, which also means 4.9.z. 

Thanks in advance.

Comment 4 Pablo Alonso Rodriguez 2022-04-04 14:32:05 UTC

*** This bug has been marked as a duplicate of bug 1981549 ***


Note You need to log in before you can comment on or make changes to this bug.