Bug 1759363

Summary: [Proxy]machine configs are not updated when proxy.config.openshift.io gets updated
Product: OpenShift Container Platform Reporter: Daneyon Hansen <dhansen>
Component: Machine Config OperatorAssignee: Antonio Murdaca <amurdaca>
Status: CLOSED ERRATA QA Contact: Michael Nguyen <mnguyen>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 4.3.0CC: bhershbe, jniu, mnguyen, scuppett, skumari, xiuwang
Target Milestone: ---Flags: mnguyen: needinfo+
Target Release: 4.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1759611 (view as bug list) Environment:
Last Closed: 2020-05-13 21:27:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1759611, 1762607    
Attachments:
Description Flags
mco pod log none

Description Daneyon Hansen 2019-10-08 00:10:34 UTC 
Description of problem:
machine configs do not get updated when proxy.config.openshift.io gets updated. I do not see any of the machine-config related objects get updated not do I see any of the systemd conf files get updated.

Version-Release number of selected component (if applicable):
4.3.0-0.okd-2019-10-07-210756

How reproducible:
Always

Steps to Reproduce:
1. Create a proxy-enabled cluster.
2. Modify the proxy object (i.e. add noProxy: example.com)
3. Grep systemd conf files for the expected modification.

Actual results:
No updates to systemd files. For example, after adding `noProxy: .example.com` to proxy/cluster:

$ grep -ri NO_PROXY /etc/systemd/system
/etc/systemd/system/kubelet.service.d/10-default-env.conf:Environment=NO_PROXY=.cluster.local,.svc,.us-west-2.compute.internal,10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.proxy-tres.devcluster.openshift.com,etcd-0.proxy-tres.devcluster.openshift.com,etcd-1.proxy-tres.devcluster.openshift.com,etcd-2.proxy-tres.devcluster.openshift.com,localhost
/etc/systemd/system/machine-config-daemon-host.service.d/10-default-env.conf:Environment=NO_PROXY=.cluster.local,.svc,.us-west-2.compute.internal,10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.proxy-tres.devcluster.openshift.com,etcd-0.proxy-tres.devcluster.openshift.com,etcd-1.proxy-tres.devcluster.openshift.com,etcd-2.proxy-tres.devcluster.openshift.com,localhost
/etc/systemd/system/pivot.service.d/10-default-env.conf:Environment=NO_PROXY=.cluster.local,.svc,.us-west-2.compute.internal,10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.proxy-tres.devcluster.openshift.com,etcd-0.proxy-tres.devcluster.openshift.com,etcd-1.proxy-tres.devcluster.openshift.com,etcd-2.proxy-tres.devcluster.openshift.com,localhost
/etc/systemd/system/crio.service.d/10-default-env.conf:Environment=NO_PROXY=.cluster.local,.svc,.us-west-2.compute.internal,10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.proxy-tres.devcluster.openshift.com,etcd-0.proxy-tres.devcluster.openshift.com,etcd-1.proxy-tres.devcluster.openshift.com,etcd-2.proxy-tres.devcluster.openshift.com,localhost

Expected results:
proxy-related systemd env files to get updated.

Additional info:
Comment 1 Antonio Murdaca 2019-10-08 08:45:40 UTC 
changing proxy/cluster should result in an update rolling, can you grab `oc get mcp`? the MCO operator is responsible for that and we have an informer/lister on it so we should be notified when that changes and react to that.

```
...
		proxyInformer.Informer(),
		...
	} {
		i.AddEventHandler(optr.eventHandler())
	}
```
```
Comment 2 Antonio Murdaca 2019-10-08 08:45:46 UTC 
changing proxy/cluster should result in an update rolling, can you grab `oc get mcp`? the MCO operator is responsible for that and we have an informer/lister on it so we should be notified when that changes and react to that.

```
...
		proxyInformer.Informer(),
		...
	} {
		i.AddEventHandler(optr.eventHandler())
	}
```
```
Comment 4 Daneyon Hansen 2019-10-08 16:47:40 UTC 
$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT
master   rendered-master-e4b0cd61985703d224c0de1eb5f36529   True      False      False      3              3                   3                     0
worker   rendered-worker-3aa96601745a04dda78cbe5324204714   True      False      False      3              3                   3                     0
Comment 5 Daneyon Hansen 2019-10-08 16:48:12 UTC 
4.2 does have the same issue.
Comment 6 Daneyon Hansen 2019-10-08 16:50:35 UTC 
Created https://bugzilla.redhat.com/show_bug.cgi?id=1759611 for 4.2.z backport.
Comment 7 Daneyon Hansen 2019-10-08 16:56:16 UTC 
The only proxy-related log message I see from the mco pod is:

W1008 13:33:26.428257       1 reflector.go:299] github.com/openshift/client-go/config/informers/externalversions/factory.go:101: watch of *v1.Proxy ended with: too old resource version: 309871 (313005)
Comment 8 Daneyon Hansen 2019-10-08 16:57:19 UTC 
Created attachment 1623519 [details]
mco pod log
Comment 10 Ryan Phillips 2019-11-05 21:14:35 UTC 
*** Bug 1762999 has been marked as a duplicate of this bug. ***
Comment 11 Sinny Kumari 2019-11-07 05:45:47 UTC 
Can we please get this bug verified in order to unblock BZ#1759611
Comment 13 Neelesh Agrawal 2019-11-27 19:19:38 UTC 
*** Bug 1762607 has been marked as a duplicate of this bug. ***
Comment 15 errata-xmlrpc 2020-05-13 21:27:11 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062