Bug 1759363 - [Proxy]machine configs are not updated when proxy.config.openshift.io gets updated
Summary: [Proxy]machine configs are not updated when proxy.config.openshift.io gets up...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Machine Config Operator
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.3.0
Assignee: Antonio Murdaca
QA Contact: Michael Nguyen
URL:
Whiteboard:
: 1762607 1762999 (view as bug list)
Depends On:
Blocks: 1759611 1762607
TreeView+ depends on / blocked
 
Reported: 2019-10-08 00:10 UTC by Daneyon Hansen
Modified: 2020-05-13 21:27 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1759611 (view as bug list)
Environment:
Last Closed: 2020-05-13 21:27:11 UTC
Target Upstream Version:
mnguyen: needinfo+


Attachments (Terms of Use)
mco pod log (19.05 KB, text/plain)
2019-10-08 16:57 UTC, Daneyon Hansen
no flags Details


Links
System ID Priority Status Summary Last Updated
Github openshift machine-config-operator pull 1174 'None' closed Bug 1759363: lib/resourcemerge: account for proxy changes 2020-07-03 08:43:54 UTC
Red Hat Knowledge Base (Solution) 4898081 None None None 2020-03-20 13:08:12 UTC
Red Hat Product Errata RHBA-2020:0062 None None None 2020-05-13 21:27:16 UTC

Description Daneyon Hansen 2019-10-08 00:10:34 UTC
Description of problem:
machine configs do not get updated when proxy.config.openshift.io gets updated. I do not see any of the machine-config related objects get updated not do I see any of the systemd conf files get updated.

Version-Release number of selected component (if applicable):
4.3.0-0.okd-2019-10-07-210756

How reproducible:
Always

Steps to Reproduce:
1. Create a proxy-enabled cluster.
2. Modify the proxy object (i.e. add noProxy: example.com)
3. Grep systemd conf files for the expected modification.

Actual results:
No updates to systemd files. For example, after adding `noProxy: .example.com` to proxy/cluster:

$ grep -ri NO_PROXY /etc/systemd/system
/etc/systemd/system/kubelet.service.d/10-default-env.conf:Environment=NO_PROXY=.cluster.local,.svc,.us-west-2.compute.internal,10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.proxy-tres.devcluster.openshift.com,etcd-0.proxy-tres.devcluster.openshift.com,etcd-1.proxy-tres.devcluster.openshift.com,etcd-2.proxy-tres.devcluster.openshift.com,localhost
/etc/systemd/system/machine-config-daemon-host.service.d/10-default-env.conf:Environment=NO_PROXY=.cluster.local,.svc,.us-west-2.compute.internal,10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.proxy-tres.devcluster.openshift.com,etcd-0.proxy-tres.devcluster.openshift.com,etcd-1.proxy-tres.devcluster.openshift.com,etcd-2.proxy-tres.devcluster.openshift.com,localhost
/etc/systemd/system/pivot.service.d/10-default-env.conf:Environment=NO_PROXY=.cluster.local,.svc,.us-west-2.compute.internal,10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.proxy-tres.devcluster.openshift.com,etcd-0.proxy-tres.devcluster.openshift.com,etcd-1.proxy-tres.devcluster.openshift.com,etcd-2.proxy-tres.devcluster.openshift.com,localhost
/etc/systemd/system/crio.service.d/10-default-env.conf:Environment=NO_PROXY=.cluster.local,.svc,.us-west-2.compute.internal,10.0.0.0/16,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,api-int.proxy-tres.devcluster.openshift.com,etcd-0.proxy-tres.devcluster.openshift.com,etcd-1.proxy-tres.devcluster.openshift.com,etcd-2.proxy-tres.devcluster.openshift.com,localhost

Expected results:
proxy-related systemd env files to get updated.

Additional info:

Comment 1 Antonio Murdaca 2019-10-08 08:45:40 UTC
changing proxy/cluster should result in an update rolling, can you grab `oc get mcp`? the MCO operator is responsible for that and we have an informer/lister on it so we should be notified when that changes and react to that.

```
...
		proxyInformer.Informer(),
		...
	} {
		i.AddEventHandler(optr.eventHandler())
	}
```
```

Comment 2 Antonio Murdaca 2019-10-08 08:45:46 UTC
changing proxy/cluster should result in an update rolling, can you grab `oc get mcp`? the MCO operator is responsible for that and we have an informer/lister on it so we should be notified when that changes and react to that.

```
...
		proxyInformer.Informer(),
		...
	} {
		i.AddEventHandler(optr.eventHandler())
	}
```
```

Comment 4 Daneyon Hansen 2019-10-08 16:47:40 UTC
$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT
master   rendered-master-e4b0cd61985703d224c0de1eb5f36529   True      False      False      3              3                   3                     0
worker   rendered-worker-3aa96601745a04dda78cbe5324204714   True      False      False      3              3                   3                     0

Comment 5 Daneyon Hansen 2019-10-08 16:48:12 UTC
4.2 does have the same issue.

Comment 6 Daneyon Hansen 2019-10-08 16:50:35 UTC
Created https://bugzilla.redhat.com/show_bug.cgi?id=1759611 for 4.2.z backport.

Comment 7 Daneyon Hansen 2019-10-08 16:56:16 UTC
The only proxy-related log message I see from the mco pod is:

W1008 13:33:26.428257       1 reflector.go:299] github.com/openshift/client-go/config/informers/externalversions/factory.go:101: watch of *v1.Proxy ended with: too old resource version: 309871 (313005)

Comment 8 Daneyon Hansen 2019-10-08 16:57:19 UTC
Created attachment 1623519 [details]
mco pod log

Comment 10 Ryan Phillips 2019-11-05 21:14:35 UTC
*** Bug 1762999 has been marked as a duplicate of this bug. ***

Comment 11 Sinny Kumari 2019-11-07 05:45:47 UTC
Can we please get this bug verified in order to unblock BZ#1759611

Comment 13 Neelesh Agrawal 2019-11-27 19:19:38 UTC
*** Bug 1762607 has been marked as a duplicate of this bug. ***

Comment 15 errata-xmlrpc 2020-05-13 21:27:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.