Bug 1791995

Summary: baremetal: proxy-based install fails due to mismatch in no_proxy configuration in installer vs. cluster-network-operator
Product: OpenShift Container Platform Reporter: Stephen Benjamin <stbenjam>
Component: InstallerAssignee: Stephen Benjamin <stbenjam>
Installer sub component: OpenShift on Bare Metal IPI QA Contact: Victor Voronkov <vvoronko>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: unspecified CC: adahiya, augol, dtrainor, jialiu, vvoronko, william.caban
Version: 4.3.zKeywords: Triaged
Target Milestone: ---   
Target Release: 4.3.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1791993 Environment:
Last Closed: 2020-04-01 19:10:45 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1791993    
Bug Blocks:    

Description Stephen Benjamin 2020-01-16 20:18:56 UTC 
+++ This bug was initially created as a clone of Bug #1791993 +++

Description of problem:

The installer creates a manifest for proxy configuration, automatically
adding specific addresses to NO_PROXY depending on the platform. One of
those addresses is the metadata service, hosted at 169.254.169.254. The
installer assumes this must be done for all platforms other than None of
vSphere, whereas the cluster-network-operator has an explicit list of
platforms:

https://github.com/openshift/cluster-network-operator/blob/adaf257b4d63661726443ab2b059a9b4209a02d1/pkg/util/proxyconfig/no_proxy.go#L67-L69

When using a proxy with baremetal IPI, the installer adds this address,
however when the CNO comes up, it does not, causing the rendered
machine configs to differ, and installation to fail, with MCO reporting
errors like:

pool master has not progressed to latest configuration: configuration
status for pool master is empty: pool is degraded because nodes fail
with "3 nodes are reporting degraded status on sync": "Node master-1 is
reporting: \"machineconfig.machineconfiguration.openshift.io
\\\"rendered-master-982b8698753da7e31b5f902aa4dc135e\\\" not found\""



Version-Release number of the following components:

Seen in:
4.3.0-0.nightly-2020-01-11-070223
4.4.0-0.nightly-2020-01-12-221811


How reproducible:

Always

Steps to Reproduce:
1. Install baremetal IPI with a proxy enabled

Actual results:

Installation failure with these errors:

pool master has not progressed to latest configuration: configuration
status for pool master is empty: pool is degraded because nodes fail
with "3 nodes are reporting degraded status on sync": "Node master-1 is
reporting: \"machineconfig.machineconfiguration.openshift.io
\\\"rendered-master-982b8698753da7e31b5f902aa4dc135e\\\" not found\""


Expected results:

Install succeeds

Additional info:
Comment 1 Stephen Benjamin 2020-03-19 16:56:47 UTC 
4.3.z is accurate
Comment 9 Victor Voronkov 2020-03-25 20:10:00 UTC 
[kni@provisionhost-0 ~]$ oc version
Client Version: 4.3.0-0.nightly-2020-03-10-182422
Server Version: 4.3.9

[kni@provisionhost-0 ~]$ head -20 install-config.yaml 
apiVersion: v1
baseDomain: qe.lab.redhat.com
proxy:
  httpProxy: http://192.168.123.1:3128
  httpsProxy: http://192.168.123.1:3128
  noProxy: 172.22.0.0/24
networking:
  networkType: OpenShiftSDN
  machineCIDR: 192.168.123.0/24
Comment 11 errata-xmlrpc 2020-04-01 19:10:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0930