Bug 1947427 - Bootstrap ignition shim doesn't follow proxy settings
Summary: Bootstrap ignition shim doesn't follow proxy settings
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.8
Hardware: All
OS: All
medium
medium
Target Milestone: ---
: 4.7.z
Assignee: Emilien Macchi
QA Contact: Jon Uriarte
URL:
Whiteboard:
Depends On: 1945236
Blocks: 1949150
TreeView+ depends on / blocked
 
Reported: 2021-04-08 13:06 UTC by OpenShift BugZilla Robot
Modified: 2021-06-15 09:27 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 1949150 (view as bug list)
Environment:
Last Closed: 2021-06-15 09:26:45 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4830 0 None open [release-4.7] Bug 1947427: add proxy params to bootstrap ignition 2021-05-13 01:02:43 UTC
Red Hat Product Errata RHSA-2021:2286 0 None None None 2021-06-15 09:27:26 UTC

Comment 2 Emilien Macchi 2021-05-28 02:11:16 UTC
Jon, could we please verify it?

Comment 5 Jon Uriarte 2021-06-11 10:16:50 UTC
Verified in 4.7.0-0.nightly-2021-06-10-213131 on top of OSP 13.0.15 (2021-03-24.1) with TLS (and self-signed cert)
in public endpoints enabled.

A squid proxy has been deployed and configured in install-config.yaml:
proxy:
    httpProxy: "http://dummy:dummy@10.46.22.225:3128"
    httpsProxy: "https://dummy:dummy@10.46.22.225:3130"
    noProxy: oauth-openshift.apps.ostest.shiftstack.com

In OCP 4.7, as the cacert param from the clouds.yaml is considered instead of system wide CA trust bundle, the cacert
param in clouds.yaml needs to point to a pem file that includes both proxy's CA cert and Openstack's CA cert.

On the other hand, the proxy has Openstack's CA cert in it's trusted bundle.

10.46.22.225 is the proxy's IP
10.46.22.204:13292 is Openstack Glance API endpoint
10.46.22.227 is the bootstrap VM's IP

$ openstack server list                                                                                                                                                                            
+--------------------------------------+------------------------+--------+----------------------------------------------------+---------------------------------------+-----------+
| ID                                   | Name                   | Status | Networks                                           | Image                                 | Flavor    |
+--------------------------------------+------------------------+--------+----------------------------------------------------+---------------------------------------+-----------+
|...                                                        
| 9b8270ca-f5c4-47ba-a027-b59e61d1cabf | ostest-t8tq9-bootstrap | ACTIVE | ostest-t8tq9-openshift=10.196.3.196, 10.46.22.227  | ostest-t8tq9-rhcos                    | m4.xlarge |
|...
+--------------------------------------+------------------------+--------+----------------------------------------------------+---------------------------------------+-----------+

The bootstrap VM shows this message when booting up (note the host it queries is the final glance endpoint):
[   10.837268] ignition[718]: GET https://10.46.22.204:13292//v2/images/ce4da679-cb37-48ec-a2cb-448261d515e4/file: attempt #1
[   12.836066] ignition[718]: GET result: OK

And the access.log squid file shows:
1623405757.805   1900 10.46.22.227 TCP_TUNNEL/200 323265 CONNECT 10.46.22.204:13292 - HIER_DIRECT/10.46.22.204 -

which means the tunnel between the bootstrap (10.46.22.227) and OSP glance endpoint (10.46.22.204:13292) has been stablished.

I've run additional comprobations by stopping the proxy server when the bootstrap VM is booting up, and got this messages from the VM:

[    9.708206] ignition[720]: GET https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file: attempt #1                                                                                                                
[    9.715040] ignition[720]: GET error: Get "https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file": proxyconnect tcp: dial tcp 10.46.22.225:3130: connect: connection refused                                    
[    9.908533] ignition[720]: GET https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file: attempt #2
[    9.915513] ignition[720]: GET error: Get "https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file": proxyconnect tcp: dial tcp 10.46.22.225:3130: connect: connection refused
[   10.316173] ignition[720]: GET https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file: attempt #3
[   10.323627] ignition[720]: GET error: Get "https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file": proxyconnect tcp: dial tcp 10.46.22.225:3130: connect: connection refused
[   11.117828] ignition[720]: GET https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file: attempt #4
[   11.124789] ignition[720]: GET error: Get "https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file": proxyconnect tcp: dial tcp 10.46.22.225:3130: connect: connection refused
[   12.719991] ignition[720]: GET https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file: attempt #5
[   12.727253] ignition[720]: GET error: Get "https://10.46.22.204:13292//v2/images/77e6e802-4f64-4d5d-93fc-bc9a76c98768/file": proxyconnect tcp: dial tcp 10.46.22.225:3130: connect: connection refused

which means
 1. the GET request is towards OSP glance API (10.46.22.204), so it's transparent proxy
 2. the proxy is correctly configured as it fails with the connection refused from the proxy (10.46.22.225)

Comment 7 errata-xmlrpc 2021-06-15 09:26:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.16 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2286


Note You need to log in before you can comment on or make changes to this bug.