Bug 2000551

Summary: Cluster Proxy not used during installation on OSP
Product: OpenShift Container Platform Reporter: Maysa Macedo <mdemaced>
Component: Cloud ComputeAssignee: Martin André <maandre>
Cloud Compute sub component: OpenStack Provider QA Contact: rlobillo
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: high CC: adduarte, eduen, egarcia, emacchi, juriarte, m.andre, mfedosin, openshift-bugzilla-robot, pprinett, rlobillo
Version: 4.8Keywords: Triaged
Target Milestone: ---   
Target Release: 4.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: The HTTP transport to connect to OpenStack endpoints using a custom CA certificate was missing the Proxy settings. Consequence: Cluster wasn't fully operational when deployed on OpenStack with a combination of proxy and custom CA certificate. Fix: Pass the proxy settings to the HTTP transport used when connecting with a custom CA certificate. Result: All cluster components work as expected.
Story Points: ---
Clone Of: 2000542 Environment:
Last Closed: 2021-09-29 14:10:09 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2000542    
Bug Blocks: 2002752    

Comment 10 rlobillo 2021-09-10 08:42:57 UTC
Verified on 4.7.0-0.nightly-2021-09-10-001829 over OSP16.1 (RHOS-16.1-RHEL-8-20210604.n.0) with OpenShiftSDN Network type.

OCP installation on restricted network using proxy succeeded:

apiVersion: v1                                                                                                                                                                                
baseDomain: "shiftstack.com"                                                                                                                                                                  
clusterID:  "31439e98-34c5-56b2-8552-f49ed93fccce"                                                                                                                                            
compute:                                                                                                                                                                                      
- name: worker                                                                                                                                                                                
  platform:                                                                                                                                                                                   
    openstack:                                                                                                                                                                                
      zones: []                                                                                                                                                                               
      additionalNetworkIDs: []                                                                                                                                                                
  replicas: 3                                                                                                                                                                                 
controlPlane:                                                                                                                                                                                 
  name: master
  platform:
    openstack:
      zones: []
  replicas: 3
metadata:
  name: "ostest"
networking:
  clusterNetworks:
  - cidr:             10.128.0.0/14
    hostSubnetLength: 9
  serviceCIDR: 172.30.0.0/16
  machineCIDR: "172.16.0.0/24"
  type: "OpenShiftSDN"
platform:
  openstack:
    cloud:            "shiftstack"
    externalNetwork:  ""
    region:           "regionOne"
    computeFlavor:    "m4.xlarge"
    machinesSubnet: 04eaa3fa-84a7-44ad-af6d-af62c6f54fd9
    apiVIP: "172.16.0.5"
    ingressVIP: "172.16.0.7"
proxy:
  httpProxy: http://dummy:dummy@172.16.0.3:3128/
  httpsProxy: https://dummy:dummy@172.16.0.3:3130/
pullSecret: <hidden>
sshKey: <hidden>
additionalTrustBundle: <hidden>

$ tail -f ostest/.openshift_install.log 
time="2021-09-10T04:21:34-04:00" level=info msg="To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/cloud-user/ostest/auth/kubeconfig'"
time="2021-09-10T04:21:34-04:00" level=info msg="Access the OpenShift web-console here: https://console-openshift-console.apps.ostest.shiftstack.com"
time="2021-09-10T04:21:34-04:00" level=info msg="Login to the console with user: \"kubeadmin\", and password: \"s7d8C-bPT3g-IJwPB-GwT3M\""
time="2021-09-10T04:21:34-04:00" level=debug msg="Time elapsed per stage:"
time="2021-09-10T04:21:34-04:00" level=debug msg="    Infrastructure: 2m13s"
time="2021-09-10T04:21:34-04:00" level=debug msg="Bootstrap Complete: 31m2s"
time="2021-09-10T04:21:34-04:00" level=debug msg="               API: 6m38s"
time="2021-09-10T04:21:34-04:00" level=debug msg=" Bootstrap Destroy: 39s"
time="2021-09-10T04:21:34-04:00" level=debug msg=" Cluster Operators: 27m40s"
time="2021-09-10T04:21:34-04:00" level=info msg="Time elapsed: 1h7m30s"

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.nightly-2021-09-10-001829   True        False         16m     Cluster version is 4.7.0-0.nightly-2021-09-10-001829
$ oc get clusteroperators
NAME                                       VERSION                             AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                             4.7.0-0.nightly-2021-09-10-001829   True        False         False      19m
baremetal                                  4.7.0-0.nightly-2021-09-10-001829   True        False         False      49m
cloud-credential                           4.7.0-0.nightly-2021-09-10-001829   True        False         False      60m
cluster-autoscaler                         4.7.0-0.nightly-2021-09-10-001829   True        False         False      53m
config-operator                            4.7.0-0.nightly-2021-09-10-001829   True        False         False      56m
console                                    4.7.0-0.nightly-2021-09-10-001829   True        False         False      25m
csi-snapshot-controller                    4.7.0-0.nightly-2021-09-10-001829   True        False         False      54m
dns                                        4.7.0-0.nightly-2021-09-10-001829   True        False         False      49m
etcd                                       4.7.0-0.nightly-2021-09-10-001829   True        False         False      53m
image-registry                             4.7.0-0.nightly-2021-09-10-001829   True        False         False      30m
ingress                                    4.7.0-0.nightly-2021-09-10-001829   True        False         False      29m
insights                                   4.7.0-0.nightly-2021-09-10-001829   True        False         False      48m
kube-apiserver                             4.7.0-0.nightly-2021-09-10-001829   True        False         False      52m
kube-controller-manager                    4.7.0-0.nightly-2021-09-10-001829   True        False         False      50m
kube-scheduler                             4.7.0-0.nightly-2021-09-10-001829   True        False         False      52m
kube-storage-version-migrator              4.7.0-0.nightly-2021-09-10-001829   True        False         False      30m
machine-api                                4.7.0-0.nightly-2021-09-10-001829   True        False         False      45m
machine-approver                           4.7.0-0.nightly-2021-09-10-001829   True        False         False      51m
machine-config                             4.7.0-0.nightly-2021-09-10-001829   True        False         False      50m
marketplace                                4.7.0-0.nightly-2021-09-10-001829   True        False         False      53m
monitoring                                 4.7.0-0.nightly-2021-09-10-001829   True        False         False      27m
network                                    4.7.0-0.nightly-2021-09-10-001829   True        False         False      55m
node-tuning                                4.7.0-0.nightly-2021-09-10-001829   True        False         False      53m
openshift-apiserver                        4.7.0-0.nightly-2021-09-10-001829   True        False         False      46m
openshift-controller-manager               4.7.0-0.nightly-2021-09-10-001829   True        False         False      53m
openshift-samples                          4.7.0-0.nightly-2021-09-10-001829   True        False         False      48m
operator-lifecycle-manager                 4.7.0-0.nightly-2021-09-10-001829   True        False         False      53m
operator-lifecycle-manager-catalog         4.7.0-0.nightly-2021-09-10-001829   True        False         False      52m
operator-lifecycle-manager-packageserver   4.7.0-0.nightly-2021-09-10-001829   True        False         False      51m
service-ca                                 4.7.0-0.nightly-2021-09-10-001829   True        False         False      56m
storage                                    4.7.0-0.nightly-2021-09-10-001829   True        False         False      50m

$ oc get nodes
NAME                          STATUS   ROLES    AGE   VERSION
ostest-9n6g9-master-0         Ready    master   64m   v1.20.0+9689d22
ostest-9n6g9-master-1         Ready    master   64m   v1.20.0+9689d22
ostest-9n6g9-master-2         Ready    master   64m   v1.20.0+9689d22
ostest-9n6g9-worker-0-86rzx   Ready    worker   37m   v1.20.0+9689d22
ostest-9n6g9-worker-0-wdlrj   Ready    worker   30m   v1.20.0+9689d22

$ oc get proxy cluster -o json | jq .status
{
  "httpProxy": "http://dummy:dummy@172.16.0.3:3128/",
  "httpsProxy": "https://dummy:dummy@172.16.0.3:3130/",
  "noProxy": ".cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.16.0.0/24,172.30.0.0/16,api-int.ostest.shiftstack.com,localhost"                                               
}


$ oc logs machine-api-controllers-656cb464b5-t8xr2 -n openshift-machine-api -c machine-controller | grep "Failed to authenticate"
$
$ oc -n openshift-machine-api -c machine-controller rsh machine-api-controllers-656cb464b5-t8xr2
sh-4.4$ env | grep -i proxy
HTTP_PROXY=http://dummy:dummy@172.16.0.3:3128/
NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.16.0.0/24,172.30.0.0/16,api-int.ostest.shiftstack.com,localhost
HTTPS_PROXY=https://dummy:dummy@172.16.0.3:3130/

Comment 13 errata-xmlrpc 2021-09-29 14:10:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.7.32 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3636