Bug 2084337 - Ingresscontroller reconcilations failing but not shown in operator logs or status of ingresscontroller.
Summary: Ingresscontroller reconcilations failing but not shown in operator logs or st...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.8
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.8.z
Assignee: Miciah Dashiel Butler Masters
QA Contact: Arvind iyengar
URL:
Whiteboard:
Depends On: 2084336
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-11 22:09 UTC by Miciah Dashiel Butler Masters
Modified: 2022-09-21 09:14 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Before OpenShift 4.8, the IngressController API did not have any subfields under the "status.endpointPublishingStrategy.hostNetwork" and "status.endpointPublishingStrategy.nodePort" fields. As result, these fields could be null even if the "spec.endpointPublishingStrategy.type" field was set to "HostNetwork" or "NodePortService". OpenShift 4.8 added the "status.endpointPublishingStrategy.hostNetwork.protocol" and "status.endpointPublishingStrategy.nodePort.protocol" subfields, and the ingress operator now sets default values for these subfields when the operator admits or re-admits an IngressController that specifies the "HostNetwork" or "NodePortService" strategy type, respectively. However, a cluster that was upgraded from an earlier version of OpenShift could have an already admitted IngressController with null values for these status fields even when the IngressController specified the "HostNetwork" or "NodePortService" endpoint publishing strategy type. In this case, the operator ignored updates to these spec fields. Consequence: Updating "spec.endpointPublishingStrategy.hostNetwork.protocol" or "spec.endpointPublishingStrategy.nodePort.protocol" to "PROXY" to enable PROXY protocol on an existing IngressController had no effect, and it was necessary to delete and recreate the IngressController to enable PROXY protocol. Fix: The ingress operator was changed so that it correctly updates the status fields when "status.endpointPublishingStrategy.hostNetwork" or "status.endpointPublishingStrategy.nodePort" is null and the IngressController's spec fields specify PROXY protocol with the "HostNetwork" or "NodePortService" endpoint publishing strategy type, respectively. Result: Setting "spec.endpointPublishingStrategy.hostNetwork.protocol" or "spec.endpointPublishingStrategy.nodePort.protocol" to "PROXY" now takes proper effect on upgraded clusters.
Clone Of: 2084336
Environment:
Last Closed: 2022-09-21 09:14:38 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-ingress-operator pull 758 0 None open [release-4.8] Bug 2084337: Fix enabling PROXY protocol on an upgraded cluster 2022-07-15 01:01:19 UTC
Red Hat Product Errata RHBA-2022:6511 0 None None None 2022-09-21 09:14:53 UTC

Comment 1 Arvind iyengar 2022-07-19 06:22:10 UTC 
Verified with "4.8.0-0.ci.test-2022-07-19-051628-ci-ln-0d4rz1k-latest" build with the fix. With this release, it is observed that that the "PROXY" protocol type gets properly applied on the routers pods:
------
Before change:
oc get clusterversion                                              
NAME      VERSION                                                  AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.ci.test-2022-07-19-051628-ci-ln-0d4rz1k-latest   True        False         109s    Cluster version is 4.8.0-0.ci.test-2022-07-19-051628-ci-ln-0d4rz1k-latest


oc -n openshift-ingress-operator get ingresscontroller default -ojsonpath='{.status.endpointPublishingStrategy}' | jq .
{
  "hostNetwork": {
    "protocol": "TCP"
  },
  "type": "HostNetwork"
}

oc -n openshift-ingress get pods -o wide                                
oc 0n oNAME                              READY   STATUS    RESTARTS   AGE   IP           NODE                                 NOMINATED NODE   READINESS GATES
router-default-5fc745dd6c-9757p   1/1     Running   0          23m   10.0.1.33    0d4rz1k-b5564-m424v-worker-0-vkt5l   <none>           <none>
router-default-5fc745dd6c-97d7k   1/1     Running   0          23m   10.0.2.137   0d4rz1k-b5564-m424v-worker-0-vqlz8   <none>           <none>


After change:
 oc -n openshift-ingress-operator get ingresscontroller default -ojsonpath='{.spec.endpointPublishingStrategy}' | jq .
{
  "hostNetwork": {
    "protocol": "PROXY"
  },
  "type": "HostNetwork"
}

oc -n openshift-ingress-operator get ingresscontroller default -ojsonpath='{.status.endpointPublishingStrategy}' | jq .
{
  "hostNetwork": {
    "protocol": "PROXY"
  },
  "type": "HostNetwork"
}

 oc -n openshift-ingress get pods -o wide                                 
NAME                              READY   STATUS    RESTARTS   AGE   IP           NODE                                 NOMINATED NODE   READINESS GATES
router-default-64bbdbc845-vtfzw   1/1     Running   0          51s   10.0.1.33    0d4rz1k-b5564-m424v-worker-0-vkt5l   <none>           <none>
router-default-64bbdbc845-z8b6t   1/1     Running   0          86s   10.0.0.248   0d4rz1k-b5564-m424v-worker-0-m2msr   <none>           <none>

oc -n openshift-ingress exec router-default-64bbdbc845-vtfzw -- env | grep ROUTER_USE_PROXY_PROTOCOL
ROUTER_USE_PROXY_PROTOCOL=true

oc -n openshift-ingress exec router-default-64bbdbc845-vtfzw -- cat haproxy.config | grep "accept-proxy"
  bind :80 accept-proxy
  bind :443 accept-proxy

------
Comment 6 errata-xmlrpc 2022-09-21 09:14:38 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.50 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:6511
Note You need to log in before you can comment on or make changes to this bug.