Bug 2093454

Summary: Router proxy protocol doesn't work with dual-stack (IPv4 and IPv6) clusters
Product: OpenShift Container Platform Reporter: Grant Spence <gspence>
Component: NetworkingAssignee: Grant Spence <gspence>
Networking sub component: router QA Contact: Arvind iyengar <aiyengar>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: medium CC: aos-bugs, hongli, mmasters
Version: 4.11   
Target Milestone: ---   
Target Release: 4.11.0   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: A change to the HAProxy configuration template in OpenShift 4.8 caused the "accept-proxy" option not to be set on all "bind" lines when the configuration had more than one "bind". Consequence: On dual-stack clusters with PROXY protocol configured, PROXY protocol was only enabled for IPv6 and was not enabled for IPv4. Fix: The HAProxy configuration template was corrected to set "accept-proxy" on every "bind" line when PROXY protocol is configured. Result: OpenShift now enables PROXY protocol for both IPv4 and IPv6 on dual-stack clusters with PROXY protocol configured.
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-08-10 11:16:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 2096362    

Description Grant Spence 2022-06-03 18:37:38 UTC
Description of problem:
There is a logic error in the haproxy template code that the "accept-proxy" specifier doesn't get appropriately applied to both IPv4 and IPv6 haproxy interfaces if BOTH IPv4 and IPv6 are enabled.

The "accept-proxy" specifier is added via when the ENV variable ROUTER_USE_PROXY_PROTOCOL is true.

OpenShift release version:

Cluster Platform:

How reproducible:

Steps to Reproduce (in detail):
1. Enable IPv4 and IPv6 via ROUTER_IP_V4_V6_MODE="v4v6" on router deployment
2. Set ROUTER_USE_PROXY_PROTOCOL to true on router deployment
3. RSH into router and confirm that "accept-proxy" is on both "bind :<PORT>" and "bind :::<PORT>" lines for "frontend public" and "frontend public_ssl"

Actual results:
"accept-proxy" is only on "bind :::<PORT>" and missing from "bind :<PORT>"

Expected results:
"accept-proxy" should be on both "bind :<PORT>" and "bind :::<PORT>"

Impact of the problem:
Can't have a dual stack IPv4 and IPv6 configuration with "accept-protocol" on both stacks.

Additional info:

** Please do not disregard the report template; filling the template out as much as possible will allow us to help you. Please consider attaching a must-gather archive (via `oc adm must-gather`). Please review must-gather contents for sensitive information before attaching any must-gathers to a bugzilla report.  You may also mark the bug private if you wish.

Comment 2 Miciah Dashiel Butler Masters 2022-06-06 14:21:28 UTC
The issue was caused by <https://github.com/openshift/router/pull/83>, which shipped in 4.8.0, to fix bug 1801407.

Comment 6 Arvind iyengar 2022-06-16 09:33:28 UTC
Verified in "4.11.0-0.nightly-2022-06-15-222801". With this payload it is observed that "accept-proxy" specifier are getting properly applied for ipv4 and ipv6 bind option in the frontend:
  oc get clusterversion 
  NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
  version   4.11.0-0.nightly-2022-06-15-222801   True        False         49m     Cluster version is 4.11.0-0.nightly-2022-06-15-222801

oc -n openshift-ingress get deployment.apps/router-internalapps2 -o yaml | grep -ie ROUTER_IP_V4_V6_MODE -ie ROUTER_USE_PROXY_PROTOCOL -A1
        - name: ROUTER_IP_V4_V6_MODE
          value: v4v6
          value: "true"

Inside the router pod:
  frontend public

  bind :9080 accept-proxy
  bind :::9080 v6only accept-proxy
  mode http
  tcp-request inspect-delay 5s
  tcp-request content accept if HTTP
  monitor-uri /_______internal_router_healthz

frontend public_ssl
  option tcplog
  bind :9443 accept-proxy
  bind :::9443 v6only accept-proxy
  tcp-request inspect-delay 5s
  tcp-request content accept if { req_ssl_hello_type 1 }

Comment 7 errata-xmlrpc 2022-08-10 11:16:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.