Bug 1837458 - Container health checks fail to honor no_proxy CIDR notation
Summary: Container health checks fail to honor no_proxy CIDR notation
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 16.2 (Train)
Hardware: All
OS: Unspecified
low
low
Target Milestone: ga
: 16.2 (Train on RHEL 8.4)
Assignee: Cédric Jeanneret
QA Contact: David Rosenfeld
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-05-19 13:48 UTC by Cody Swanson
Modified: 2023-09-07 23:10 UTC (History)
12 users (show)

Fixed In Version: openstack-tripleo-common-11.6.1-2.20210528132130.7599a38.el8ost.1
Doc Type: Release Note
Doc Text:
The no_proxy notation needs to be explained in a clearer way. Comma-separated list of hosts which do not use a proxy, if one is specified. The only wildcard is a single * character, which matches all hosts, and effectively disables the proxy. Each name in this list is matched as either a domain which contains the hostname, or the hostname itself. For example, local.com would match local.com, local.com:80, and www.local.com, but not www.notlocal.com.
Clone Of:
Environment:
Last Closed: 2021-09-15 07:07:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 771422 0 None MERGED Conditionally use python instead of cURL 2021-07-28 08:37:28 UTC
Red Hat Bugzilla 1828559 0 unspecified CLOSED healthchecks using curl are failing to honor no_proxy cidr notation 2023-09-07 23:05:30 UTC
Red Hat Issue Tracker OSP-6434 0 None None None 2022-08-10 16:23:09 UTC
Red Hat Knowledge Base (Solution) 5019691 0 None None None 2020-05-19 13:52:03 UTC
Red Hat Product Errata RHEA-2021:3483 0 None None None 2021-09-15 07:08:31 UTC

Description Cody Swanson 2020-05-19 13:48:50 UTC
Description of problem:

When when users follow our documented[1] process for installing the undercloud behind an HTTP/HTTPS proxy the install works as intended however the container health checks do not. 

We determined in bug 1828559 that this is a known limitation of curl, it does not follow CIDR notation in the no_proxy env. In a large environment it's not practical to individually list IP addresses so cidr notation is needed to ensure the undercloud doesn't try to send local traffic through the proxy server leading to communication failures. The use of no_proxy cidr notation appears to be correct, the podman containers all observed no_proxy cidr notation and the overcloud install worked fine. The only observed issue is that the container health check scripts ignore the no_proxy cidr addresses and fail to communicate with the running containers. 

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html-single/director_installation_and_usage/index#configuring-an-undercloud-proxy

Version-Release number of selected component (if applicable):

RHOSP 16.0

How reproducible:

Every time

Steps to Reproduce:
1. Deploy undercloud behind an HTTP/HTTPS proxy following our documentation.

Actual results:

Container health checks report errors even though the containers are operational.

Expected results:

Container health checks do try to use the designated http proxy for the subnets noted in the no_proxy env with cidr notation. 

Additional info:

The workaround is to disable container healthchecks on the undercloud.

Comment 2 Cédric Jeanneret 2020-06-10 11:13:39 UTC
Sooo..... the issue is: there isn't any real support for the no_proxy for CIDR notation. No RFE, nothing. no_proxy should even be a *domain* if we read the current state of the art and examples.....

Since the healthchecks are using cURL, we depend on their way to handle those variables and options. For the records, wget doesn't support CIDR notation afaik. There are endless discussions about this on different issue trackers, but no real things is done so far.

Since nothing properly describes the format NOR the way to handle those no_proxy, proxy_http and proxy_https variables (no RFE), everyone is doing it on their own and, of course, we have nice divergences.

So, as already said in different places, I'm not really sure WHAT we should do:
1. document that nope, CIDR notation aren't supported (and won't be)
2. implement our own thing in go, python, perl, haskell, whatever and maintain it
3. push for cURL or wget to implement something supporting CIDR notation (this suppose they handle the no_proxy as an IP address instead of a STRING, which is the case right now)

I would frankly go for the 1, since it seems to be the overall consensus at least at the tooling level...

Cheers,

C.

Comment 7 Cédric Jeanneret 2020-10-20 07:48:45 UTC
Some updates:
- moving to 16.1, since it's still a thing
- setting target milestone to z3, since it's a trivial doc update
- moving to MODIFIED - no code is needed, only a doc update
- adding some doc text content for the Doc Team to review

Comment 8 Cédric Jeanneret 2021-01-19 11:49:31 UTC
Quick update:
I'm currently trying to get a python script replacing cURL upstream: https://review.opendev.org/c/openstack/tripleo-common/+/771422

But it will be hard to get it in, since cURL is probably the most efficient and fastest app we can get for such a check..... Let's see how it goes but I'm not really optimistic.

Comment 10 Dan Macpherson 2021-02-25 06:17:24 UTC
Reassigning to 16.2 due to the patch landing early.

Comment 16 errata-xmlrpc 2021-09-15 07:07:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform (RHOSP) 16.2 enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:3483


Note You need to log in before you can comment on or make changes to this bug.