Bug 1835882

Summary: While executing "Configure Cloud Connector" playbook on Satellite 6.7 server it does not honour HTTP Proxy setting
Product: Red Hat Satellite Reporter: Ashish Humbe <ahumbe>
Component: RH Cloud - Cloud ConnectorAssignee: Adam Ruzicka <aruzicka>
Status: CLOSED ERRATA QA Contact: Lukáš Hellebrandt <lhellebr>
Severity: high Docs Contact:
Priority: high    
Version: 6.7.0CC: achadha, ahuchcha, alsanche, aruzicka, ben.formosa, bshahu, dmule, dvoss, egolov, jjeffers, jpathan, jyejare, mawerner, mhulan, michiel.smit, mmccune, pcreech, pete.perfetti
Target Milestone: 6.8.0Keywords: PrioBumpGSS, Regression, TestBlocker, Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: receptor-0.6.3, ansiblerole-satellite-receptor-installer-0.6.12, tfm-rubygem-foreman_ansible-5.0.2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1851155 1862239 1862277 (view as bug list) Environment:
Last Closed: 2020-10-27 13:02:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1861422    
Bug Blocks:    

Description Ashish Humbe 2020-05-14 16:44:05 UTC
Description of problem:

While executing "Configure Cloud Connector" playbook on Satellite 6.7 server it does not honour HTTP Proxy setting. 

Here is the output of playbook where we can see "Failed to connect to cert.cloud.redhat.com at port 443: [Errno 101] Network is unreachable\" error message 

  10: TASK [project-receptor.satellite_receptor_installer : Can connect to cloud.redhat.com] ***
  11: fatal: [satellite.example.com]: FAILED! => {"changed": false, "msg": "Failed to connect to cert.cloud.redhat.com at port 443: [Errno 101] Network is unreachable", "status": -1, "url": "https://cert.cloud.redhat.com/api"}
  12:

Using TCPdump customer has confirmed that the connection request to cert.cloud.redhat.com does not go through HTTP proxy

Version-Release number of selected component (if applicable):
Satellite 6.7

How reproducible:
Always when there is HTTP Proxy in the picture. 

Steps to Reproduce:
1. Configure HTTP Proxy on the satellite server
2. Try to execute "Configure Cloud Connector" playbook on Satellite 6.7 to setup Cloud Connector 
3. 

Actual results:
Connection request does not go through HTTP Proxy server. 

Expected results:
HTTP Proxy setting should be honoured while executing "Configure Cloud Connector" playbook.

Additional info:
There is an upstream issue: https://github.com/project-receptor/receptor/issues/191

Comment 1 Marek Hulan 2020-05-21 09:39:21 UTC
The receptor 0.6.3 honors HTTPS_PROXY env variable. The wss:// connection will then be proxied. However the proxy must be HTTP not HTTPS, this is the limitation of python asyncio library. In other words

HTTPS_PROXY=http://my-proxy-server       <-- okay
HTTPS_PROXY=https://my-proxy-server      <-- not okay

Matt/David, this should be documented in Satellite product documentation for the Cloud Connector, do you need specific BZ for the documentation or can we use this one? This should most likely be mentioned somewhere around https://access.redhat.com/documentation/en-us/red_hat_insights/2020-04/html-single/remediating_issues_across_your_red_hat_satellite_infrastructure_using_red_hat_insights/index?lb_target=production#configuring-cloud-connector-on-satellite-server_remediating-satellite. Note this was tested with Squid proxy. Please reach out to Graham Mainwaring for more information about the receptor.

Adam, can you please take a look if this could be enhanced in the playbook so that the env variable could be set as ansible variable? If not, this can be moved to POST, but if there are more chahnges required on our side, we need to address that too.

Thanks all

Comment 2 Adam Ruzicka 2020-05-21 09:47:27 UTC
I think it could be set. Is having it as a user input good enough?

Comment 8 Matt Werner 2020-05-27 12:33:12 UTC
Thanks for adding the Docs BZ, David.

Comment 9 Adam Ruzicka 2020-06-04 10:46:23 UTC
Created redmine issue http://projects.theforeman.org/issues/30013 from this bug

Comment 10 Lukáš Hellebrandt 2020-07-15 12:59:12 UTC
FailedQA with Sat 6.8 snap 8.0.

This caused regression. After Receptor installation, the receptor can't connect (c.rh.c says "Connection issue - Receptor not responding" when trying to execute a remediation) and the following gets spammed in log:

    ==> /var/log/messages <==
    Jul 15 08:42:57 dhcp-3-176 receptor: ERROR 2020-07-15 08:42:57,905 449ac547-22af-4ecf-9966-c8716614b175 ws ws.connect
    Jul 15 08:42:57 dhcp-3-176 receptor: Traceback (most recent call last):
    Jul 15 08:42:57 dhcp-3-176 receptor: File "/usr/lib/python3.6/site-packages/receptor/connection/ws.py", line 53, in connect
    Jul 15 08:42:57 dhcp-3-176 receptor: proxy=proxy, proxy_auth=proxy_auth
    Jul 15 08:42:57 dhcp-3-176 receptor: File "/usr/lib64/python3.6/site-packages/aiohttp/client.py", line 1012, in __aenter__
    Jul 15 08:42:57 dhcp-3-176 receptor: self._resp = await self._coro
    Jul 15 08:42:57 dhcp-3-176 receptor: File "/usr/lib64/python3.6/site-packages/aiohttp/client.py", line 728, in _ws_connect
    Jul 15 08:42:57 dhcp-3-176 receptor: proxy_headers=proxy_headers)
    Jul 15 08:42:57 dhcp-3-176 receptor: File "/usr/lib64/python3.6/site-packages/aiohttp/client.py", line 473, in _request
    Jul 15 08:42:57 dhcp-3-176 receptor: ssl=ssl, proxy_headers=proxy_headers, traces=traces)
    Jul 15 08:42:57 dhcp-3-176 receptor: File "/usr/lib64/python3.6/site-packages/aiohttp/client_reqrep.py", line 291, in __init__
    Jul 15 08:42:57 dhcp-3-176 receptor: self.update_proxy(proxy, proxy_auth, proxy_headers)
    Jul 15 08:42:57 dhcp-3-176 receptor: File "/usr/lib64/python3.6/site-packages/aiohttp/client_reqrep.py", line 519, in update_proxy
    Jul 15 08:42:57 dhcp-3-176 receptor: raise ValueError("Only http proxies are supported")
    Jul 15 08:42:57 dhcp-3-176 receptor: ValueError: Only http proxies are supported
    Jul 15 08:42:57 dhcp-3-176 receptor: Unclosed client session
    Jul 15 08:42:57 dhcp-3-176 receptor: client_session: <aiohttp.client.ClientSession object at 0x7f524a135518>

Note that I haven't even specified any proxy yet, this hit me while testing something else.

Comment 11 Lukáš Hellebrandt 2020-07-15 13:58:04 UTC
# cat /etc/systemd/system/receptor\@.service 
[Unit]
Description=Receptor Node for %i
After=network.target

[Service]
ExecStart=/usr/bin/receptor -c /etc/receptor/%i/receptor.conf -d /var/data/receptor/%i node
Environment=HTTP_PROXY=""
Environment=HTTPS_PROXY=""

[Install]
WantedBy=multi-user.target

Comment 17 Adam Ruzicka 2020-08-13 19:05:40 UTC
*** Bug 1868739 has been marked as a duplicate of this bug. ***

Comment 24 errata-xmlrpc 2020-10-27 13:02:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366