Bug 2068450

Summary: httplib request failed: ConnectionRefusedError: [Errno 111] Connection refused
Product: Red Hat Enterprise Linux 8 Reporter: Sudhir Menon <sumenon>
Component: ipaAssignee: Florence Blanc-Renaud <frenaud>
Status: CLOSED MIGRATED QA Contact: ipa-qe <ipa-qe>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.6CC: edewata, ftrivino, pasik, rcritten, tscherf
Target Milestone: rcKeywords: MigratedToJIRA, Regression, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-18 18:58:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sudhir Menon 2022-03-25 11:11:41 UTC
Description of problem: httplib request failed: ConnectionRefusedError: [Errno 111] Connection refused

Version-Release number of selected component (if applicable):
pki-ca-10.12.0-2.module+el8.6.0+14115+8b467244.noarch
pki-kra-10.12.0-2.module+el8.6.0+14115+8b467244.noarch
pki-server-10.12.0-2.module+el8.6.0+14115+8b467244.noarch
ipa-server-4.9.8-7.module+el8.6.0+14337+19b76db2.x86_64
389-ds-base-1.4.3.28-6.module+el8.6.0+14129+983ceada.x86_64
selinux-policy-3.14.3-94.el8.noarch

How reproducible:
Always

Steps to Reproduce:
1. Install IPA server and check the ipaserver-install.log file

Actual results:
Although the IPA server install is successful, there is traceback message in the install log due to which httplib request fails

2022-03-21T17:21:50Z DEBUG   [24/29]: importing IPA certificate profiles
2022-03-21T17:21:50Z DEBUG request GET https://ip-addr.testrealm.test:443/ca/rest/account/login
2022-03-21T17:21:50Z DEBUG request body ''
2022-03-21T17:21:50Z DEBUG httplib request failed:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ipapython/dogtag.py", line 272, in _httplib_request
    conn.request(method, path, body=request_body, headers=headers)
  File "/usr/lib64/python3.6/http/client.py", line 1273, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1319, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1268, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1044, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 982, in send
    self.connect()
  File "/usr/lib64/python3.6/http/client.py", line 1433, in connect
    super().connect()
  File "/usr/lib64/python3.6/http/client.py", line 954, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib64/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/lib64/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
2022-03-21T17:21:50Z DEBUG Overriding CA port: cannot connect to 'https://ip-addr.testrealm.test:443/ca/rest/account/login': [Errno 111] Connection refused

Expected results:
Fix the issue causing the traceback.

Additional info:

Comment 2 Rob Crittenden 2022-03-25 12:27:53 UTC
Moving back to ipa as the connection originates there. This may be related to gymnastics we have to do when loading profiles against the local server.

Note that the hostname isn't an IP address. It's a hostname that contains the address within it for automation purposes.

Interestingly this isn't a traceback. The logging includes exc_info=True so the logging prints the traceback but this isn't an uncaught error.

Comment 3 Rob Crittenden 2022-03-25 12:34:57 UTC
This is intended behavior for the most part, not necessarily the traceback, but IMHO its worth keeping the exc_info=True.

This is related to upstream issue https://pagure.io/freeipa/issue/9100 and is commit https://github.com/freeipa/freeipa/commit/8c241869dda779d38fee4c23e44c89eb81a2a4d4

Basically it is trying port 443 first to connect to the CA to import the profiles which is failing because we haven't configured or started Apache yet. That fails so it next tries 8443 which is available only to the local machine (no open firewall port). That succeeds and installation continues.

Comment 4 Endi Sukma Dewata 2022-04-12 15:20:43 UTC
Resetting Assignee and QE Contact to default.

Comment 5 Trivino 2022-08-29 11:27:38 UTC
@sumenon do you know if this is still happening?

Comment 7 Rob Crittenden 2022-08-29 14:53:41 UTC
The decision to make is whether to disable exc_info=True since it produces a scary traceback.

This is otherwise a red herring.

Comment 8 Florence Blanc-Renaud 2023-06-20 08:30:12 UTC
The relevant portion of code is https://github.com/freeipa/freeipa/blob/ae6549ffae1ffe2bb6a1ba7dce0620ec0c20cabf/ipapython/dogtag.py#L269-L280:

    try:
        conn = connection_factory(host, port, **connection_options)
        conn.request(method, path, body=request_body, headers=headers)
        res = conn.getresponse()

        http_status = res.status
        http_headers = res.msg
        http_body = res.read()
        conn.close()
    except Exception as e:
        logger.debug("httplib request failed:", exc_info=True)
        raise NetworkError(uri=uri, error=str(e))


If we remove exc_info=True, the traceback won't be printed any more in the debug logs but we will still see the next message:
2022-03-21T17:21:50Z DEBUG httplib request failed:
2022-03-21T17:21:50Z DEBUG Overriding CA port: cannot connect to 'https://ip-addr.testrealm.test:443/ca/rest/account/login': [Errno 111] Connection refused

IMO we can remove exc_info=True as the debug would contain enough information anyway.

Comment 9 Rob Crittenden 2023-06-20 13:14:07 UTC
I dug a little deeper. This traceback was originally displayed to the user and was addressed in https://pagure.io/freeipa/issue/6674 with 53c8e9a53f026d83d5328896d1ea0cf72690cf24 . The log level was changed from exception to debug and exc_info=True was added. I don't think that in the ipactl case it adds much value, but dogtag.https_request is called in other contexts too where a connection failure is unexpected. Perhaps we can add an argument to https_request to provide the traceback or not.

Comment 11 RHEL Program Management 2023-09-18 18:56:27 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 12 RHEL Program Management 2023-09-18 18:58:48 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.