Bug 1354667

Hi there, i think i'm the guy who reported this via the red hat support system but we never actually mentioned 7.1 in the tickets. I'm curious if this bug really applies to 7.1 as well because it's gone for us in 7.2 and reappears in 7.3 and 7.4 beta.

(Just to let you know).

Sincerely,
Tim Kopplow

Tim,

> it's gone for us in 7.2 and reappears in 7.3 and 7.4 beta

What do you mean by "gone for us in 7.2"? If you can please elaborate on exact behaviors, that would be very helpful.

Sorry for being so unspecific. The phrasing was also a bit incorrect. 
What i meant was that the subscription manager works perfectly fine for us in 7.2 but not in 7.3 and above. I was confused that after Josh Foots tested it it was marked as a bug in 7.1. We haven't tested it in 7.1 but that would be quite weird, wouldn't it?

> Broken in 7.1, Fixed in 7.2, and back to be broken in 7.3.

I'm going to test it with the release version of 7.4 tomorrow.

Alright. Tested it again in 7.4 (release) and the issue still occurs. What other infos could be useful for you?

> What other infos could be useful for you?

The behavior of the proxy with respect to consumer certificates. If the proxy is intercepting (man-in-the-middle) communication between subscription-manager and RHSM, then what certificate is it using to re-encrypt the communication? Especially does this behavior differ between 7.2 and 7.3?

One possible cause for the difference in behavior: the default URL for RHSM changed in 7.3 to subscription.rhsm.redhat.com from subscription.rhn.redhat.com .  You can try changing it back (by editing /etc/rhsm/rhsm.conf) on a 7.3 install to confirm if this is the issue. If this is the issue your proxy probably needs updated configuration (to configure subscription.rhsm.redhat.com similarly to subscription.rhn.redhat.com).

Yes Kevin,
The default hostname was changed from "subscription.rhn.redhat.com" to "subscription.rhsm.redhat.com" by RFE Bug 1278472 and first introduced by python-rhsm-1.16.6-1 and newer.

This could easily be the cause for an SSL error for a proxy environment that is only configured to pass the former "subscription.rhn.redhat.com".  For compatibility, customers should update their proxy servers to tolerate both hostnames:  "subscription.rhn.redhat.com" and "subscription.rhsm.redhat.com".

That's absolutely the reason and resolves our problems. I edited the URL in the config file and it worked like it is supposed to. I'm going to hand over this to the people that are in charge for our proxy.

I assume then that we're just the one who didn't get the memo? I don't know how i should've known that this action is required now. 

Anyhow, thanks for the help!

Reproducer:

[root@bkr-hv03-guest37 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 2.2.0-1
subscription management rules: 5.26
subscription-manager: 1.19.21-1.el7
python-rhsm: 1.19.9-1.el7


[root@bkr-hv03-guest37 ~]# subscription-manager config --server.proxy_hostname=auto-services.usersys.redhat.com --server.proxy_port=3128 --server.proxy_user=redhat --server.proxy_password=redhat --server.insecure=1

[root@bkr-hv03-guest37 ~]# subscription-manager register --serverurl=F21-candlepin.usersys.redhat.com:8443/candlepin --force
Registering to: F21-candlepin.usersys.redhat.com:8443/candlepin
Username: admin
Password: 
Organization: admin
The system has been registered with ID: a7120960-b37d-47ee-bd40-c7d3a6ac7bae 
1 local certificate has been deleted.
[root@bkr-hv03-guest37 ~]# subscription-manager identity
system identity: a7120960-b37d-47ee-bd40-c7d3a6ac7bae
name: bkr-hv03-guest37.dsal.lab.eng.bos.redhat.com
org name: Admin Owner
org ID: admin
[root@bkr-hv03-guest37 ~]# subscription-manager config --server.proxy_port=3130
[root@bkr-hv03-guest37 ~]# subscription-manager identity
Invalid credentials.


RHSM.log:
2017-11-13 06:02:54,577 [INFO] subscription-manager:870:MainThread @managercli.py:518 - X-Correlation-ID: be871dfc0a69403891ee16d17539f532
2017-11-13 06:02:54,577 [INFO] subscription-manager:870:MainThread @managercli.py:407 - Client Versions: {'python-rhsm': '1.19.9-1.el7', 'subscription-manager': '1.19.21-1.el7'}
2017-11-13 06:02:54,578 [INFO] subscription-manager:870:MainThread @connection.py:822 - Connection built: http_proxy=auto-services.usersys.redhat.com:3130 host=F21-candlepin.usersys.redhat.com port=8443 handler=/candlepin auth=identity_cert ca_dir=/etc/rhsm/ca/ insecure=True
2017-11-13 06:02:54,578 [INFO] subscription-manager:870:MainThread @connection.py:822 - Connection built: http_proxy=auto-services.usersys.redhat.com:3130 host=F21-candlepin.usersys.redhat.com port=8443 handler=/candlepin auth=none
2017-11-13 06:02:54,593 [INFO] subscription-manager:870:MainThread @managercli.py:382 - Consumer Identity name=bkr-hv03-guest37.dsal.lab.eng.bos.redhat.com uuid=a7120960-b37d-47ee-bd40-c7d3a6ac7bae
2017-11-13 06:02:56,022 [INFO] subscription-manager:870:MainThread @connection.py:552 - Response: status=401, requestUuid=ce10644c-f213-4878-be6d-f27b240a2c8c, request="GET /candlepin/consumers/a7120960-b37d-47ee-bd40-c7d3a6ac7bae/owner"
2017-11-13 06:02:56,023 [ERROR] subscription-manager:870:MainThread @managercli.py:780 - Invalid credentials.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/subscription_manager/managercli.py", line 747, in _do_command
    owner = self.cp.getOwner(consumerid)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 1090, in getOwner
    return self.conn.request_get(method)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 646, in request_get
    return self._request("GET", method, headers=headers)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 672, in _request
    info=info, headers=headers)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 561, in _request
    self.validateResponse(result, request_type, handler)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 610, in validateResponse
    raise RestlibException(response['status'], error_msg, response.get('headers'))
RestlibException: Invalid credentials.

Verification:

[root@dhcp35-121 rhn]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 2.2.0-1
subscription management rules: 5.26
subscription-manager: 1.20.5-1.el7

1.Bad registration credentials with good proxy credentials throws invalid credentials error:

[root@dhcp35-121 rhn]# subscription-manager register --username=testuser1 --password=BAD_password --proxy=auto-services.usersys.redhat.com:3128 --proxyuser=redhat --proxypassword=redhat --force
Unregistering from: F21-candlepin.usersys.redhat.com:8443/candlepin
The system with UUID 0818f11c-2ba9-454a-a6db-f0b04c0f7b96 has been unregistered
All local data removed
Registering to: F21-candlepin.usersys.redhat.com:8443/candlepin
Invalid Credentials

2.The new feedback "Unable to make a connection using SSL client certificate. Please review proxy configuration and connectivity." is thrown when an Unauthorized 401 response is received while employing a configured proxy server.


[root@dhcp35-121 rhn]# subscription-manager config --server.proxy_hostname=auto-services.usersys.redhat.com --server.proxy_port=3128 --server.proxy_user=redhat --server.proxy_password=redhat --server.insecure=1
[root@dhcp35-121 rhn]# 
[root@dhcp35-121 rhn]# subscription-manager register --force
Unregistering from: F21-candlepin.usersys.redhat.com:8443/candlepin
The system with UUID d7eaa7c1-2b47-4d9b-aa7a-1bec77c6c37b has been unregistered
All local data removed
Registering to: F21-candlepin.usersys.redhat.com:8443/candlepin
Username: admin
Password: 
Organization: admin
The system has been registered with ID: 0818f11c-2ba9-454a-a6db-f0b04c0f7b96
The registered system name is: dhcp35-121.lab.eng.blr.redhat.com
[root@dhcp35-121 rhn]# subscription-manager identity 
system identity: 0818f11c-2ba9-454a-a6db-f0b04c0f7b96
name: dhcp35-121.lab.eng.blr.redhat.com
org name: Admin Owner
org ID: admin
[root@dhcp35-121 rhn]# subscription-manager config --server.proxy_port=3130
[root@dhcp35-121 rhn]# subscription-manager identity
Unable to make a connection using SSL client certificate. Please review proxy configuration and connectivity.

Tail from rhsm.log:

2017-11-13 16:37:44,614 [INFO] subscription-manager:22872:MainThread @connection.py:836 - Connection built: http_proxy=auto-services.usersys.redhat.com:3130 host=F21-candlepin.usersys.redhat.com port=8443 handler=/candlepin auth=none
2017-11-13 16:37:44,626 [INFO] subscription-manager:22872:MainThread @managercli.py:317 - Consumer Identity name=dhcp35-121.lab.eng.blr.redhat.com uuid=0818f11c-2ba9-454a-a6db-f0b04c0f7b96
2017-11-13 16:37:47,394 [INFO] subscription-manager:22872:MainThread @connection.py:556 - Response: status=401, requestUuid=e64d67fd-e1de-4d2e-84d5-99975386a697, request="GET /candlepin/consumers/0818f11c-2ba9-454a-a6db-f0b04c0f7b96/owner"
2017-11-13 16:37:47,395 [ERROR] subscription-manager:22872:MainThread @managercli.py:715 - Unable to make a connection using SSL client certificate. Please review proxy configuration and connectivity.
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/subscription_manager/managercli.py", line 682, in _do_command
    owner = self.cp.getOwner(consumerid)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 1104, in getOwner
    return self.conn.request_get(method)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 660, in request_get
    return self._request("GET", method, headers=headers)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 686, in _request
    info=info, headers=headers)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 565, in _request
    self.validateResponse(result, request_type, handler)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 620, in validateResponse
    response.get('headers'))
RestlibException: Unable to make a connection using SSL client certificate. Please review proxy configuration and connectivity.
2017-11-13 16:37:47,396 [ERROR] subscription-manager:22872:MainThread @managercli.py:716 - Error: Unable to generate a new identity for the system: Unable to make a connection using SSL client certificate. Please review proxy configuration and connectivity.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0681