Scaling out an overcloud with nodes registered to a Satellite 6 server fails with "msg": "Unable to reach the server at $sat_url:443/subscription", "rc": 69
Previously, the Satellite URL was not correctly set in the role. This prevented the system from getting the Satellite server version, and registration failed. This fix adds the capability to get the `rhsm_satellite_url` value from the `rhsm_baseurl` parameter by default, passes the URL to the registration task to allow force registration, and adds the option to ignore certificate errors. You can override the default value or configure the options as needed.
This is happening due to the changes the katello-rhsm-consumer script is making to the system. I'm working to convert that script into Ansible tasks in order to make the role work reliably.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHEA-2019:0045
Description of problem: Scaling out an overcloud with nodes registered to a Sallite 6 server fails with "msg": "Unable to reach the server at $sat_url:443/subscription", "rc": 69 2018-10-19 15:40:17,817 p=27958 u=mistral | TASK [redhat-subscription : Manage Red Hat subscription] *********************** 2018-10-19 15:40:17,817 p=27958 u=mistral | Friday 19 October 2018 15:40:17 -0400 (0:00:00.430) 0:04:37.678 ******** 2018-10-19 15:40:19,327 p=27958 u=mistral | fatal: [controller-0]: FAILED! => {"changed": false, "cmd": "/sbin/subscription-manager register --baseurl https://rhos-compute-node-08.lab.eng.rdu2.redhat.com/pulp/repos --org Default_Organization --activationkey osp14-dev", "msg": "Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription", "rc": 69, "stderr": "Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription\n", "stderr_lines": ["Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription"], "stdout": "", "stdout_lines": []} 2018-10-19 15:40:19,331 p=27958 u=mistral | fatal: [controller-2]: FAILED! => {"changed": false, "cmd": "/sbin/subscription-manager register --baseurl https://rhos-compute-node-08.lab.eng.rdu2.redhat.com/pulp/repos --org Default_Organization --activationkey osp14-dev", "msg": "Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription", "rc": 69, "stderr": "Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription\n", "stderr_lines": ["Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription"], "stdout": "", "stdout_lines": []} 2018-10-19 15:40:19,347 p=27958 u=mistral | fatal: [controller-1]: FAILED! => {"changed": false, "cmd": "/sbin/subscription-manager register --baseurl https://rhos-compute-node-08.lab.eng.rdu2.redhat.com/pulp/repos --org Default_Organization --activationkey osp14-dev", "msg": "Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription", "rc": 69, "stderr": "Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription\n", "stderr_lines": ["Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription"], "stdout": "", "stdout_lines": []} 2018-10-19 15:40:19,349 p=27958 u=mistral | NO MORE HOSTS LEFT ************************************************************* 2018-10-19 15:40:19,350 p=27958 u=mistral | PLAY RECAP ********************************************************************* 2018-10-19 15:40:19,350 p=27958 u=mistral | ceph-0 : ok=76 changed=20 unreachable=0 failed=0 2018-10-19 15:40:19,350 p=27958 u=mistral | ceph-1 : ok=76 changed=20 unreachable=0 failed=0 2018-10-19 15:40:19,350 p=27958 u=mistral | ceph-2 : ok=76 changed=20 unreachable=0 failed=0 2018-10-19 15:40:19,351 p=27958 u=mistral | compute-0 : ok=76 changed=20 unreachable=0 failed=0 2018-10-19 15:40:19,351 p=27958 u=mistral | compute-1 : ok=76 changed=20 unreachable=0 failed=0 2018-10-19 15:40:19,351 p=27958 u=mistral | compute-2 : ok=69 changed=21 unreachable=0 failed=0 2018-10-19 15:40:19,351 p=27958 u=mistral | controller-0 : ok=158 changed=25 unreachable=0 failed=1 2018-10-19 15:40:19,351 p=27958 u=mistral | controller-1 : ok=158 changed=25 unreachable=0 failed=1 2018-10-19 15:40:19,351 p=27958 u=mistral | controller-2 : ok=160 changed=25 unreachable=0 failed=1 2018-10-19 15:40:19,351 p=27958 u=mistral | undercloud : ok=2 changed=0 unreachable=0 failed=0 2018-10-19 15:40:19,352 p=27958 u=mistral | Friday 19 October 2018 15:40:19 -0400 (0:00:01.534) 0:04:39.212 ******** 2018-10-19 15:40:19,352 p=27958 u=mistral | =============================================================================== Version-Release number of selected component (if applicable): openstack-tripleo-heat-templates-9.0.0-0.20181001174822.90afd18.0rc2.el7ost.noarch ansible-role-redhat-subscription-1.0.1-1.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy overcloud with nodes registered to satellite 2. Assert successful deployment 3. Run overcloud deploy command for a second time Actual results: Failure during TASK [redhat-subscription : Manage Red Hat subscription] Expected results: No failures. Additional info: Running the command manually on the overcloud nodes: [root@controller-0 heat-admin]# /sbin/subscription-manager register --baseurl https://rhos-compute-node-08.lab.eng.rdu2.redhat.com/pulp/repos --org Default_Organization --activationkey osp14-dev; echo $? Unable to reach the server at rhos-compute-node-08.lab.eng.rdu2.redhat.com:443/subscription 69 [root@controller-0 heat-admin]# subscription-manager status +-------------------------------------------+ System Status Details +-------------------------------------------+ Overall Status: Current [root@controller-0 heat-admin]# cat /var/log/rhsm/rhsm.log 2018-10-19 19:46:45,227 [INFO] subscription-manager:478159:MainThread @managercli.py:452 - X-Correlation-ID: 8c21206b44b94d049caecb85527f49cd 2018-10-19 19:46:45,227 [INFO] subscription-manager:478159:MainThread @managercli.py:341 - Client Versions: {'subscription-manager': '1.20.11-1.el7_5'} 2018-10-19 19:46:45,228 [INFO] subscription-manager:478159:MainThread @connection.py:868 - Connection built: host=rhos-compute-node-08.lab.eng.rdu2.redhat.com port=443 handler=/subscription auth=identity_cert ca_dir=/etc/rhsm/ca/ insecure=True 2018-10-19 19:46:45,229 [INFO] subscription-manager:478159:MainThread @connection.py:868 - Connection built: host=rhos-compute-node-08.lab.eng.rdu2.redhat.com port=443 handler=/subscription auth=none 2018-10-19 19:46:45,249 [INFO] subscription-manager:478159:MainThread @connection.py:586 - Response: status=404, request="GET /subscription/status/" 2018-10-19 19:46:45,250 [ERROR] subscription-manager:478159:MainThread @utils.py:149 - Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/subscription_manager/utils.py", line 141, in is_valid_server_info conn.ping() File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 929, in ping return self.conn.request_get("/status/") File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 690, in request_get return self._request("GET", method, headers=headers) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 716, in _request info=info, headers=headers) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 595, in _request self.validateResponse(result, request_type, handler) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 654, in validateResponse raise RestlibException(response['status'], error_msg, response.get('headers')) RestlibException