Bug 1565861
Summary: | enable_ssh_admin task unable to signal undercloud due to SSL handshake error | ||
---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | John Fulton <johfulto> |
Component: | ceph-ansible | Assignee: | John Fulton <johfulto> |
Status: | CLOSED NOTABUG | QA Contact: | Yogev Rabl <yrabl> |
Severity: | low | Docs Contact: | |
Priority: | low | ||
Version: | 13.0 (Queens) | CC: | akaris, ebarrera, gfidente, gkadam, ukalifon |
Target Milestone: | ga | Keywords: | Triaged |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2018-04-11 03:01:04 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
John Fulton
2018-04-11 00:12:37 UTC
/var/log/messages on controller node (192.168.213.201) shows that the tripleo-admin user was created BUT that heat was unable to communicate this back to the undercloud because of SSL hand shake error: Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:11,966] (heat-config) [INFO] {"deploy_stdout": "\nPLAY [localhost] ***************************************************************\n\nTASK [Gather ing Facts] *********************************************************\nok: [localhost]\n\nTASK [create user tripleo-admin] ***********************************************\nchanged: [localhost]\n\nTASK [grant admi n rights to user tripleo-admin] ********************************\nchanged: [localhost]\n\nTASK [ensure .ssh dir exists for user tripleo-admin] ***************************\nchanged: [localhost]\n\nTASK [ensure au thorized_keys file exists for user tripleo-admin] ***************\nchanged: [localhost]\n\nTASK [authorize TripleO Mistral key for user tripleo-admin] ********************\nchanged: [localhost]\n\nPLAY RECAP *** ******************************************************************\nlocalhost : ok=6 changed=5 unreachable=0 failed=0 \n\n", "deploy_stderr": "", "deploy_status_code": 0} Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:11,966] (heat-config) [DEBUG] [2018-04-09 16:15:07,047] (heat-config) [DEBUG] Running ansible-playbook -i localhost, /var/lib/heat-config/heat-con fig-ansible/8dc0ec12-e105-404e-af40-5bdde7543cc8_playbook.yaml --extra-vars @/var/lib/heat-config/heat-config-ansible/8dc0ec12-e105-404e-af40-5bdde7543cc8_variables.json Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:11,962] (heat-config) [INFO] Return code 0 Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:11,962] (heat-config) [INFO] Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: PLAY [localhost] *************************************************************** Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: TASK [Gathering Facts] ********************************************************* Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: ok: [localhost] Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: TASK [create user tripleo-admin] *********************************************** Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: changed: [localhost] Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: TASK [grant admin rights to user tripleo-admin] ******************************** Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: changed: [localhost] Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: TASK [ensure .ssh dir exists for user tripleo-admin] *************************** Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: changed: [localhost] Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: TASK [ensure authorized_keys file exists for user tripleo-admin] *************** Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: changed: [localhost] Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: TASK [authorize TripleO Mistral key for user tripleo-admin] ******************** Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: changed: [localhost] Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: PLAY RECAP ********************************************************************* Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: localhost : ok=6 changed=5 unreachable=0 failed=0 Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:11,962] (heat-config) [INFO] Completed /var/lib/heat-config/heat-config-ansible/8dc0ec12-e105-404e-af40-5bdde7543cc8_playbook.yaml Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:11,966] (heat-config) [INFO] Completed /usr/libexec/heat-config/hooks/ansible Apr 9 16:15:11 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:11,966] (heat-config) [DEBUG] Running heat-config-notify /var/lib/heat-config/deployed/8dc0ec12-e105-404e-af40-5bdde7543cc8.json < /var/lib/heat-config/deployed/8dc0ec12-e105-404e-af40-5bdde7543cc8.notify.json Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:12,572] (heat-config) [INFO] Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:12,573] (heat-config) [ERROR] Error running heat-config-notify. [1] Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: [2018-04-09 16:15:12,573] (heat-config) [ERROR] [2018-04-09 16:15:12,517] (heat-config-notify) [DEBUG] Signaling to https://192.168.213.2:13808/v1/AUTH_3bc5279c95b74b46bf0362a0929ad107/create_admin-4d0f4c6f-5843-440e-9322-09fb0999fb76/dcc32759-9e21-43e7-8125-75d2907d1628?temp_url_sig=e7194f36c40f1627f2d614af3ce47145fd9b563b&temp_url_expires=1523322887 via PUT Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: Traceback (most recent call last): Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: File "/usr/bin/heat-config-notify", line 179, in <module> Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: sys.exit(main(sys.argv, sys.stdin)) Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: File "/usr/bin/heat-config-notify", line 126, in main Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: headers={'content-type': 'application/json'}) Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in put Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: return self.request('PUT', url, data=data, **kwargs) Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 518, in request Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: resp = self.send(prep, **send_kwargs) Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 639, in send Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: r = adapter.send(request, **kwargs) Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 512, in send Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: raise SSLError(e, request=request) Apr 9 16:15:12 rhosp-ctrl0 os-collect-config: requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",) ROOT CAUSE The task which created the tripleo-admin user on the overcloud was unable to signal to the undercloud that it succeeded because it got an SSL handshake error. The undercloud didn't receive a message back from the overcloud so it timed out. I've seen this error in the past and addressed it by either: 1. Adding "generate_service_certificate = false" to my undercloud.conf 2. Using inject-trust-anchor.yaml as per https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ssl.html It looks like neither of the above are the case in this deployment [1] I'm closing this as not a bug as things are working as expected given the inputs/outputs. [1] A. undercloud.conf contained: #undercloud_service_certificate = /etc/pki/instack-certs/undercloud.pem generate_service_certificate = true certificate_generation_ca = local B. deployment command was: openstack overcloud deploy --templates --ntp-server 10.5.27.10 -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml --environment-directory /home/stack/mytemplates/env --roles-file /home/stack/mytemplates/rhosp-roles-data.yaml --networks-file /home/stack/mytemplates/rhosp-network-data.yaml --stack rhosp --debug --log-file overcloudDeploy.log No trust anchor settings in /home/stack/mytemplates/env (undercloud) [stack@rhosp-director env]$ ll total 40 -rw-r--r--. 1 stack stack 1935 Apr 9 19:28 25-rhosp-networks.yaml -rw-rw-r--. 1 stack stack 4602 Apr 9 19:28 30-ips-from-pools.yaml -rw-r--r--. 1 stack stack 764 Apr 9 19:28 35-rhosp-IP-pools.yaml -rw-r--r--. 1 stack stack 1349 Apr 9 19:28 50-rhosp-ext-ceph.yaml -rw-r--r--. 1 stack stack 879 Apr 9 19:28 60-sat6-registration.yaml -rw-r--r--. 1 stack stack 351 Apr 9 19:28 70-rhosp-hostmap.yaml -rw-rw-r--. 1 stack stack 6641 Apr 9 19:28 90-rhosp-rhos13-images.yaml -rw-r--r--. 1 stack stack 556 Apr 9 19:28 99-rhosp-misc.yaml (undercloud) [stack@rhosp-director env]$ *** Bug 1637013 has been marked as a duplicate of this bug. *** |