Description of problem: This happens when trying to convert Director endpoints from non-SSL to SSL according to https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/7/html/Director_Installation_and_Usage/sect-Advanced-Scenario_3_Using_the_CLI_to_Create_an_Advanced_Overcloud_with_Ceph_Nodes.html#sect-Advanced-Enabling_SSL_TLS_on_the_Overcloud Initial deployment with this step-by-step guide works. However, if I first deploy a new environment without SSL, and then try to convert it to SSL, the following happens: - endpoints are badly configured Version-Release number of selected component (if applicable): 7.3 How reproducible: all of the time Steps to Reproduce: 1. deploy Director without SSL 2. configure SSL 3. redeploy Director Actual results: +----------------------------------+-----------+--------------------------------------------+-----------------------------------------------+-----------------------------------------+----------------------------------+ | id | region | publicurl | internalurl | adminurl | service_id | +----------------------------------+-----------+--------------------------------------------+-----------------------------------------------+-----------------------------------------+----------------------------------+ | 1ac53a0aee6a4c84bf627ca5132bd7ab | regionOne | http://10.0.0.4:8004/v1/%(tenant_id)s | http://172.16.2.4:8004/v1/%(tenant_id)s | http://172.16.2.4:8004/v1/%(tenant_id)s | 3e36d03c27d64dc9a3e2fc8842a057ef | | 2a007e52e36e4d5baddc52cff8951a8e | regionOne | http://10.0.0.4:9696/ | http://172.16.2.4:9696/ | http://172.16.2.4:9696/ | c6cbcd1d39d34a119d189511541960b2 | | 6150a642cc8c46b7a75e0b784ecf86f8 | regionOne | http://10.0.0.4:8776/v1/%(tenant_id)s | http://172.16.2.4:8776/v1/%(tenant_id)s | http://172.16.2.4:8776/v1/%(tenant_id)s | 95fc966bc34e48649ab11b5d5db9bb91 | | 77841d30267c4bfc8f71545140fad4d9 | regionOne | http://10.0.0.4:8777/ | http://172.16.2.4:8777/ | http://172.16.2.4:8777/ | 27a420a5a7cc40e18b33b50fda80f672 | | a98aea83fe934149b99d9cbc451ba7fb | regionOne | http://10.0.0.4:9292/ | http://172.18.0.10:9292/ | http://172.18.0.10:9292/ | 4aff5165a98a42d39765f33eceb8529c | | abb934a1eb2d4889804a2b607746a46c | regionOne | http://10.0.0.4:8774/v2/$(tenant_id)s | http://172.16.2.4:8774/v2/$(tenant_id)s | http://172.16.2.4:8774/v2/$(tenant_id)s | 5a2c224b493c49be8ac189d2a0739f7c | | b208bb10f6a14a06bae32e05cf0bff3c | regionOne | http://10.0.0.4:5000/v2.0 | http://172.16.2.4:5000/v2.0 | http://192.0.2.19:35357/v2.0 | 2b1a9869752c462099c389bbda5bcddd | | c2fc9d47aa704424addb734c4d079357 | regionOne | http://10.0.0.4:8774/v3 | http://172.16.2.4:8774/v3 | http://172.16.2.4:8774/v3 | 021e736d8fda486dbff2f28ca39bc5b1 | | e8c3f6bb31ee4bdb93f8e364949ae137 | regionOne | http://10.0.0.4:8080/v1/AUTH_%(tenant_id)s | http://172.18.0.10:8080/v1/AUTH_%(tenant_id)s | http://172.18.0.10:8080/v1 | bc489c937e124a499de535da454f17f0 | | f784e74c37b24c1bb2add4202fc93d75 | regionOne | http://10.0.0.4:8776/v2/%(tenant_id)s | http://172.16.2.4:8776/v2/%(tenant_id)s | http://172.16.2.4:8776/v2/%(tenant_id)s | d0cd68bc7908434caf509c0de8272f67 | | fb27d5a10ef94b88adfcc392dd8dfa43 | regionOne | http://10.0.0.4:80/dashboard/ | http://10.0.0.4:80/dashboard/ | http://10.0.0.4:80/dashboard/admin | 6b2fa795cff1452db858fe2e27e12766 | +----------------------------------+-----------+--------------------------------------------+-----------------------------------------------+-----------------------------------------+----------------------------------+ Expected results: +----------------------------------+-----------+-----------------------------------------------------+-----------------------------------------------+-------------------------------------------+----------------------------------+ | id | region | publicurl | internalurl | adminurl | service_id | +----------------------------------+-----------+-----------------------------------------------------+-----------------------------------------------+-------------------------------------------+----------------------------------+ | 088182a67c504d47ba0e0e29e4642c85 | regionOne | https://osp.example.net:13808/v1/AUTH_%(tenant_id)s | http://172.18.0.10:8080/v1/AUTH_%(tenant_id)s | http://172.18.0.10:8080/v1 | f5ce3d3fc3d04fdc9bf59ea51211b629 | | 0f352e5b58174b4489d34becfdc618f1 | regionOne | https://osp.example.net:13000/v2.0 | http://172.16.2.5:5000/v2.0 | http://192.0.2.8:35357/v2.0 | 93d3a2f1655849abb4954ef2769f608a | | 22b1f36030be47b4a685f9c35558cd50 | regionOne | https://osp.example.net:13696/ | http://172.16.2.5:9696/ | http://172.16.2.5:9696/ | 4302251c1ad34db58afcd48f57eb1e5f | | 3d3cc3fd12754d138d5297de9646cab3 | regionOne | https://osp.example.net:13776/v2/%(tenant_id)s | http://172.16.2.5:8776/v2/%(tenant_id)s | http://172.16.2.5:8776/v2/%(tenant_id)s | 52453f5a1d1943cc8d2f70f4dea63b6d | | 40a1d0feb6374667857f333ccfaebe08 | regionOne | http://osp.example.net:80/dashboard/ | http://osp.example.net:80/dashboard/ | http://osp.example.net:80/dashboard/admin | 0c954194a95c471b9353b7e5c4803d0c | | 6a789ea0bc2a40f3a1852e3d1b3aeb05 | regionOne | https://osp.example.net:13774/v3 | http://172.16.2.5:8774/v3 | http://172.16.2.5:8774/v3 | 0c62e551fd9a4dd7926a1c5038ae75ca | | 8cc0d3b9fc9f4801b418eddb438f8541 | regionOne | https://osp.example.net:13292/ | http://172.18.0.10:9292/ | http://172.18.0.10:9292/ | 2d0e665d9e0e4da998aa526e3f5d78ea | | a35e87a654814c319478c6f74c3d8ea1 | regionOne | https://osp.example.net:13776/v1/%(tenant_id)s | http://172.16.2.5:8776/v1/%(tenant_id)s | http://172.16.2.5:8776/v1/%(tenant_id)s | 7bff84c8f0754bcb894b31a46484ddca | | b3ff4bdc519c4210a2653ea7fe9a1915 | regionOne | https://osp.example.net:13774/v2/$(tenant_id)s | http://172.16.2.5:8774/v2/$(tenant_id)s | http://172.16.2.5:8774/v2/$(tenant_id)s | 347d290e609d478aac7e97d9d2991ac3 | | e408e6690f3643839e9b8d9b228e5532 | regionOne | https://osp.example.net:13004/v1/%(tenant_id)s | http://172.16.2.5:8004/v1/%(tenant_id)s | http://172.16.2.5:8004/v1/%(tenant_id)s | a64fc99cdc8b4ec98d12ad869388e9df | | fb1b8bcc47bd4c4c8497067f28b77962 | regionOne | https://osp.example.net:13777/ | http://172.16.2.5:8777/ | http://172.16.2.5:8777/ | 433fe43e0d254426b5ec1d4b6b0ac567 | +----------------------------------+-----------+-----------------------------------------------------+-----------------------------------------------+-------------------------------------------+----------------------------------+ Additional info:
So I think that the issue is here: rdomanager_oscplugin/v1/overcloud_deploy.py (...) keystone.setup_endpoints( services, client=keystone_client, os_auth_url=overcloud_endpoint, public_host=overcloud_ip_or_fqdn) (...) Which at some point creates endpoints by means of this helper function which will simply abandon if endpoints already exist: os_cloud_config/keystone.py (...) def _create_endpoint(keystone, region, service_id, public_uri, admin_uri, internal_uri): """Helper for idempotent creating of endpoint. :param keystone: keystone v2 client :param region: endpoint region :param service_id: id of associated service :param public_uri: endpoint public uri :param admin_uri: endpoint admin uri :param internal_uri: endpoint internal uri """ if keystone.endpoints.findall(publicurl=public_uri): LOG.info('Endpoint for service %s and public uri %s ' 'already exists.', service_id, public_uri) else: LOG.debug('Creating endpoint for service %s.', service_id) keystone.endpoints.create( region, service_id, public_uri, admin_uri, internal_uri) (...)
that is indeed the issue; os-cloud-config is only run once in the post config. It isn't run on updates... We need to stop using that and take puppet into use for creating/updating the keystone endpoints. Because, AFAIK, os-cloud-config doesn't even have the capabilities of updating the endpoints at all.
Currently, we do not support migrating from non-SSL to SSL without a complete re-deployment. That makes this request an RFE.
Tested on openstack-tripleo-heat-templates-5.0.0-0.6.0rc3.el7ost It looks that we're missing a step to do 'pcs resource haproxy restart' while running the overcloud deploy command which enables ssl. After the overcloud update is complete I'm not able to reach the haproxy endpoints and a haproxy restart is required.
Marius, could you check if doing systemctl reload haproxy works? instead of doing pcs resource haproxy restart. Doing a reload of the configuration should work, and I have the feeling this bug https://bugs.launchpad.net/tripleo/+bug/1627254 is hitting us there.
So, it seems that the issue is that we're not restarting haproxy via pacemaker anymore, so I've submitted a fix to work around that issue.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html