Description of problem: Ever since updating an OSP8 undercloud to openstack-tripleo-heat-templates-0.8.14-13.el7ost.noarch, I get the following output when deploying an overcloud: /etc/keystone/ssl/certs/ca.pem already exists /etc/keystone/ssl/private/signing_key.pem already exists /etc/keystone/ssl/certs/signing_cert.pem already exists Connection to 10.20.0.14 closed. Skipping "horizon" postconfig because it wasn't found in the endpoint map output <====== Overcloud Endpoint: http://10.20.0.14:5000/v2.0 Overcloud Deployed real 40m34.348s user 0m37.152s sys 0m5.424s This message ("Skipping "horizon" postconfig because it wasn't found in the endpoint map output") didn't show up with the previous version of the templates. Version-Release number of selected component (if applicable): I suspect it is openstack-tripleo-heat-templates-0.8.14-13.el7ost.noarch How reproducible: tested with the default templates and a simple deployment: openstack overcloud deploy --templates --libvirt-type kvm --control-scale 3 --compute-scale 1 --ceph-storage-scale 1 --swift-storage-scale 0 --control-flavor control --compute-flavor compute --ceph-storage-flavor ceph-storage --swift-storage-flavor swift-storage --neutron-network-type vxlan --neutron-tunnel-types vxlan --ntp-server 10.20.0.1 --validation-errors-fatal Steps to Reproduce: 1. Install OSP8 (with latest patches as of today 2016/06/15) 2. openstack overcloud deploy --templates --libvirt-type kvm --control-scale 3 --compute-scale 1 --ceph-storage-scale 1 --swift-storage-scale 0 --control-flavor control --compute-flavor compute --ceph-storage-flavor ceph-storage --swift-storage-flavor swift-storage --neutron-network-type vxlan --neutron-tunnel-types vxlan --ntp-server 10.20.0.1 --validation-errors-fatal 3. Actual results: The above warning shows up and horizon appears to work in my limited testing. I have no idea if something was broken recently. Expected results: Additional info: Can provide a sosreport if needed. Regards,
We are hitting this same problem. If you "source ~/overcloudrc" and then "keystone service-list", you get: +----------------------------------+------------+---------------+------------------------------+ | id | name | type | description | +----------------------------------+------------+---------------+------------------------------+ | 44302f2369c64c0698b022903b7a1478 | ceilometer | metering | Ceilometer Service | | e275e0f6a5a1491283be2ab68a0cc37e | cinder | volume | Cinder Volume Service | | c16bba4e36f74e84bbd8694d87a4c852 | cinderv2 | volumev2 | Cinder Volume Service v2 | | 661172dce77f4ecfb9ec50b0ab65ccfa | glance | image | Glance Image Service | | 46b244f835294b499466034caaac693e | heat | orchestration | Heat Service | | f291026064cb42c2858f39e2da82f5dd | keystone | identity | Keystone Identity Service | | c78c84745d2b49178de9346489581b75 | neutron | network | Neutron Service | | 5d189f554a364d5c983b82e125f6e21b | nova | compute | Nova Compute Service | | 5683d301b7b742bf8b59c1a4265c70aa | swift | object-store | Swift Object Storage Service | +----------------------------------+------------+---------------+------------------------------+ Note that there is no horizon endpoint in the above output.
We are also having this issue. However I don't think the issue is with the heat template but rather the trippleoclient. Downgrading this from python-tripleoclient-0.3.4-5.el7ost to python-tripleoclient-0.3.4-4.el7ost.noarch fixes our deployment. Looking at a diff between the packages I can see a number of changes have been made to the methods that setup the endpoints specifically in: /usr/lib/python2.7/site-packages/tripleoclient/utils.py /usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py I haven't done any detailed debug but I suspect the issues lies somewhere in this code. Cheers
We are also having the same issue. I did downgrading "python-tripleoclient-0.3.4-5.el7ost" to "python-tripleoclient-0.3.4-4.el7ost.noarch", then redeploy, it does NOT fixes the issue ! We tested deploy with 1 Controller + 1 Compute, it went through, changed to 3 controller + 1 Compute node, it failed with no "horizon" fully deployed, there is no "network tab" on "horizon". When login to Gui, it give messages "Error: Invalid service catalog service: compute" on TOP Right Window. ..... 2016-06-29 04:55:14 [overcloud]: CREATE_COMPLETE Stack CREATE completed successfully Stack overcloud CREATE_COMPLETE ..... No handlers could be found for logger "oslo_config.cfg" 2016-06-28 23:54:33.703 11795 WARNING keystone.cmd.cli [-] keystone-manage pki_setup is not recommended for production use. The following cert files already exist, use --rebuild to remove the existing files before regenerating: /etc/keystone/ssl/certs/ca.pem already exists /etc/keystone/ssl/private/signing_key.pem already exists /etc/keystone/ssl/certs/signing_cert.pem already exists Connection to 192.0.3.6 closed. The request you have made requires authentication. (HTTP 401) (Request-ID: req-72b49c48-fc5e-45b0-841f-2a225cf8b0c2) =========================================================================== $ source ~/overcloudrc $ keystone endpoint-list /usr/lib/python2.7/site-packages/keystoneclient/shell.py:64: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient. 'python-keystoneclient.', DeprecationWarning) /usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning: Constructing an instance of the keystoneclient.v2_0.client.Client class without a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release. 'the 2.0.0 release.', DeprecationWarning) /usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning: Using the 'tenant_name' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_name' argument instead super(Client, self).__init__(**kwargs) /usr/lib/python2.7/site-packages/debtcollector/renames.py:43: DeprecationWarning: Using the 'tenant_id' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_id' argument instead return f(*args, **kwargs) /usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:376: DeprecationWarning: Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release. 'the 2.0.0 release.', DeprecationWarning) +----------------------------------+-----------+-------------------------------+------------------------------+-----------------------------+----------------------------------+ | id | region | publicurl | internalurl | adminurl | service_id | +----------------------------------+-----------+-------------------------------+------------------------------+-----------------------------+----------------------------------+ | d0b1dbb2a28244309b47ce62833892e6 | regionOne | http://173.37.49.70:5000/v2.0 | http://172.16.1.10:5000/v2.0 | http://192.0.3.6:35357/v2.0 | 37da772860754b00b5b1c2eb2f53e504 | +----------------------------------+-----------+-------------------------------+------------------------------+-----------------------------+----------------------------------+ $ $ neutron agent-list publicURL endpoint for network service not found
In my case with errors, I find out it cause from one node slow in response within cluster, also NTP services has issues with hostname, I had to change NTP to point to IP address of the NTP server, that fixed all issues, the re-deploy went successfully. Fix we did: 1). downgrading "python-tripleoclient-0.3.4-5.el7ost" to "python-tripleoclient-0.3.4-4.el7ost.noarch"; 2). remove the slow response server out of "ironic node-list" pool. 3). change NTP server point from hostname to IP.
Workaround: yum downgrade python-tripleoclient-0.3.4-4.el7ost.noarch
KCS workaround verified & published: https://access.redhat.com/solutions/2446961
Hi, I confirm that the fix is working. Thanks for that. Regards,
On a fully patched undercloud with HEAT template package versions: openstack-tripleo-heat-templates.noarch 0:0.8.14-14.el7ost openstack-tripleo-heat-templates-kilo.noarch 0:0.8.14-14.el7ost downgrading just the tripleoclient to: python-tripleoclient-0.3.4-4.el7ost.noarch causes a new error: 2016-07-27 06:38:19 [overcloud]: CREATE_FAILED Resource CREATE failed: resources.Controller: Property error: resources[1].properties: Property MysqlClustercheckPassword not assigned Stack overcloud CREATE_FAILED Heat Stack create failed. You also need to run: sudo yum downgrade -y openstack-tripleo-heat-templates-kilo-0.8.14-11.el7ost.noarch openstack-tripleo-heat-templates-0.8.14-11.el7ost.noarch Can you confirm and update the KB appropriately?
I can confirm Charlie's result. Just downgraded tripleo client only in my OSP8 lab. Now seeing the MysqlClustercheckPassword error on overcloud deploy. 2016-07-27 15:09:47 [overcloud-Controller-2ink7cvzrmb2]: CREATE_IN_PROGRESS Stack CREATE started 2016-07-27 15:09:47 [overcloud-Controller-2ink7cvzrmb2]: CREATE_COMPLETE Stack CREATE completed successfully 2016-07-27 15:09:47 [0]: CREATE_IN_PROGRESS state changed 2016-07-27 15:09:48 [Controller]: CREATE_FAILED resources.Controller: Property error: resources[1].properties: Property MysqlClustercheckPassword not assigned
Confirm that downgrading tripleo-heat-templates got passed the MysqlClustercheckPassword error. 2016-07-27 15:20:51 [Controller]: CREATE_IN_PROGRESS state changed 2016-07-27 15:20:51 [overcloud-BlockStorage-ytl4omnx6kmv]: UPDATE_COMPLETE Stack UPDATE completed successfully 2016-07-27 15:20:51 [overcloud-Controller-fw2hcg2pbqnm]: CREATE_IN_PROGRESS Stack CREATE started 2016-07-27 15:20:52 [overcloud-Controller-fw2hcg2pbqnm]: CREATE_COMPLETE Stack CREATE completed successfully 2016-07-27 15:20:54 [Compute]: CREATE_IN_PROGRESS state changed 2016-07-27 15:20:54 [overcloud-Controller-fw2hcg2pbqnm]: UPDATE_IN_PROGRESS Stack UPDATE started 2016-07-27 15:20:54 [1]: CREATE_IN_PROGRESS state changed 2016-07-27 15:20:54 [overcloud-Compute-d2wa47xuvvgf]: CREATE_IN_PROGRESS Stack CREATE started 2016-07-27 15:20:54 [overcloud-Compute-d2wa47xuvvgf]: CREATE_COMPLETE Stack CREATE completed successfully 2016-07-27 15:20:55 [0]: CREATE_IN_PROGRESS state changed 2016-07-27 15:20:55 [overcloud-Compute-d2wa47xuvvgf]: UPDATE_IN_PROGRESS Stack UPDATE started 2016-07-27 15:20:56 [BlockStorage]: CREATE_COMPLETE state changed 2016-07-27 15:20:56 [overcloud-Controller-fw2hcg2pbqnm-1-xxknwzz46m4h]: CREATE_IN_PROGRESS Stack CREATE started Workaround: sudo yum downgrade -y openstack-tripleo-heat-templates-kilo-0.8.14-11.el7ost.noarch openstack-tripleo-heat-templates-0.8.14-11.el7ost.noarch
Verified no message about skipping Horizon PostConfig in my latest deployment after downgrading tripleo-client and tripleo-heat-templates. Stack overcloud CREATE_COMPLETE /home/stack/.ssh/known_hosts updated. Original contents retained as /home/stack/.ssh/known_hosts.old PKI initialization in init-keystone is deprecated and will be removed. Warning: Permanently added '192.168.104.100' (ECDSA) to the list of known hosts. No handlers could be found for logger "oslo_config.cfg" 2016-07-27 15:50:25.784 16828 WARNING keystone.cmd.cli [-] keystone-manage pki_setup is not recommended for production use. The following cert files already exist, use --rebuild to remove the existing files before regenerating: /etc/keystone/ssl/certs/ca.pem already exists /etc/keystone/ssl/private/signing_key.pem already exists /etc/keystone/ssl/certs/signing_cert.pem already exists Connection to 192.168.104.100 closed. Overcloud Endpoint: http://10.22.54.101:5000/v2.0 Overcloud Deployed
Since KCS solution intent is 1 issue, 1 solution; I've created another article to highlight this issue, and related it to the first solution. Customers who run into this issue should see this fix, and the previous fix in the linked solution. Thank you for verifying. KCS workaround verified & published: https://access.redhat.com/solutions/2472361
Hi, I can confirm Charlie's comment. Downgrading heat templates works. Thanks,
This also affects OSP9. python-tripleoclient-2.0.0-3.el7ost.noarch
Ran into the same issue on OSP 8 with python-tripleoclient-0.3.4-6.el7ost.noarch. Downgraded to python-tripleoclient-0.3.4-4.el7ost.noarch. Issue resolved.