Description of problem: After the overcloud is deployed (possibly not completely) running any heat command returns authorization or other failures as in: failed: [undercloud] => {"changed": true, "cmd": "source /home/stack/stackrc; heat output-show overcloud KeystoneURL | grep -E -o \"([0-9]{1,3}[\\.]){3}[0-9]{1,3}\";", "delta": "0:00:01.001769", "end": "2015-06-16 14:08:20.375668", "rc": 1, "start": "2015-06-16 14:08:19.373899", "warnings": []} 14:08:19 stderr: Unable to establish connection to http://192.0.2.6:8004/v1/%(tenant_id)s/stacks/overcloud failed: [undercloud] => {"changed": true, "cmd": "source /home/stack/stackrc; heat output-show overcloud KeystoneURL | grep -E -o \"([0-9]{1,3}[\\.]){3}[0-9]{1,3}\";", "delta": "0:00:01.001769", "end": "2015-06-16 14:08:20.375668", "rc": 1, "start": "2015-06-16 14:08:19.373899", "warnings": []} 14:08:19 stderr: Unable to establish connection to http://192.0.2.6:8004/v1/%(tenant_id)s/stacks/overcloud Possibly the overcloud deploy never completes correctly: 1:25:40 source /home/stack/stackrc; if [ -f "/home/stack/deploy-overcloudrc" ]; then 11:25:40 source /home/stack/deploy-overcloudrc; 11:25:40 fi; openstack overcloud deploy --plan-uuid 5b5fdea0-f001-4b8b-ae7b-d94615dc2b03; 11:25:40 11:25:40 start: 11:25:40 2015-06-16 11:13:40.746207 11:25:40 11:25:40 end: 11:25:40 2015-06-16 11:25:41.433885 11:25:40 11:25:40 delta: 11:25:40 0:12:00.687678 11:25:40 11:25:40 stdout: 11:25:40 The following cert files already exist, use --rebuild to remove the existing files before regenerating: 11:25:40 /etc/keystone/ssl/certs/ca.pem already exists 11:25:40 /etc/keystone/ssl/private/signing_key.pem already exists 11:25:40 /etc/keystone/ssl/certs/signing_cert.pem already exists 11:25:40 The following templates will be written: 11:25:40 /tmp/tmpHIH6Gw/puppet/manifests/overcloud_volume.pp 11:25:40 /tmp/tmpHIH6Gw/hieradata/object.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/hieradata/common.yaml 11:25:40 /tmp/tmpHIH6Gw/provider-Swift-Storage-1.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/controller-config-pacemaker.yaml 11:25:40 /tmp/tmpHIH6Gw/provider-Cinder-Storage-1.yaml 11:25:40 /tmp/tmpHIH6Gw/provider-Compute-1.yaml 11:25:40 /tmp/tmpHIH6Gw/network/noop.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/bootstrap-config.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/cinder-storage-post.yaml 11:25:40 /tmp/tmpHIH6Gw/provider-Ceph-Storage-1.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/controller-post-puppet.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/cinder-storage-puppet.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/manifests/overcloud_cephstorage.pp 11:25:40 /tmp/tmpHIH6Gw/puppet/hieradata/object.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/controller-puppet.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/manifests/overcloud_compute.pp 11:25:40 /tmp/tmpHIH6Gw/net-config-bridge.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/swift-storage-post.yaml 11:25:40 /tmp/tmpHIH6Gw/provider-Controller-1.yaml 11:25:40 /tmp/tmpHIH6Gw/network/networks.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/manifests/overcloud_object.pp 11:25:40 /tmp/tmpHIH6Gw/hieradata/controller.yaml 11:25:40 /tmp/tmpHIH6Gw/network/ports/ctlplane_vip.yaml 11:25:40 /tmp/tmpHIH6Gw/hieradata/volume.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/compute-post-puppet.yaml 11:25:40 /tmp/tmpHIH6Gw/extraconfig/tasks/yum_update.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/swift-storage-puppet.yaml 11:25:40 /tmp/tmpHIH6Gw/extraconfig/tasks/yum_update.sh 11:25:40 /tmp/tmpHIH6Gw/puppet/swift-devices-and-proxy-config.yaml 11:25:40 /tmp/tmpHIH6Gw/network/ports/net_ip_map.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/compute-puppet.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/hieradata/volume.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/ceph-storage-post-puppet.yaml 11:25:40 /tmp/tmpHIH6Gw/extraconfig/controller/noop.yaml 11:25:40 /tmp/tmpHIH6Gw/network/ports/noop.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/ceph-storage-puppet.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/hieradata/ceph.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/vip-config.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/hieradata/controller.yaml 11:25:40 /tmp/tmpHIH6Gw/plan.yaml 11:25:40 /tmp/tmpHIH6Gw/environment.yaml 11:25:40 /tmp/tmpHIH6Gw/network/ports/net_ip_list_map.yaml 11:25:40 /tmp/tmpHIH6Gw/hieradata/compute.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/hieradata/compute.yaml 11:25:40 /tmp/tmpHIH6Gw/hieradata/ceph.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/manifests/overcloud_controller_pacemaker.pp 11:25:40 /tmp/tmpHIH6Gw/hieradata/common.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/manifests/ringbuilder.pp 11:25:40 /tmp/tmpHIH6Gw/extraconfig/post_deploy/default.yaml 11:25:40 /tmp/tmpHIH6Gw/net-config-noop.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/ceph-cluster-config.yaml 11:25:40 /tmp/tmpHIH6Gw/firstboot/userdata_default.yaml 11:25:40 /tmp/tmpHIH6Gw/puppet/all-nodes-config.yaml 11:25:40 PKI initialization in init-keystone is deprecated and will be removed. 11:25:40 11:25:40 stderr: 11:25:40 /usr/lib/python2.7/site-packages/novaclient/v1_1/__init__.py:30: UserWarning: Module novaclient.v1_1 is deprecated (taken as a basis for novaclient.v2). The preferable way to get client class or object you can find in novaclient.client module. 11:25:40 warnings.warn("Module novaclient.v1_1 is deprecated (taken as a basis for " 11:25:40 Warning: Permanently added '192.0.2.6' (ECDSA) to the list of known hosts. 11:25:40 Connection to 192.0.2.6 closed. 11:25:40 (note no overcloud deployed message) Version-Release number of selected component (if applicable): Poodle rpms: [stack@instack ~]$ rpm -qa | grep openstack openstack-nova-console-2015.1.0-11.el7ost.noarch openstack-neutron-2015.1.0-8.el7ost.noarch openstack-ironic-conductor-2015.1.0-4.el7ost.noarch openstack-ceilometer-alarm-2015.1.0-2.el7ost.noarch openstack-swift-account-2.3.0-1.el7ost.noarch python-django-openstack-auth-1.2.0-2.el7ost.noarch openstack-tuskar-ui-0.3.0-2.el7ost.noarch openstack-utils-2014.2-1.el7ost.noarch openstack-heat-api-cloudwatch-2015.1.0-3.el7ost.noarch openstack-ceilometer-notification-2015.1.0-2.el7ost.noarch openstack-neutron-openvswitch-2015.1.0-8.el7ost.noarch openstack-nova-api-2015.1.0-11.el7ost.noarch openstack-tripleo-heat-templates-0.8.6-9.el7ost.noarch openstack-tripleo-image-elements-0.9.6-1.el7ost.noarch python-openstackclient-1.0.3-2.el7ost.noarch openstack-ironic-discoverd-1.1.0-3.el7ost.noarch openstack-tripleo-puppet-elements-0.0.1-2.el7ost.noarch openstack-swift-object-2.3.0-1.el7ost.noarch openstack-tripleo-0.0.6-0.1.git812abe0.el7ost.noarch openstack-nova-common-2015.1.0-11.el7ost.noarch openstack-heat-common-2015.1.0-3.el7ost.noarch openstack-tuskar-0.4.18-2.el7ost.noarch openstack-dashboard-theme-2015.1.0-10.el7ost.noarch openstack-tuskar-ui-extras-0.0.4-1.el7ost.noarch openstack-tempest-kilo-20150507.2.el7ost.noarch openstack-swift-2.3.0-1.el7ost.noarch openstack-neutron-ml2-2015.1.0-8.el7ost.noarch openstack-nova-novncproxy-2015.1.0-11.el7ost.noarch openstack-keystone-2015.1.0-1.el7ost.noarch openstack-swift-plugin-swift3-1.7-3.el7ost.noarch openstack-tripleo-common-0.0.1.dev6-0.git49b57eb.el7ost.noarch openstack-neutron-common-2015.1.0-8.el7ost.noarch openstack-heat-engine-2015.1.0-3.el7ost.noarch openstack-ceilometer-common-2015.1.0-2.el7ost.noarch openstack-heat-api-cfn-2015.1.0-3.el7ost.noarch openstack-ceilometer-api-2015.1.0-2.el7ost.noarch openstack-ironic-api-2015.1.0-4.el7ost.noarch openstack-swift-proxy-2.3.0-1.el7ost.noarch openstack-heat-templates-0-0.6.20150605git.el7ost.noarch openstack-ceilometer-collector-2015.1.0-2.el7ost.noarch openstack-ironic-common-2015.1.0-4.el7ost.noarch openstack-selinux-0.6.32-1.el7ost.noarch openstack-nova-compute-2015.1.0-11.el7ost.noarch openstack-nova-conductor-2015.1.0-11.el7ost.noarch openstack-swift-container-2.3.0-1.el7ost.noarch redhat-access-plugin-openstack-7.0.0-0.el7ost.noarch openstack-glance-2015.1.0-6.el7ost.noarch openstack-heat-api-2015.1.0-3.el7ost.noarch openstack-ceilometer-central-2015.1.0-2.el7ost.noarch openstack-puppet-modules-2015.1.6-1.el7ost.noarch openstack-nova-scheduler-2015.1.0-11.el7ost.noarch openstack-nova-cert-2015.1.0-11.el7ost.noarch openstack-dashboard-2015.1.0-10.el7ost.noarch [stack@instack ~]$ rpm -qa | grep python-rdomanager-oscplugin python-rdomanager-oscplugin-0.0.7-2.el7ost.noarch How reproducible: always on poodle jobs Steps to Reproduce: 1. Install from latest poodle 2. Deploy overcloud - see results, try heat commands afterwards 3. Compare with RDO job results Actual results: Failure to deploy and complete post steps Expected results: Deploy should pass and post deploy steps should succeed Additional info:
Reproduced.
Brad, will you please check to see if this is still an issue?
python-rdomanager-oscplugin downstream was not fully up to where midstream was. After a rebase, this went away.
No issues that I can see on virt or on BM
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2015:1549