Bug 1308542 - overcloud deploy complete with Authorization Failed
Summary: overcloud deploy complete with Authorization Failed
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 8.0 (Liberty)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 8.0 (Liberty)
Assignee: Angus Thomas
QA Contact: yeylon@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-02-15 13:21 UTC by Asaf Hirshberg
Modified: 2020-06-11 12:48 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-16 13:13:17 UTC
Target Upstream Version:


Attachments (Terms of Use)
controller-0 keystone log (12.78 KB, text/plain)
2016-02-15 13:21 UTC, Asaf Hirshberg
no flags Details
controller-1 keystone log (4.49 KB, text/plain)
2016-02-15 13:21 UTC, Asaf Hirshberg
no flags Details
controller-2 keystone log (4.37 KB, text/plain)
2016-02-15 13:22 UTC, Asaf Hirshberg
no flags Details

Description Asaf Hirshberg 2016-02-15 13:21:25 UTC
Created attachment 1127273 [details]
controller-0 keystone log

Description of problem:

2016-02-15 12:44:48 [overcloud]: CREATE_COMPLETE  Stack CREATE completed successfully
Stack overcloud CREATE_COMPLETE
/home/stack/.ssh/known_hosts updated.
Original contents retained as /home/stack/.ssh/known_hosts.old
PKI initialization in init-keystone is deprecated and will be removed.
Warning: Permanently added '192.0.2.8' (ECDSA) to the list of known hosts.
 No handlers could be found for logger "oslo_config.cfg"
2016-02-15 07:45:21.257 3332 WARNING keystone.cmd.cli [-] keystone-manage pki_setup is not recommended for production use.
The following cert files already exist, use --rebuild to remove the existing files before regenerating:
/etc/keystone/ssl/certs/ca.pem already exists
/etc/keystone/ssl/private/signing_key.pem already exists
/etc/keystone/ssl/certs/signing_cert.pem already exists
Connection to 192.0.2.8 closed.
Authorization Failed: Unable to establish connection to http://10.35.180.10:5000/v2.0/tokens
[stack@puma33 ~]$ heat stack-list
+--------------------------------------+------------+-----------------+---------------------+--------------+
| id                                   | stack_name | stack_status    | creation_time       | updated_time |
+--------------------------------------+------------+-----------------+---------------------+--------------+
| 8bc81aab-75d6-474d-962d-87daa3bec1d2 | overcloud  | CREATE_COMPLETE | 2016-02-15T12:02:15 | None         |
+--------------------------------------+------------+-----------------+---------------------+--------------+

[root@overcloud-controller-0 ~]# openstack-status
== Nova services ==
openstack-nova-api:                     active    (disabled on boot)
openstack-nova-cert:                    inactive  (disabled on boot)
openstack-nova-compute:                 inactive  (disabled on boot)
openstack-nova-network:                 inactive  (disabled on boot)
openstack-nova-scheduler:               active    (disabled on boot)
openstack-nova-conductor:               active    (disabled on boot)
== Glance services ==
openstack-glance-api:                   active    (disabled on boot)
openstack-glance-registry:              active    (disabled on boot)
== Keystone service ==
openstack-keystone:                     active    (disabled on boot)
== Horizon service ==
openstack-dashboard:                    uncontactable
== neutron services ==
neutron-server:                         active    (disabled on boot)
neutron-dhcp-agent:                     active    (disabled on boot)
neutron-l3-agent:                       active    (disabled on boot)
neutron-metadata-agent:                 active    (disabled on boot)
neutron-lbaas-agent:                    inactive  (disabled on boot)
neutron-openvswitch-agent:              active    (disabled on boot)
neutron-metering-agent:                 inactive  (disabled on boot)
== Swift services ==
openstack-swift-proxy:                  active
openstack-swift-account:                active
openstack-swift-container:              active
openstack-swift-object:                 active
== Cinder services ==
openstack-cinder-api:                   active    (disabled on boot)
openstack-cinder-scheduler:             active    (disabled on boot)
openstack-cinder-volume:                active    (disabled on boot)
openstack-cinder-backup:                inactive  (disabled on boot)
== Ceilometer services ==
openstack-ceilometer-api:               active    (disabled on boot)
openstack-ceilometer-central:           active    (disabled on boot)
openstack-ceilometer-compute:           inactive  (disabled on boot)
openstack-ceilometer-collector:         active    (disabled on boot)
openstack-ceilometer-alarm-notifier:    active    (disabled on boot)
openstack-ceilometer-alarm-evaluator:   active    (disabled on boot)
openstack-ceilometer-notification:      active    (disabled on boot)
== Heat services ==
openstack-heat-api:                     active    (disabled on boot)
openstack-heat-api-cfn:                 active    (disabled on boot)
openstack-heat-api-cloudwatch:          active    (disabled on boot)
openstack-heat-engine:                  active    (disabled on boot)
== Support services ==
mysqld:                                 inactive  (disabled on boot)
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
rabbitmq-server:                        inactive  (disabled on boot)
memcached:                              active    (disabled on boot)
== Keystone users ==
Warning keystonerc not sourced
[root@overcloud-controller-0 ~]# 


Version-Release number of selected component (if applicable):
director 8 puddle - 2016-02-11.1

How reproducible:
3/3

Steps to Reproduce:
setup: 3-controllers, 3-computes, external-ceph
Deploy command:
openstack overcloud deploy -e /home/stack/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e ceph-external.yaml --templates --control-scale 3 --compute-scale 3 --ntp-server clock.redhat.com --timeout 180

Comment 1 Asaf Hirshberg 2016-02-15 13:21:59 UTC
Created attachment 1127274 [details]
controller-1 keystone log

Comment 2 Asaf Hirshberg 2016-02-15 13:22:23 UTC
Created attachment 1127275 [details]
controller-2 keystone log

Comment 3 Marius Cornea 2016-02-15 13:35:47 UTC
Hi Asaf, 

Are you able to ping 10.35.180.10 from the undercloud node? In case not this indicates a network connectivity issue and thus the error message:

Authorization Failed: Unable to establish connection to http://10.35.180.10:5000/v2.0/tokens

Comment 5 Asaf Hirshberg 2016-02-16 13:13:17 UTC
Closed.

Thanks Marius, you were right, the problem have caused by asymetric routing problem; disabled by default in rhel.
configuring enp5s0f0 in the controller nic template and disabling dhcp solved the problem.


Note You need to log in before you can comment on or make changes to this bug.