Bug 1326587

Description of problem:
Fail to deploy OSP 7 using director. Installation failed at ControllerOvercloudServicesDeployment_Step7.

Version-Release number of selected component (if applicable):
instack.noarch               0.0.7-2.el7ost    @rhel-7-server-openstack-7.0-director-rpms
openstack-tripleo.noarch     0.0.7-0.1.1664e566.el7ost
                                               @rhel-7-server-openstack-7.0-director-rpms
openstack-tripleo-common.noarch
                             0.0.1.dev6-6.git49b57eb.el7ost
                                               @rhel-7-server-openstack-7.0-director-rpms
openstack-tripleo-heat-templates.noarch
                             0.8.6-123.el7ost  @rhel-7-server-openstack-7.0-director-rpms
openstack-tripleo-image-elements.noarch
                             0.9.6-10.el7ost   @rhel-7-server-openstack-7.0-director-rpms
openstack-tripleo-puppet-elements.noarch
                             0.0.1-5.el7ost    @rhel-7-server-openstack-7.0-director-rpms


How reproducible:
Run the deploy command (from deploy.sh):
#Deploy Controller + Compute + Ceph
time openstack overcloud deploy --templates ~/templates/my-overcloud -e ~/templates/my-overcloud/environments/network-isolation.yaml -e ~/templates/network-environment.yaml -e ~/templates/my-overcloud/environments/storage-environment.yaml -e ~/templates/userdata.yaml --control-scale 3 --compute-scale 2 --ceph-storage-scale 3 --control-flavor control --compute-flavor compute --ceph-storage-flavor ceph-storage --ntp-server pool.ntp.org --debug

with the attached templates

Steps to Reproduce:
1. Run deploy.sh command (show above)
2. Wait 60 minutes
3. Failed at Step7

Actual results:
{
  "status": "FAILED", 
  "server_id": "00521eb3-71c9-4675-9f19-09348411aca5", 
  "config_id": "df56311a-4ecb-46db-9a9f-58a46aa19d76", 
  "output_values": {
    "deploy_stdout": "\u001b[mNotice: Compiled catalog for controller-0.localdomain in environment production in 12.76 seconds\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Package_manifest[/var/lib/tripleo/installed-packages/overcloud_controller_pacemaker6]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Exec[galera-ready]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[controller-0]/File[/tmp/ceph-mon-keyring-controller-0]/ensure: defined content as '{md5}717dce14662e0eaf2b16b9f59847a134'\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[controller-0]/Exec[ceph-mon-mkfs-controller-0]/returns: ++ ceph-mon --id controller-0 --show-config-value mon_data\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[controller-0]/Exec[ceph-mon-mkfs-controller-0]/returns: + mon_data=/var/lib/ceph/mon/ceph-controller-0\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[controller-0]/Exec[ceph-mon-mkfs-controller-0]/returns: + '[' '!' -d /var/lib/ceph/mon/ceph-controller-0 ']'\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[controller-0]/Exec[ceph-mon-mkfs-controller-0]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[controller-0]/Exec[rm-keyring-controller-0]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[controller-0]/Exec[ceph-mon-ceph.client.admin.keyring-controller-0]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[controller-0]/Service[ceph-mon-controller-0]/ensure: ensure changed 'stopped' to 'running'\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns: Traceback (most recent call last):\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/bin/heat-keystone-setup-domain\", line 157, in <module>\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     main()\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/bin/heat-keystone-setup-domain\", line 113, in main\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     description=HEAT_DOMAIN_DESCRIPTION)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/utils.py\", line 318, in inner\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     return func(*args, **kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/v3/domains.py\", line 43, in create\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     **kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/base.py\", line 73, in func\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     return f(*args, **new_kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/base.py\", line 328, in create\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     self.key)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/base.py\", line 151, in _create\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     return self._post(url, body, response_key, return_raw, **kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/base.py\", line 165, in _post\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     resp, body = self.client.post(url, body=body, **kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/adapter.py\", line 176, in post\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     return self.request(url, 'POST', **kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/adapter.py\", line 206, in request\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/adapter.py\", line 95, in request\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     return self.session.request(url, method, **kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/utils.py\", line 318, in inner\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     return func(*args, **kwargs)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:   File \"/usr/lib/python2.7/site-packages/keystoneclient/session.py\", line 397, in request\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns:     raise exceptions.from_response(resp, method, url)\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns: keystoneclient.openstack.common.apiclient.exceptions.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-c63f2e2f-defa-442e-9920-ae39ad2d2bfd)\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Exec[enable-not-start-tripleo_cluster]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Exec[Set password for hacluster user on tripleo_cluster]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Exec[auth-successful-across-all-nodes]/returns: executed successfully\u001b[0m\n\u001b[mNotice: Pacemaker has reported quorum achieved\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Notify[pacemaker settled]/message: defined 'message' as 'Pacemaker has reported quorum achieved'\u001b[0m\n\u001b[mNotice: Finished catalog run in 38.80 seconds\u001b[0m\n", 
    "deploy_stderr": "Device \"br_ex\" does not exist.\nDevice \"br_int\" does not exist.\nDevice \"br_tun\" does not exist.\nDevice \"ovs_system\" does not exist.\n\u001b[1;31mWarning: Scope(Class[Keystone]): Execution of db_sync does not depend on $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Glance::Registry]): Execution of db_sync does not depend on $manage_service or $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_host'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Concat::Setup]): concat::setup is deprecated as a public API of the concat module and should no longer be directly included in the manifest.\u001b[0m\n\u001b[1;31mError: heat-keystone-setup-domain returned 1 instead of one of [0]\u001b[0m\n\u001b[1;31mError: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]/returns: change from notrun to 0 failed: heat-keystone-setup-domain returned 1 instead of one of [0]\u001b[0m\n", 
    "deploy_status_code": 6
  }, 
  "creation_time": "2016-04-13T06:11:50Z", 
  "updated_time": "2016-04-13T06:13:32Z", 
  "input_values": {}, 
  "action": "CREATE", 
  "status_reason": "deploy_status_code : Deployment exited with non-zero status code: 6", 
  "id": "d051a6d9-0aef-4888-be53-a668f49fa61f"
}

Expected results:
Successful OSP deployment.

Additional info:
Was able to deploy 3 controllers + 2 compute nodes without issue, by running:
time openstack overcloud deploy --templates ~/templates/my-overcloud -e ~/templates/my-overcloud/environments/network-isolation.yaml -e ~/templates/network-environment.yaml -e ~/templates/userdata.yaml --control-scale 3 --compute-scale 2 --control-flavor control --compute-flavor compute --ntp-server pool.ntp.org --debug