Description of problem:

It is not possible to upscalwe the cloud correctly, we are facing the folloing errors:

# openstack overcloud deploy --templates ~/templates/my-overcloud --compute-scale 4 --compute-flavor compute --ntp-server 10.256.256.256 --neutron-network-type vxlan --neutron-tunnel-types vxlan -e ~/templates/global-environment-dsn.yaml -e ~/templates/my-overcloud/environments/network-isolation.yaml -e ~/templates/my-overcloud/environments/storage-environment.yaml -e /home/stack/templates/my-overcloud/enable-tls.yaml -e /home/stack/ssl/enable-tls-key.yaml -e /home/stack/templates/my-overcloud/inject-trust-anchor.yaml
Deploying templates in the directory /home/stack/templates/my-overcloud
Stack failed with status: resources.ControllerNodesPostDeployment: resources.ControllerOvercloudServicesDeployment_Step6: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6
ERROR: openstack Heat Stack update failed.


]$ heat resource-list -n 5 overcloud | grep FAILED
| ControllerNodesPostDeployment                 | 9bc778fc-03f8-4071-9d94-4147590b94c4          | OS::TripleO::ControllerPostDeployment             | UPDATE_FAILED   | 2016-03-17T08:59:08Z |                                               |
| ControllerOvercloudServicesDeployment_Step6   | f6cb421c-de01-4b80-9055-4c820a50245c          | OS::Heat::StructuredDeployments                   | UPDATE_FAILED   | 2016-03-17T09:14:07Z | ControllerNodesPostDeployment                 |
| 0                                             | 11b06f99-2075-4cbe-be94-bdc1c4af83a0          | OS::Heat::StructuredDeployment                    | UPDATE_FAILED   | 2016-03-17T09:14:09Z | ControllerOvercloudServicesDeployment_Step6   |


$ heat deployment-show 11b06f99-2075-4cbe-be94-bdc1c4af83a0
{
  "status": "FAILED", 
  "server_id": "65a55e8e-31be-4023-b5bf-6bfc53a9f350", 
  "config_id": "d825e229-0068-4036-a1e2-210dadab4970", 
  "output_values": {
    "deploy_stdout": "\u001b[mNotice: Compiled catalog for ost-ah1-controller0.localdomain in environment production in 42.17 seconds\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Exec[galera-ready]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[ost-ah1-controller0]/File[/tmp/ceph-mon-keyring-ost-ah1-controller0]/ensure: defined content as '{md5}6e76a887817a550ce891521930b1010f'\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[ost-ah1-controller0]/Exec[ceph-mon-mkfs-ost-ah1-controller0]/returns: ++ ceph-mon --id ost-ah1-controller0 --show-config-value mon_data\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[ost-ah1-controller0]/Exec[ceph-mon-mkfs-ost-ah1-controller0]/returns: + mon_data=/var/lib/ceph/mon/ceph-ost-ah1-controller0\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[ost-ah1-controller0]/Exec[ceph-mon-mkfs-ost-ah1-controller0]/returns: + '[' '!' -d /var/lib/ceph/mon/ceph-ost-ah1-controller0 ']'\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[ost-ah1-controller0]/Exec[ceph-mon-mkfs-ost-ah1-controller0]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[ost-ah1-controller0]/Exec[ceph-mon-ceph.client.admin.keyring-ost-ah1-controller0]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[ost-ah1-controller0]/Service[ceph-mon-ost-ah1-controller0]/ensure: ensure changed 'stopped' to 'running'\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceph::Profile::Mon/Ceph::Mon[ost-ah1-controller0]/Exec[rm-keyring-ost-ah1-controller0]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Keystone::Roles::Admin/Keystone_user[admin]: Dependency Keystone_tenant[admin] has failures: true\u001b[0m\n\u001b[mNotice: /Stage[main]/Keystone::Roles::Admin/Keystone_user_role[admin@admin]: Dependency Keystone_tenant[admin] has failures: true\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]: Dependency Keystone_tenant[services] has failures: true\u001b[0m\n\u001b[mNotice: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]: Dependency Keystone_tenant[admin] has failures: true\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Exec[enable-not-start-tripleo_cluster]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Exec[Set password for hacluster user on tripleo_cluster]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Exec[auth-successful-across-all-nodes]/returns: executed successfully\u001b[0m\n\u001b[mNotice: Pacemaker has reported quorum achieved\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Notify[pacemaker settled]/message: defined 'message' as 'Pacemaker has reported quorum achieved'\u001b[0m\n\u001b[mNotice: Finished catalog run in 167.19 seconds\u001b[0m\n", 
    "deploy_stderr": "Device \"br_bond0\" does not exist.\nDevice \"br_ex\" does not exist.\nDevice \"br_int\" does not exist.\nDevice \"br_tun\" does not exist.\nDevice \"ovs_system\" does not exist.\n\u001b[1;31mWarning: Scope(Class[Keystone]): Execution of db_sync does not depend on $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Glance::Registry]): Execution of db_sync does not depend on $manage_service or $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_host'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Concat::Setup]): concat::setup is deprecated as a public API of the concat module and should no longer be directly included in the manifest.\u001b[0m\n\u001b[1;31mError: Could not prefetch keystone_tenant provider 'openstack': Execution of '/usr/bin/openstack project list --quiet --format csv --long' returned 1: ERROR: openstack 'ascii' codec can't encode character u'\\xf3' in position 89: ordinal not in range(128)\n\"ID\",\"Name\",\"Description\",\"Enabled\"\n\"0161c65a362b446abbf896fac457f4d6\",\"DSN-ENG1\",\"Tenant reserved for migration from OST-PRE\",True\n\"038d38558df54e30ba72f8cf625515cd\",\"EndPointsTestJSON-446276246\",\"EndPointsTestJSON-446276246-desc\",True\n\"042cf907720145fea8ea1324394fd576\",\"demo\",\"Default project\",True\n\"05973815d133410192348bf3f4477c9e\",\"Test\",\"Test project for Admins\",True\n\"07ec661cbb6447759a1389bc69786c91\",\"AccountNegativeTest-825982852\",\"AccountNegativeTest-825982852-desc\",True\n\"08313c47cc5340ba8f03fee1e646e46c\",\"ContainerTest-619159989\",\"ContainerTest-619159989-desc\",True\n\"0b55ec4f0bf74e5e8933f55b284c8518\",\"ObjectTestACLs-1275316143\",\"ObjectTestACLs-1275316143-desc\",True\n\"11e5824e6e0749d2aa52412b87b8dae9\",\"ProjectsTestJSON-926624388\",\"ProjectsTestJSON-926624388-desc\",True\u001b[0m\n\u001b[1;31mError: Execution of '/usr/bin/openstack project create --format shell admin --enable --description admin tenant' returned 1: ERROR: openstack Conflict occurred attempting to store project - Duplicate Entry (HTTP 409) (Request-ID: req-efaca269-2a83-4803-b2ea-006f91dce361)\u001b[0m\n\u001b[1;31mError: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[admin]/ensure: change from absent to present failed: Execution of '/usr/bin/openstack project create --format shell admin --enable --description admin tenant' returned 1: ERROR: openstack Conflict occurred attempting to store project - Duplicate Entry (HTTP 409) (Request-ID: req-efaca269-2a83-4803-b2ea-006f91dce361)\u001b[0m\n\u001b[1;31mError: Execution of '/usr/bin/openstack project create --format shell services --enable --description Tenant for the openstack services' returned 1: ERROR: openstack Conflict occurred attempting to store project - Duplicate Entry (HTTP 409) (Request-ID: req-1f87a68a-d9ef-4425-be5b-0334bf7475c0)\u001b[0m\n\u001b[1;31mError: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[services]/ensure: change from absent to present failed: Execution of '/usr/bin/openstack project create --format shell services --enable --description Tenant for the openstack services' returned 1: ERROR: openstack Conflict occurred attempting to store project - Duplicate Entry (HTTP 409) (Request-ID: req-1f87a68a-d9ef-4425-be5b-0334bf7475c0)\u001b[0m\n\u001b[1;31mWarning: /Stage[main]/Keystone::Roles::Admin/Keystone_user[admin]: Skipping because of failed dependencies\u001b[0m\n\u001b[1;31mWarning: /Stage[main]/Keystone::Roles::Admin/Keystone_user_role[admin@admin]: Skipping because of failed dependencies\u001b[0m\n\u001b[1;31mWarning: /Stage[main]/Heat::Keystone::Domain/Exec[heat_domain_create]: Skipping because of failed dependencies\u001b[0m\n", 
    "deploy_status_code": 6
  }, 
  "creation_time": "2016-03-08T17:32:54Z", 
  "updated_time": "2016-03-17T09:19:55Z", 
  "input_values": {}, 
  "action": "UPDATE", 
  "status_reason": "deploy_status_code : Deployment exited with non-zero status code: 6", 
  "id": "11b06f99-2075-4cbe-be94-bdc1c4af83a0"
}

please note:

Error: Execution of '/usr/bin/openstack project create --format shell admin --enable --description admin tenant' returned 1: ERROR: openstack Conflict occurred attempting to store project - Duplicate Entry (HTTP 409) (Request-ID: req-efaca269-2a83-4803-b2ea-006f91dce361)\u001b[0m\n\u001b[1;31mError: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[admin]/ensure: change from absent to present failed: Execution of '/usr/bin/openstack project create --format shell admin --enable --description admin tenant'


Other things in the logs:

# /usr/bin/openstack project list --quiet --format csv --long' returned 1: ERROR: openstack 'ascii' codec can't encode character u'\\xf3' in position 89: ordinal not in range(12

Version-Release number of selected component (if applicable):


How reproducible:
Always


Steps to Reproduce:
1. deploy an initial cloud
2. upscale the environment 
3.

Actual results:
above errors

Expected results:
env upscaled successfuly

Additional info: