Hide Forgot
Created attachment 1132005 [details] soreport1 Description of problem: Attempt to deploy rhosp 8 beta 7 with vlan and linux bonding failed. This is one controller and one compute node deployment. sosreport from both controller and compute is uploaded to box with bugzilla number. [stack@c5220-01 ~]$ openstack overcloud deploy -e /home/stack/templates/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml --neutron-network-type vlan --neutron-network-vlan-ranges datacentre:50:90 --neutron-disable-tunneling --compute-scale 1 --control-scale 1 --ceph-storage-scale 0 --templates --control-flavor control --compute-flavor compute --ntp-server 0.rhel.pool.ntp.org --timeout 60 --debug 2>&1 | tee haha.log curl -g -i -X GET -H 'X-Auth-Token: {SHA1}a7e8163e7393944ce930c93ae29be13338c91be9' -H 'Content-Type: application/json' -H 'X-Auth-Url: http://192.168.0.1:5000/v2.0' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud-ComputeCephDeployment-qgaxvlkdk54n/1dae7af0-045f-4fda-8e90-262b1e435435/resources HTTP/1.1 200 OK date: Tue, 01 Mar 2016 17:59:56 GMT connection: keep-alive content-type: application/json; charset=UTF-8 content-length: 783 x-openstack-request-id: req-a6f08769-57a6-4492-89af-9ebbeb996fc3 {"resources": [{"parent_resource": "ComputeCephDeployment", "resource_name": "0", "links": [{"href": "http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud-ComputeCephDeployment-qgaxvlkdk54n/1dae7af0-045f-4fda-8e90-262b1e435435/resources/0", "rel": "self"}, {"href": "http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud-ComputeCephDeployment-qgaxvlkdk54n/1dae7af0-045f-4fda-8e90-262b1e435435", "rel": "stack"}], "logical_resource_id": "0", "creation_time": "2016-03-01T17:54:08", "resource_status": "CREATE_COMPLETE", "updated_time": "2016-03-01T17:54:08", "required_by": [], "resource_status_reason": "state changed", "physical_resource_id": "64b5db43-5d9e-4e27-8e8b-83d0467a52f3", "resource_type": "OS::Heat::StructuredDeployment"}]} Heat Stack create failed. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cliff/app.py", line 374, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/site-packages/cliff/command.py", line 54, in run self.take_action(parsed_args) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 991, in take_action self._deploy_tripleo_heat_templates(stack, parsed_args) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 342, in _deploy_tripleo_heat_templates environments, parsed_args.timeout) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 274, in _heat_deploy raise Exception("Heat Stack create failed.") Exception: Heat Stack create failed. clean_up DeployOvercloud: Heat Stack create failed. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 112, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/lib/python2.7/site-packages/cliff/app.py", line 255, in run result = self.run_subcommand(remainder) File "/usr/lib/python2.7/site-packages/cliff/app.py", line 374, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/site-packages/cliff/command.py", line 54, in run self.take_action(parsed_args) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 991, in take_action self._deploy_tripleo_heat_templates(stack, parsed_args) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 342, in _deploy_tripleo_heat_templates environments, parsed_args.timeout) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 274, in _heat_deploy raise Exception("Heat Stack create failed.") Exception: Heat Stack create failed. END return value: 1 Stack CREATE started 2016-03-01 17:58:53 [overcloud-ControllerNodesPostDeployment-djf2fqndaesc-ControllerOvercloudServicesDeployment_Step4-zetcvrnpk4kz]: CREATE_COMPLETE Stack CREATE completed successfully 2016-03-01 17:58:54 [NetworkDeployment]: SIGNAL_COMPLETE Unknown 2016-03-01 17:58:54 [overcloud-ControllerNodesPostDeployment-djf2fqndaesc-ControllerOvercloudServicesDeployment_Step4-zetcvrnpk4kz]: UPDATE_IN_PROGRESS Stack UPDATE started 2016-03-01 17:58:54 [0]: SIGNAL_COMPLETE Unknown 2016-03-01 17:58:54 [0]: SIGNAL_COMPLETE Unknown 2016-03-01 17:58:55 [0]: CREATE_IN_PROGRESS state changed 2016-03-01 17:58:55 [0]: SIGNAL_COMPLETE Unknown 2016-03-01 17:58:55 [0]: SIGNAL_COMPLETE Unknown 2016-03-01 17:59:38 [ControllerNodesPostDeployment]: CREATE_FAILED CREATE aborted 2016-03-01 17:59:38 [ComputeNodesPostDeployment]: CREATE_FAILED CREATE aborted 2016-03-01 17:59:38 [overcloud]: CREATE_FAILED Resource CREATE failed: resources.ComputeAllNodesValidationDeployment: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1 Stack overcloud CREATE_FAILED [stack@c5220-01 ~]$ heat resource-list overcloud +-------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+---------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | +-------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+---------------------+ | AllNodesValidationConfig | f3a2a342-88a3-487c-af62-82e617813545 | OS::TripleO::AllNodes::Validation | CREATE_COMPLETE | 2016-03-01T17:43:54 | | BlockStorageAllNodesDeployment | b00dc4de-017b-4be4-9644-6d17e3f30427 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | BlockStorageAllNodesValidationDeployment | ec71edee-8374-47d7-b9e6-f01142106c90 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | BlockStorageNodesPostDeployment | | OS::TripleO::BlockStoragePostDeployment | INIT_COMPLETE | 2016-03-01T17:43:54 | | CephStorageAllNodesDeployment | 46052174-7deb-4dc0-b1ad-c1ebb7469667 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | CephStorageAllNodesValidationDeployment | 4c45f008-1259-41e4-b317-e26d7e5aa634 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | CephStorageCephDeployment | e79c2aec-91ad-417a-8d99-36ec563b41fc | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | CephStorageNodesPostDeployment | | OS::TripleO::CephStoragePostDeployment | INIT_COMPLETE | 2016-03-01T17:43:54 | | ComputeAllNodesValidationDeployment | 43cf7036-2b47-4680-b243-9ec7091976cf | OS::Heat::StructuredDeployments | CREATE_FAILED | 2016-03-01T17:43:54 | | ComputeNodesPostDeployment | 091876a5-1db6-4edf-bbcb-e36f3262ff97 | OS::TripleO::ComputePostDeployment | CREATE_FAILED | 2016-03-01T17:43:54 | | ControllerAllNodesDeployment | cf1bc3d3-7c3e-40e8-bb22-a69ecde3a4d3 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | ControllerAllNodesValidationDeployment | 2273575d-256d-4dda-974f-bbe6510c8d52 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | ControllerBootstrapNodeConfig | e81aad47-75a6-4d8b-9a54-415bf1f7ace5 | OS::TripleO::BootstrapNode::SoftwareConfig | CREATE_COMPLETE | 2016-03-01T17:43:54 | | ControllerBootstrapNodeDeployment | 57ea99a1-0864-424d-a037-6e56397d85b5 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | ControllerCephDeployment | 466ecc00-0bad-4366-b9af-72f91e7867b8 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | ControllerClusterConfig | 7ac0f2de-79b8-4fe1-ad69-62942901dd3c | OS::Heat::StructuredConfig | CREATE_COMPLETE | 2016-03-01T17:43:54 | | ControllerClusterDeployment | 60f1d9ea-81ba-4162-838e-f194a8e836e5 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | ControllerNodesPostDeployment | 71fb57e6-b759-419f-8e31-5931d25426c3 | OS::TripleO::ControllerPostDeployment | CREATE_FAILED | 2016-03-01T17:43:54 | | ControllerSwiftDeployment | cf25d67f-b1f6-457c-a85c-299302db83d6 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | ObjectStorageAllNodesValidationDeployment | 392b03c9-d7a7-4d01-8886-921ab8da2c6c | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | VipDeployment | e3527b5f-270c-47b3-9cab-656ec588f72a | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:54 | | AllNodesExtraConfig | 68cd843c-cbf1-4cad-b8f4-3b55e72b6c7b | OS::TripleO::AllNodesExtraConfig | CREATE_COMPLETE | 2016-03-01T17:43:55 | | BlockStorage | 865c65cd-90da-4631-8e48-1ee9f71d7072 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2016-03-01T17:43:55 | | CephClusterConfig | c5c9b3e5-39ad-4479-8d3d-aef58953b33e | OS::TripleO::CephClusterConfig::SoftwareConfig | CREATE_COMPLETE | 2016-03-01T17:43:55 | | CephStorage | 625bcc2c-8e43-414e-a3d1-24459baba1b6 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2016-03-01T17:43:55 | | Compute | 39d7c08c-d565-4123-a26a-fe7ae4dbd77e | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2016-03-01T17:43:55 | | ComputeAllNodesDeployment | 50b271d9-b985-4f43-92e4-2a74cf7b3d39 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:55 | | ComputeCephDeployment | 1dae7af0-045f-4fda-8e90-262b1e435435 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:55 | | ControlVirtualIP | 8e2955ed-4aba-40ad-b208-dbc94bd52bef | OS::Neutron::Port | CREATE_COMPLETE | 2016-03-01T17:43:55 | | Controller | c9b8b08b-438d-46d5-906f-3a436131f375 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2016-03-01T17:43:55 | | ControllerIpListMap | 0bc9bffa-0cc1-4338-86f7-743fd38a8704 | OS::TripleO::Network::Ports::NetIpListMap | CREATE_COMPLETE | 2016-03-01T17:43:55 | | EndpointMap | 1fabbe73-dbcf-4b1e-a38c-5584d93b6542 | OS::TripleO::EndpointMap | CREATE_COMPLETE | 2016-03-01T17:43:55 | | HeatAuthEncryptionKey | overcloud-HeatAuthEncryptionKey-c3k5x25wkbuu | OS::Heat::RandomString | CREATE_COMPLETE | 2016-03-01T17:43:55 | | HorizonSecret | overcloud-HorizonSecret-k5v55lqzalma | OS::Heat::RandomString | CREATE_COMPLETE | 2016-03-01T17:43:55 | | InternalApiVirtualIP | 8055918b-c505-43ef-a885-45f923aea629 | OS::TripleO::Network::Ports::InternalApiVipPort | CREATE_COMPLETE | 2016-03-01T17:43:55 | | MysqlClusterUniquePart | overcloud-MysqlClusterUniquePart-xreblerxe5qj | OS::Heat::RandomString | CREATE_COMPLETE | 2016-03-01T17:43:55 | | MysqlRootPassword | overcloud-MysqlRootPassword-rqkabdqvg32n | OS::Heat::RandomString | CREATE_COMPLETE | 2016-03-01T17:43:55 | | Networks | 1c251b74-01b8-4a66-ab03-93f5e8f2ee3d | OS::TripleO::Network | CREATE_COMPLETE | 2016-03-01T17:43:55 | | ObjectStorage | 074f4f30-5058-4308-9733-83992d33d37c | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2016-03-01T17:43:55 | | ObjectStorageAllNodesDeployment | d3f09d9c-c558-4d91-9dca-f1e374da9e3b | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:55 | | ObjectStorageNodesPostDeployment | 2079983e-9321-4988-a228-99d42884f467 | OS::TripleO::ObjectStoragePostDeployment | CREATE_COMPLETE | 2016-03-01T17:43:55 | | ObjectStorageSwiftDeployment | 398fa5c0-5bcf-4a2a-9e1b-2d816d0ca986 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-03-01T17:43:55 | | PcsdPassword | overcloud-PcsdPassword-dij4kuhw5ss2 | OS::Heat::RandomString | CREATE_COMPLETE | 2016-03-01T17:43:55 | | PublicVirtualIP | 61bcb73c-d4af-4194-a55b-04a008bac5d7 | OS::TripleO::Network::Ports::ExternalVipPort | CREATE_COMPLETE | 2016-03-01T17:43:55 | | RabbitCookie | overcloud-RabbitCookie-euke3l23hnpx | OS::Heat::RandomString | CREATE_COMPLETE | 2016-03-01T17:43:55 | | RedisVirtualIP | 8db47c46-6668-47ca-905a-883f96e4706e | OS::TripleO::Network::Ports::RedisVipPort | CREATE_COMPLETE | 2016-03-01T17:43:55 | | StorageMgmtVirtualIP | ceddd403-0cfc-4bb1-a0dd-3b78611c2b34 | OS::TripleO::Network::Ports::StorageMgmtVipPort | CREATE_COMPLETE | 2016-03-01T17:43:55 | | StorageVirtualIP | 7028f098-7c86-4b69-a889-e30acd093ddd | OS::TripleO::Network::Ports::StorageVipPort | CREATE_COMPLETE | 2016-03-01T17:43:55 | | SwiftDevicesAndProxyConfig | ffa57c89-469a-4d37-80cc-ac3bf6c69151 | OS::TripleO::SwiftDevicesAndProxy::SoftwareConfig | CREATE_COMPLETE | 2016-03-01T17:43:55 | | VipConfig | 13d93df6-2659-4585-bae1-7f9dbb38cdf6 | OS::TripleO::VipConfig | CREATE_COMPLETE | 2016-03-01T17:43:55 | | VipMap | 33c6a918-1a8f-452b-88b1-a88c14bb0d16 | OS::TripleO::Network::Ports::NetVipMap | CREATE_COMPLETE | 2016-03-01T17:43:55 | | allNodesConfig | 9c5fc304-56e0-4fa1-8b46-907e97888ff9 | OS::TripleO::AllNodes::SoftwareConfig | CREATE_COMPLETE | 2016-03-01T17:43:55 | +-------------------------------------------+-----------------------------------------------+---------------------------------------------------+-----------------+---------------------+ [stack@c5220-01 ~]$ heat resource-show overcloud ControllerNodesPostDeployment +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attributes | {} | | creation_time | 2016-03-01T17:43:54 | | description | | | links | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud/4fe63c70-796e-4ac4-b3c4-a7d618dee0ed/resources/ControllerNodesPostDeployment (self) | | | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud/4fe63c70-796e-4ac4-b3c4-a7d618dee0ed (stack) | | | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud-ControllerNodesPostDeployment-djf2fqndaesc/71fb57e6-b759-419f-8e31-5931d25426c3 (nested) | | logical_resource_id | ControllerNodesPostDeployment | | physical_resource_id | 71fb57e6-b759-419f-8e31-5931d25426c3 | | required_by | BlockStorageNodesPostDeployment | | | CephStorageNodesPostDeployment | | resource_name | ControllerNodesPostDeployment | | resource_status | CREATE_FAILED | | resource_status_reason | CREATE aborted | | resource_type | OS::TripleO::ControllerPostDeployment | | updated_time | 2016-03-01T17:43:54 | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ [stack@c5220-01 ~]$ heat resource-show overcloud ComputeNodesPostDeployment +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attributes | {} | | creation_time | 2016-03-01T17:43:54 | | description | | | links | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud/4fe63c70-796e-4ac4-b3c4-a7d618dee0ed/resources/ComputeNodesPostDeployment (self) | | | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud/4fe63c70-796e-4ac4-b3c4-a7d618dee0ed (stack) | | | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud-ComputeNodesPostDeployment-oc5acsstn36q/091876a5-1db6-4edf-bbcb-e36f3262ff97 (nested) | | logical_resource_id | ComputeNodesPostDeployment | | physical_resource_id | 091876a5-1db6-4edf-bbcb-e36f3262ff97 | | required_by | | | resource_name | ComputeNodesPostDeployment | | resource_status | CREATE_FAILED | | resource_status_reason | CREATE aborted | | resource_type | OS::TripleO::ComputePostDeployment | | updated_time | 2016-03-01T17:43:54 | +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ [stack@c5220-01 ~]$ heat resource-show overcloud ComputeAllNodesValidationDeployment +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attributes | { | | | "deploy_stderrs": null, | | | "deploy_stdouts": null, | | | "deploy_status_codes": null | | | } | | creation_time | 2016-03-01T17:43:54 | | description | | | links | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud/4fe63c70-796e-4ac4-b3c4-a7d618dee0ed/resources/ComputeAllNodesValidationDeployment (self) | | | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud/4fe63c70-796e-4ac4-b3c4-a7d618dee0ed (stack) | | | http://192.168.0.1:8004/v1/f4f5d6685a0c4a398775946b100a1dc9/stacks/overcloud-ComputeAllNodesValidationDeployment-rv3ytglpxia5/43cf7036-2b47-4680-b243-9ec7091976cf (nested) | | logical_resource_id | ComputeAllNodesValidationDeployment | | physical_resource_id | 43cf7036-2b47-4680-b243-9ec7091976cf | | required_by | | | resource_name | ComputeAllNodesValidationDeployment | | resource_status | CREATE_FAILED | | resource_status_reason | resources.ComputeAllNodesValidationDeployment: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1 | | resource_type | OS::Heat::StructuredDeployments | | updated_time | 2016-03-01T17:43:54 | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ [stack@c5220-01 ~]$ Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Created attachment 1132006 [details] sosreport-compute
sosreport is attached
Added journalctl output from controller: [root@overcloud-controller-0 tmp]# journalctl -fn 100000 | grep -ni error 1513:Mar 01 12:51:39 localhost.localdomain cloud-init[877]: 2016-03-01 12:51:39,266 - util.py[WARNING]: Route info failed: Unexpected error while running command. 1519:Mar 01 12:51:39 localhost.localdomain cloud-init[877]: [CLOUDINIT] util.py[WARNING]: Route info failed: Unexpected error while running command. 1525:Mar 01 12:51:39 localhost.localdomain cloud-init[877]: [CLOUDINIT] util.py[DEBUG]: Route info failed: Unexpected error while running command. 1538: ProcessExecutionError: Unexpected error while running command. 2485:Mar 01 12:52:16 overcloud-controller-0.localdomain dracut[3561]: -rwxr-xr-x 1 root root 145 Jan 15 03:06 usr/bin/kdump-error-handler.sh 2731:Mar 01 12:52:17 overcloud-controller-0.localdomain dracut[3561]: -rw-r--r-- 1 root root 993 Jan 15 03:06 usr/lib/systemd/system/kdump-error-handler.service 2884:Mar 01 12:52:17 overcloud-controller-0.localdomain dracut[3561]: lrwxrwxrwx 1 root root 22 Mar 1 12:52 usr/lib64/libgpg-error.so -> libgpg-error.so.0.10.0 2885:Mar 01 12:52:17 overcloud-controller-0.localdomain dracut[3561]: lrwxrwxrwx 1 root root 22 Mar 1 12:52 usr/lib64/libgpg-error.so.0 -> libgpg-error.so.0.10.0 2886:Mar 01 12:52:17 overcloud-controller-0.localdomain dracut[3561]: -rwxr-xr-x 1 root root 19384 Mar 1 12:52 usr/lib64/libgpg-error.so.0.10.0 5079:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: Resource start-up disabled since no STONITH resources have been defined 5080:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: Either configure some or disable STONITH with the stonith-enabled option 5081:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: NOTE: Clusters with shared data need STONITH to ensure data integrity 5084:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: notice: Configuration ERRORs found during PE processing. Please run "crm_verify -L" to identify issues. 5089:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: Resource start-up disabled since no STONITH resources have been defined 5090:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: Either configure some or disable STONITH with the stonith-enabled option 5091:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: error: NOTE: Clusters with shared data need STONITH to ensure data integrity 5094:Mar 01 12:56:58 overcloud-controller-0.localdomain pengine[30046]: notice: Configuration ERRORs found during PE processing. Please run "crm_verify -L" to identify issues. 5108:Mar 01 12:57:02 overcloud-controller-0.localdomain os-collect-config[7424]: e]/returns: executed successfully\u001b[0m\n\u001b[mNotice: Pacemaker has reported quorum achieved\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Corosync/Notify[pacemaker settled]/message: defined 'message' as 'Pacemaker has reported quorum achieved'\u001b[0m\n\u001b[mNotice: /Stage[main]/Pacemaker::Stonith/Exec[Disable STONITH]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Tripleo::Loadbalancer/Haproxy::Listen[redis]/Concat::Fragment[haproxy-redis_listen_block]/File[/var/lib/puppet/concat/_etc_haproxy_haproxy.cfg/fragments/20-redis-00_haproxy-redis_listen_block]/ensure: defined content as '{md5}f11dc73be476a8ce3267d965fedc63f1'\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]: Triggered 'refresh' from 41 events\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/File[/etc/haproxy/haproxy.cfg]/content: content changed '{md5}1f337186b0e1ba5ee82760cb437fb810' to '{md5}40706bafaaa45c5cdc110ab5b717f865'\u001b[0m\n\u001b[mNotice: /File[/etc/haproxy/haproxy.cfg]/seluser: seluser changed 'unconfined_u' to 'system_u'\u001b[0m\n\u001b[mNotice: Finished catalog run in 43.66 seconds\u001b[0m\n", "deploy_stderr": "\u001b[1;31mWarning: Scope(Class[Mongodb::Server]): Replset specified, but no replset_members or replset_config provided.\u001b[0m\n\u001b[1;31mWarning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' 5109:Mar 01 12:57:02 overcloud-controller-0.localdomain os-collect-config[7424]: returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n", "deploy_status_code": 0} 5213:Mar 01 12:57:02 overcloud-controller-0.localdomain os-collect-config[7424]: Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) 5214:Mar 01 12:57:02 overcloud-controller-0.localdomain os-collect-config[7424]: Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) 5817:Mar 01 12:57:42 overcloud-controller-0.localdomain galera(galera)[3738]: ERROR: MySQL is not running 5861:Mar 01 12:57:48 overcloud-controller-0.localdomain galera(galera)[4905]: ERROR: MySQL is not running 6115:Mar 01 12:58:14 overcloud-controller-0.localdomain os-collect-config[7424]: int::Base[storage_mgmt_vip-then-haproxy]/Exec[Creating order constraint storage_mgmt_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Colocation[control_vip-with-haproxy]/Pcmk_constraint[colo-ip-192.168.0.12-haproxy-clone]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[public_vip-then-haproxy]/Exec[Creating order constraint public_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[redis_vip-then-haproxy]/Exec[Creating order constraint redis_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[storage_vip-then-haproxy]/Exec[Creating order constraint storage_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: Finished catalog run in 35.84 seconds\u001b[0m\n", "deploy_stderr": "\u001b[1;31mWarning: Scope(Class[Mongodb::Server]): Replset specified, but no replset_members or replset_config provided.\u001b[0m\n\u001b[1;31mWarning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n", "deploy_status_code": 0} 6219:Mar 01 12:58:14 overcloud-controller-0.localdomain os-collect-config[7424]: Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) 6220:Mar 01 12:58:14 overcloud-controller-0.localdomain os-collect-config[7424]: Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) 9454:Mar 01 13:02:23 overcloud-controller-0.localdomain os-collect-config[7424]: ]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Db/Ceilometer_config[database/retry_interval]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Agent::Notification/Ceilometer_config[notification/ack_on_event_error]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/rpc_backend]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Alarm::Evaluator/Ceilometer_config[alarm/evaluation_service]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[oslo_messaging_rabbit/rabbit_hosts]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Agent::Central/Ceilometer_config[coordination/backend_url]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/http_timeout]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[DEFAULT/notification_topics]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Db/Ceilometer_config[database/max_retries]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Agent::Auth/Ceilometer_config[service_credentials/os_auth_url]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Logging/Ceilometer_config[DEFAULT/use_syslog]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Collector/Ceilometer_config[collector/udp_address]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Expirer/Ceilometer_config[database/time_to_live]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Alarm::Evaluator/Ceilometer_config[alarm/record_history]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Logging/Ceilometer_config[DEFAULT/use_stderr]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer::Alarm::Evaluator/Ceilometer_config[alarm/evaluation_interval]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Ceilometer/Ceilometer_config[oslo_messaging_rabbit/rabbit_ha_queues]/ensure: created\u001 9489:Mar 01 13:02:23 overcloud-controller-0.localdomain os-collect-config[7424]: /swift/object-server/]/seluser: seluser changed 'unconfined_u' to 'system_u'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat[/etc/swift/proxy-server.conf]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat[/etc/swift/proxy-server.conf]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments.concat]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat[/etc/swift/proxy-server.conf]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments.concat.out]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat[/etc/swift/proxy-server.conf]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy/Concat::Fragment[swift_proxy]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/00_swift_proxy]/ensure: defined content as '{md5}d40310054460199f880f40207d9a92f4'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy::Catch_errors/Concat::Fragment[swift_catch_errors]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/24_swift_catch_errors]/ensure: defined content as '{md5}e07f0e5b125db7d6c8b4724c1648bcd5'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy::Healthcheck/Concat::Fragment[swift_healthcheck]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/25_swift_healthcheck]/ensure: defined content as '{md5}8c92056c41082619d179f88ea15c5fc6'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy::Cache/Concat::Fragment[swift_cache]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/23_swift_cache]/ensure: defined content as '{md5}20006a9d2864c3a4f5d6b1df19f09fce'\u001b[0m\n\u001b[mNotice: /Stage[main]/Swift::Proxy::Tempurl/Concat::Fragment[swift-proxy-tempurl]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/29_swift-proxy-tempurl]/ensure: defined content as '{md5}2fe004eae9f03fc684f9ed90044bd9c5'\u001b[0m\n\u001b[mNotice: 9630:Mar 01 13:02:23 overcloud-controller-0.localdomain os-collect-config[7424]: Notice: /Stage[main]/Ceilometer::Agent::Notification/Ceilometer_config[notification/ack_on_event_error]/ensure: created 10179:Mar 01 13:02:23 overcloud-controller-0.localdomain os-collect-config[7424]: Notice: /Stage[main]/Swift::Proxy::Catch_errors/Concat::Fragment[swift_catch_errors]/File[/var/lib/puppet/concat/_etc_swift_proxy-server.conf/fragments/24_swift_catch_errors]/ensure: defined content as '{md5}e07f0e5b125db7d6c8b4724c1648bcd5' 11233:Mar 01 13:03:31 overcloud-controller-0.localdomain crmd[30047]: warning: Action 121 (openstack-heat-engine:0_start_0) on overcloud-controller-0 failed (target: 0 vs. rc: 7): Error 11235:Mar 01 13:03:31 overcloud-controller-0.localdomain crmd[30047]: warning: Action 121 (openstack-heat-engine:0_start_0) on overcloud-controller-0 failed (target: 0 vs. rc: 7): Error 11807:Mar 01 13:03:45 overcloud-controller-0.localdomain crmd[30047]: warning: Action 31 (neutron-server:0_monitor_0) on overcloud-controller-0 failed (target: 7 vs. rc: 0): Error 11809:Mar 01 13:03:45 overcloud-controller-0.localdomain crmd[30047]: warning: Action 31 (neutron-server:0_monitor_0) on overcloud-controller-0 failed (target: 7 vs. rc: 0): Error 11810:Mar 01 13:03:45 overcloud-controller-0.localdomain crmd[30047]: warning: Action 32 (httpd:0_monitor_0) on overcloud-controller-0 failed (target: 7 vs. rc: 0): Error 11811:Mar 01 13:03:45 overcloud-controller-0.localdomain crmd[30047]: warning: Action 32 (httpd:0_monitor_0) on overcloud-controller-0 failed (target: 7 vs. rc: 0): Error 12032:Mar 01 13:03:48 overcloud-controller-0.localdomain object-server[8046]: Object audit (ZBF) "forever" mode completed: 0.00s. Total quarantined: 0, Total errors: 0, Total files/sec: 0.00, Total bytes/sec: 0.00, Auditing time: 0.00, Rate: 0.00 12034:Mar 01 13:03:48 overcloud-controller-0.localdomain object-server[8047]: Object audit (ALL) "forever" mode completed: 0.00s. Total quarantined: 0, Total errors: 0, Total files/sec: 0.00, Total bytes/sec: 0.00, Auditing time: 0.00, Rate: 0.00 12160:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8134]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server". 12171:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8169]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server". 12175:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8168]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server". 12185:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8170]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server". 12189:Mar 01 13:03:50 overcloud-controller-0.localdomain proxy-server[8171]: Pipeline was modified. New pipeline is "catch_errors gatekeeper healthcheck cache ratelimit tempurl formpost authtoken keystone staticweb proxy-logging dlo versioned_writes proxy-server". 13110:Mar 01 13:03:58 overcloud-controller-0.localdomain ceilometer-alarm-evaluator[9169]: ImportError: No module named redis 13111:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl [req-57b7f775-9ae2-42d9-acc8-480165ed44f7 - - - - -] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=type', 'list', 'Interface', 'int-br-ex']. 13112:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last): 13113:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl 13114:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl log_fail_as_error=False).rstrip() 13115:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute 13116:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl raise RuntimeError(m) 13117:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 13118:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=type', 'list', 'Interface', 'int-br-ex'] 13119:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 1 13120:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl 13121:Mar 01 13:03:58 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:58.548 9018 ERROR neutron.agent.ovsdb.impl_vsctl 13191:Mar 01 13:03:59 overcloud-controller-0.localdomain ceilometer-polling[9170]: ImportError: No module named redis 13194:Mar 01 13:03:59 overcloud-controller-0.localdomain neutron-openvswitch-agent[9018]: 2016-03-01 13:03:59.652 9018 ERROR neutron.agent.linux.ovsdb_monitor [req-57b7f775-9ae2-42d9-acc8-480165ed44f7 - - - - -] Interface monitor is not active 13209:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils [-] Invalid format of swift store configcfg 13210:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils Traceback (most recent call last): 13211:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/utils.py", line 126, in _load_config 13212:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils reference[param] = CONFIG.get(ref, param) 13213:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib64/python2.7/ConfigParser.py", line 618, in get 13214:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils raise NoOptionError(option, section) 13215:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils NoOptionError: No option 'auth_address' in section: 'database' 13216:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.258 9020 ERROR glance_store._drivers.swift.utils 13217:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils [-] Invalid format of swift store configcfg 13218:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils Traceback (most recent call last): 13219:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/utils.py", line 126, in _load_config 13220:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils reference[param] = CONFIG.get(ref, param) 13221:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib64/python2.7/ConfigParser.py", line 618, in get 13222:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils raise NoOptionError(option, section) 13223:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils NoOptionError: No option 'auth_address' in section: 'glance_store' 13224:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.268 9020 ERROR glance_store._drivers.swift.utils 13225:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils [-] Invalid format of swift store configcfg 13226:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils Traceback (most recent call last): 13227:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/utils.py", line 126, in _load_config 13228:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils reference[param] = CONFIG.get(ref, param) 13229:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib64/python2.7/ConfigParser.py", line 618, in get 13230:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils raise NoOptionError(option, section) 13231:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils NoOptionError: No option 'auth_address' in section: 'image_format' 13232:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.269 9020 ERROR glance_store._drivers.swift.utils 13233:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils [-] Invalid format of swift store configcfg 13234:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils Traceback (most recent call last): 13235:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/utils.py", line 126, in _load_config 13236:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils reference[param] = CONFIG.get(ref, param) 13237:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils File "/usr/lib64/python2.7/ConfigParser.py", line 618, in get 13238:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils raise NoOptionError(option, section) 13239:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils NoOptionError: No option 'auth_address' in section: 'keystone_authtoken' 13240:Mar 01 13:04:00 overcloud-controller-0.localdomain glance-api[9020]: 2016-03-01 13:04:00.270 9020 ERROR glance_store._drivers.swift.utils
The failure was in the ComputeAllNodesValidationDeployment script on Compute node 0, so to diagnose the problem we'd need to see the output of that. You can get it from the Heat API with the command: heat deployment-show `heat resource-show 43cf7036-2b47-4680-b243-9ec7091976cf 0 | grep physical_resource_id | cut -d'|' -f3`
hi this is the output [stack@c5220-01 sosreport2]$ heat deployment-show `heat resource-show 43cf7036-2b47-4680-b243-9ec7091976cf 0 | grep physical_resource_id | cut -d'|' -f3` { "status": "FAILED", "server_id": "2367a924-bff0-4e8a-befc-1c3e01a85dd5", "config_id": "49025b5d-036e-49b4-8516-02cd00cfae3b", "output_values": { "deploy_stdout": "Trying to ping 172.16.0.4 for local network 172.16.0.0/24...FAILURE\n", "deploy_stderr": "172.16.0.4 is not pingable. Local Network: 172.16.0.0/24\n", "deploy_status_code": 1 }, "creation_time": "2016-03-01T17:54:54", "updated_time": "2016-03-01T17:55:35", "input_values": {}, "action": "CREATE", "status_reason": "deploy_status_code : Deployment exited with non-zero status code: 1", "id": "016747e7-896b-448b-a718-352dcafbac48" }
Ok I miss adding tenantNeworkVlanID to controller.yaml file. Will add it and redeploy
Found one problem. Looks like beta7 is not using the stable/liberty tripleo-heat-templates For example the upstream stable/liberty has following code: if 'cisco_n1kv' in hiera('neutron::plugins::ml2::mechanism_drivers') { at https://github.com/openstack/tripleo-heat-templates/blob/stable/liberty/puppet/manifests/overcloud_compute.pp#L123 However, the actual beta7 code is like following: if 'cisco_n1kv' in hiera('neutron_mechanism_drivers') {
(In reply to bigswitch from comment #9) > Found one problem. Looks like beta7 is not using the stable/liberty > tripleo-heat-templates > > For example the upstream stable/liberty has following code: > if 'cisco_n1kv' in hiera('neutron::plugins::ml2::mechanism_drivers') { > at > https://github.com/openstack/tripleo-heat-templates/blob/stable/liberty/ > puppet/manifests/overcloud_compute.pp#L123 > > However, the actual beta7 code is like following: > if 'cisco_n1kv' in hiera('neutron_mechanism_drivers') { This is both true and not true. The build included in beta 7 was openstack-tripleo-heat-templates-0.8.7-12.el7ost. Since the 0.8.7 release upstream, upstream has moved along. The change that you're seeing was done in this review: https://review.openstack.org/#/c/269898/ The plan is that beta 8 will be on the 0.8.9 release which just happened this week and will contain that patch. (In reply to bigswitch from comment #8) > Ok I miss adding tenantNeworkVlanID to controller.yaml file. Will add it and > redeploy Did this help and resolve your problem?
Thanks a lot Mike. Please close this bug :)