Bug 1571228 - Overcloud deployment failed with standalone swift
Summary: Overcloud deployment failed with standalone swift
Keywords:
Status: CLOSED EOL
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 12.0 (Pike)
Hardware: All
OS: Linux
high
high
Target Milestone: zstream
: 12.0 (Pike)
Assignee: Christian Schwede (cschwede)
QA Contact: Mike Abrams
URL:
Whiteboard:
Depends On:
Blocks: 1544802
TreeView+ depends on / blocked
 
Reported: 2018-04-24 11:09 UTC by Oksana Voshchana
Modified: 2019-01-11 16:26 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-11 16:26:38 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
OpenStack gerrit 563895 None None None 2018-04-24 11:39:02 UTC
Launchpad 1736109 None None None 2018-04-24 11:37:34 UTC

Description Oksana Voshchana 2018-04-24 11:09:47 UTC
Description of problem:
We are trying to deploy OSP12(13) with standalone swift:
parameter_defaults:
   NetworkerCount: 1
   OvercloudNetworkerFlavor: networker
   ObjectStorageCount: 1
   OvercloudSwiftStorageFlavor: swift
   ControllerCount: 1
   OvercloudControlFlavor: controller
   ComputeCount: 1
   OvercloudComputeFlavor: compute

Deployment stops in the middle of overcloud.AllNodesDeploySteps.ObjectStorageHostPrepDeployment failure

resource_type: OS::Heat::SoftwareDeployment
  physical_resource_id: c4054afa-280a-4e34-811a-43e364b1d0e0
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: |
    ...
    ok: [localhost] => (item=/srv/node)
    ok: [localhost] => (item=/var/log/swift)
    
    TASK [Create swift logging symlink] ********************************************
    fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Error while linking: [Errno 2] No such file or directory", "path": "/var/log/containers/swift", "state": "absent"}
    	to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/32d2a2c1-d48f-4873-9878-a69a4b9b21cf_playbook.retry
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=3    changed=0    unreachable=0    failed=1   
    
    (truncated, view all with --long)
  deploy_stderr: |


Version-Release number of selected component (if applicable): OSP12


How reproducible:


Steps to Reproduce:
1. Deploy undercloud with VM's
2. Deploy overcloud with standalone swift


Actual results:
Heat deployments to the overcloud do not work.

Expected results:
Successful deployment

Additional info:

OS::Heat::SoftwareDeploymentGroup | UPDATE_FAILED | 2018-04-23T16:09:13Z | overcloud-AllNodesDeploySteps-lw4gxcyx5xok |

0 5c3611a7-ae07-4064-96aa-48500ca7d27b OS::Heat::SoftwareDeployment CREATE_FAILED 2018-04-23T16:09:14Z overcloud-AllNodesDeploySteps-lw4gxcyx5xok-ObjectStorageHostPrepDeployment-ailrloor73od 
TASK [Create swift logging symlink] ********************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Error while linking: [Errno 2] No such file or directory", "path": "/var/log/containers/swift", "state": "absent"}
to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/2fb28609-e22d-4d5f-bf7c-f5907506a724_playbook.retry
In heat log:
2018-04-23 10:38:13.786 1579 INFO heat.engine.resource [req-b8a11bdd-35b8-433a-946d-66338be56a1b - admin - default default] CREATE: SoftwareDeployment "0" [e5e40fac-2009-48b9-8f98-8a2cef7d943c] Stack "overcloud-
AllNodesDeploySteps-lw4gxcyx5xok-ObjectStorageHostPrepDeployment-ailrloor73od" [5bd04457-84ea-479d-9d55-21cf6143273c]
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource Traceback (most recent call last):
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 831, in _action_recorder
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource yield
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 939, in _do_action
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource yield self.action_handler_task(action, args=handler_args)
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 351, in wrapper
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource step = next(subtask)
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 890, in action_handler_task
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource done = check(handler_data)
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/software_deployment.py", line 433, in check_create_complete
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource return self._check_complete()
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/software_deployment.py", line 299, in _check_complete
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource raise exception.Error(message)
2018-04-23 10:38:13.786 1579 ERROR heat.engine.resource Error: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2



2018-04-24 11:07:04.296 1578 INFO heat.engine.service [req-01a185af-dc5d-4d5d-85f1-f481fd0f51fa - admin - default default] validate_template
2018-04-24 11:07:04.786 1578 DEBUG heat.engine.parameter_groups [req-01a185af-dc5d-4d5d-85f1-f481fd0f51fa - admin - default default] Validating Parameter Groups: ControllerParameters, OS::project_id, ControllerCount, NovaComputeSchedulerHints, NetworkerExtraConfig, ComputeCount, PublicVirtualFixedIPs, ObjectStorageParameters, StorageMgmtVirtualFixedIPs, InternalApiVirtualFixedIPs, NetworkerRemovalPolicies, CloudNameInternal, ControllerExtraConfig, ControllerHostnameFormat, CloudDomain, NovaComputeExtraConfig, ControllerRemovalPolicies, ServerMetadata, DeploymentServerBlacklist, ControllerSchedulerHints, NeutronControlPlaneID, NetworkerHostnameFormat, NetworkerSchedulerHints, NodeCreateBatchSize, ComputeHostnameFormat, controllerExtraConfig, RedisVirtualFixedIPs, ComputeServices, HypervisorNeutronPublicInterface, ObjectStorageCount, NetworkerCount, CloudName, CloudNameCtlplane, RabbitCookieSalt, NetworkerServices, AddVipsToEtcHosts, NetworkerParameters, CloudNameStorageManagement, DeployIdentifier, NeutronPublicInterface, ControlFixedIPs, StorageVirtualFixedIPs, ObjectStorageRemovalPolicies, ObjectStorageSchedulerHints, ObjectStorageHostnameFormat, OS::stack_id, ObjectStorageServices, ControllerServices, ExtraConfig, ComputeSchedulerHints, ComputeParameters, ComputeRemovalPolicies, HypervisorNeutronPhysicalBridge, ObjectStorageExtraConfig, CloudNameStorage, UpdateIdentifier, ComputeExtraConfig validate /usr/lib/python2.7/site-packages/heat/engine/parameter_groups.py:42
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server [req-01a185af-dc5d-4d5d-85f1-f481fd0f51fa - admin - default default] Exception during message handling: ValueError: Error parsing template http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/puppet/services/swift-storage.yaml mapping values are not allowed here
  in "<unicode string>", line 54, column 16:
        description: Whether to enable Swift Storag ... 
                   ^
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 409, in wrapped
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     return func(self, ctx, *args, **kwargs)
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 1282, in validate_template
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     stack.preview_resources()
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1043, in preview_resources
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     for resource in six.itervalues(self.resources)]
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 199, in preview
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     return self.nested().preview_resources()
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1043, in preview_resources
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     for resource in six.itervalues(self.resources)]
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 199, in preview
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     return self.nested().preview_resources()
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1043, in preview_resources
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     for resource in six.itervalues(self.resources)]
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 199, in preview
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     return self.nested().preview_resources()
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1043, in preview_resources
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     for resource in six.itervalues(self.resources)]
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 189, in preview
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     child_template = self.child_template()
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/engine/resources/template_resource.py", line 179, in child_template
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     self.template_url)
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/common/template_format.py", line 116, in parse
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     tpl = simple_parse(tmpl_str, tmpl_url)
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/heat/common/template_format.py", line 79, in simple_parse
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server     raise ValueError(msg)
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server ValueError: Error parsing template http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/puppet/services/swift-storage.yaml mapping values are not allowed here
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server   in "<unicode string>", line 54, column 16:
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server         description: Whether to enable Swift Storag ... 
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server                    ^
2018-04-24 11:07:06.624 1578 ERROR oslo_messaging.rpc.server 
2018-04-24 11:07:25.872 1576 DEBUG heat.engine.service [req-931210b2-fb91-4e1f-8490-417f5e5dab4c - - - - -] Service daba7711-d30f-498e-a8d3-c5fb2afa1bb1 is updated service_manage_report /usr/lib/python2.7/site-packages/heat/engine/service.py:2423
2018-04-24 11:07:26.620 1577 DEBUG heat.engine.service [req-58bf155c-f9af-429d-8e3f-11c8877c8eea - - - - -] Service 7d30bb57-1fbf-4baf-8f03-0dbeb7bcd0e2 is updated service_manage_report /usr/lib/python2.7/site-packages/heat/engine/service.py:2423
2018-04-24 11:07:26.972 1578 DEBUG heat.engine.service [req-aeb904e4-7a04-432f-8b7c-a370b149240e - - - - -] Service d91f026a-8f9d-454d-9f62-5ef40b6d6c24 is updated service_manage_report /usr/lib/python2.7/site-packages/heat/engine/service.py:2423
2018-04-24 11:07:36.020 1579 DEBUG heat.engine.service [req-e96f28c1-b315-40e5-b851-cee9d211a005 - - - - -] Service fc51eadb-86c7-49ca-bf90-833e3a21028f is updated service_manage_report /usr/lib/python2.7/site-packages/heat/engine/service.py:2423
2018-04-24 11:07:57.017 1578 INFO heat.engine.service [req-1585d4dc-63d6-4bc6-ac97-5f27c79fe0c5 admin admin - default default] Updating stack overcloud

2018-04-24 11:08:13.743 1578 DEBUG heat.engine.stack [req-a6cda339-7387-4fc4-9f37-78499cc5d9fe - - - - -] Error parsing template http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/puppet/services/swift-storage.yaml mapping values are not allowed here
  in "<unicode string>", line 54, column 16:
        description: Whether to enable Swift Storag ... 
                   ^ validate /usr/lib/python2.7/site-packages/heat/engine/stack.py:883
2018-04-24 11:08:13.744 1578 DEBUG heat.engine.stack [req-a6cda339-7387-4fc4-9f37-78499cc5d9fe - - - - -] resources.113<http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/docker/services/swift-storage.yaml>: Error parsing template http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/puppet/services/swift-storage.yaml mapping values are not allowed here
  in "<unicode string>", line 54, column 16:
        description: Whether to enable Swift Storag ... 
                   ^ validate /usr/lib/python2.7/site-packages/heat/engine/stack.py:883
2018-04-24 11:08:13.745 1578 DEBUG heat.engine.stack [req-a6cda339-7387-4fc4-9f37-78499cc5d9fe - - - - -] resources.ServiceChain<nested_stack>.resources.113<http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/docker/services/swift-storage.yaml>: Error parsing template http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/puppet/services/swift-storage.yaml mapping values are not allowed here
  in "<unicode string>", line 54, column 16:
        description: Whether to enable Swift Storag ... 
                   ^ validate /usr/lib/python2.7/site-packages/heat/engine/stack.py:883
2018-04-24 11:08:13.757 1578 DEBUG heat.engine.stack [req-a6cda339-7387-4fc4-9f37-78499cc5d9fe - - - - -] resources.ControllerServiceChain<http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/common/services.yaml>.resources.ServiceChain<nested_stack>.resources.113<http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/docker/services/swift-storage.yaml>: Error parsing template http://192.168.24.1:8080/v1/AUTH_4eb3fe02b7d441d38c88f1d94e0f7757/overcloud/puppet/services/swift-storage.yaml mapping values are not allowed here
  in "<unicode string>", line 54, column 16:
        description: Whether to enable Swift Storag ... 
                   ^ validate /usr/lib/python2.7/site-packages/heat/engine/stack.py:883


Note You need to log in before you can comment on or make changes to this bug.