Description of problem: Running `openstack overcloud config download --config-dir config` takes hours to complete. Version-Release number of selected component (if applicable): OSP 13 Additional info: The download seems to proceed normally at first, but then seems to get severaly bogged down when downloading individual resources in calls like: GET call to orchestration for http://172.16.0.5:8004/v1/dbf7e0522d59438a936b0ccc0527b44a/stacks/overcloud-AllNodesDeploySteps-zq5zbvsplduz-ComputeDeployedServerExtraConfigPost-jue4bwaxrvay-KeystoneFederationDeployments-2zvdpmmpvioo-0-5kvttgq235s6/5984bc4d-6f7a-40f3-b05f-867eaaa51f48?resolve_outputs=False&resolve_outputs=False used request id req-cf624b16-6b47-4543-ba98-31c8df745006 REQ: curl -g -i -X GET http://172.16.0.5:8004/v1/dbf7e0522d59438a936b0ccc0527b44a/stacks/overcloud/resources?physical_resource_id=aaf39de8-d66b-4db6-bd36-6bc77bba1043&nested_depth=7 -H "User-Agent: python-heatclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}b0501da04cc9b32c884bfbc7ff17ec13c0d2e4d1" http://172.16.0.5:8004 "GET /v1/dbf7e0522d59438a936b0ccc0527b44a/stacks/overcloud/resources?physical_resource_id=aaf39de8-d66b-4db6-bd36-6bc77bba1043&nested_depth=7 HTTP/1.1" 302 533 RESP: [302] Date: Wed, 31 Oct 2018 17:17:48 GMT Server: Apache x-openstack-request-id: req-fefb4b7a-47b2-4874-b1ad-dec5b8538fab Location: http://172.16.0.5:8004/v1/dbf7e0522d59438a936b0ccc0527b44a/stacks/overcloud/4052acc8-958d-407a-b1f2-ade1e9bd328a/resources?physical_resource_id=aaf39de8-d66b-4db6-bd36-6bc77bba1043&nested_depth=7 Content-Length: 533 Keep-Alive: timeout=15, max=62 Connection: Keep-Alive Content-Type: application/json RESP BODY: {"message": "The resource was found at <a href=\"http://172.16.0.5:8004/v1/dbf7e0522d59438a936b0ccc0527b44a/stacks/overcloud/4052acc8-958d-407a-b1f2-ade1e9bd328a/resources?physical_resource_id=aaf39de8-d66b-4db6-bd36-6bc77bba1043&nested_depth=7\">http://172.16.0.5:8004/v1/dbf7e0522d59438a936b0ccc0527b44a/stacks/overcloud/4052acc8-958d-407a-b1f2-ade1e9bd328a/resources?physical_resource_id=aaf39de8-d66b-4db6-bd36-6bc77bba1043&nested_depth=7</a>;\nyou should be redirected automatically.\n\n", "code": "302 Found", "title": "Found"}
Is that captured output from when you ran the command with --debug? Either way, can you try the command with --debug, save the output and attach it for us to examine? Even a partial log would be helpful. It shouldn't take more than a few minutes, so what you've captured after 5 or so minutes would probably be helpful for us to see what is happening. How many nodes are in this deployment?
The was --debug output. I'll re-run the download and attach the log to this bz shortly. There are 3 controllers, 2 network hosts, and 15 compute nodes in this deployment.
Let us know when the log is available.
Created attachment 1500081 [details] output of config download --debug This is the log from running `openstack --debug overcloud config download --config-dir config` and waiting about 10 minutes.
can you add a name property to each of the OS::Heat::SoftwareDeploymentGroup resources in your templates/postconfig.yaml? I think there is a performance issue with trying to fetch the name dynamically if it's not set. The name can just be the name of the resource. So as an example, it would look like: KeystoneFederationDeployments: type: OS::Heat::SoftwareDeploymentGroup depends_on: CreateCloudYamlDeployments properties: name: KeystoneFederationDeployments # <------- new line servers: {get_param: servers} That needs to be done for each OS::Heat::SoftwareDeploymentGroup in your custom template.
I removed most of the SoftwareDeploymentGroups from the configuration, and added a name to the remaining one [1]. This seems to have fixed things up: just now, running 'config download' completed in under two minutes. [1]: https://github.com/CCI-MOC/rhosp-director-config/blob/master/templates/postconfig.yaml
We've improved the config download call to do something a bit more optimal if the name is not configured on a SoftwareDeploymentGroup. It's still a good idea to use a name on the SoftwareDeploymentGroup as that will use the best path in the code. *** This bug has been marked as a duplicate of bug 1738344 ***