Bug 1793847

Summary: Performance regression in scale deploy timings with config-download
Product: Red Hat OpenStack Reporter: Dave Wilson <dwilson>
Component: python-tripleoclientAssignee: RHOS Maint <rhos-maint>
Status: CLOSED DUPLICATE QA Contact: Sasha Smolyak <ssmolyak>
Severity: high Docs Contact:
Priority: high    
Version: 16.0 (Train)CC: bdobreli, hbrock, jslagle, lshort, mburns, smalleni
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-06-05 14:09:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Description Flags
ansible.log none

Description Dave Wilson 2020-01-22 04:42:53 UTC
Created attachment 1654441 [details]

Description of problem: Scale out deploy timings within the RDU scale lab of OSP16 are taking considerable longer than experiences with OSP13 (non config-download)

Version-Release number of selected component (if applicable): OSP16 (RHOS_TRUNK-16.0-RHEL-8-20200113.n.0)

How reproducible:

Steps to Reproduce:
1. Allocation of 300 baremetal nodes in scale env
2. Scale out env in increments of 50
3. defaults timeouts hit and deploy fails
4. bump up keystone token timeout

Actual results: 
1. Scale out of ~250 nodes with config download took 7.5hr (excludes stack create)

Expected results:
1. Deploy timings equal or less than documented in the  500 BM node scale deploy with OSP13 

Additional info:
1. Including ansible.log for deploy of the 250 node scale out
2  In osp13 the same scale out ( although no ceph which account for 1.5hr of the deploy) took ~2hrs

Comment 3 Luke Short 2020-06-05 14:09:25 UTC
We have various new features coming in a future release of RHOSP 16 including `openstack overcloud deploy --limit` and a new Ansible strategy to help with the deployment time.

*** This bug has been marked as a duplicate of bug 1767581 ***