Description of problem: API calls, take 1.3 percent the time 10 release. Version-Release number of selected component (if applicable): puddle 2018-10-19.3 How reproducible: Always, through CLI calls, This cause tempest scenarios tests to fail while waiting for ACTIVE status in server create rest command. U/S BZ exist and merged two days ago, this BZ it to track Fix downstream Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Created attachment 1497461 [details] Detailed API comparison response time report I would like to add detailed excel comparison test report about REST API response time between OSP 13 and OSP 14. Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3
(In reply to Yuri Obshansky from comment #3) > Created attachment 1497461 [details] > Detailed API comparison response time report > > I would like to add detailed excel comparison test report about REST API > response time between OSP 13 and OSP 14. > Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3 Is this expected? since the increase in api response is nearly doubled.
(In reply to Sanjay Upadhyay from comment #4) > (In reply to Yuri Obshansky from comment #3) > > Created attachment 1497461 [details] > > Detailed API comparison response time report > > > > I would like to add detailed excel comparison test report about REST API > > response time between OSP 13 and OSP 14. > > Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3 > > Is this expected? since the increase in api response is nearly doubled. I think, that is not expected. Please take a look at nightly builds: https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-13-3cont_2comp-restapi-perf-test/ https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-14-3cont_2comp-restapi-perf-test/ https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-compare-restapi-perf-test/ Compared puddles: osp-13-2018-10-30.1 vs osp-14-2018-10-30.6 Prometheus Dashboard user/pass (admin/admin)
Created attachment 1499534 [details] patch_apply_in_image
Hi, I've reran REST API performance test on Openstack 14 puddle 2018-10-30.6 with proposed patch. The result is pretty good. Most of APIs response time better. However there is still a degradation compare with Openstack 13. Please, find detailed results here: - Comparison Openstack 14 vs. Openstack 14 + fix https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-compare-restapi-perf-test/3/ http://10.12.68.21//osp-jmeter/result/compare-restapi-perf-test-14-14fix-2018-04-02.3/index.html - Comparison Openstack 13 vs. Openstack 14 + fix https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-compare-restapi-perf-test/2/ http://10.12.68.21//osp-jmeter/result/compare-restapi-perf-test-2018-04-02.3/index.html - REST API Sanity performance test Openstack 14 + fix https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-14-3cont_2comp-restapi-perf-test/9/ http://10.12.68.21//osp-jmeter/result/osp-restapi-perf-test-fix-14-2018-10-30.6/index.html http://10.35.64.12:3000/d/2018/openstack-stats?orgId=1&from=1541113273005&to=1541117189266
Created attachment 1500681 [details] Detailed API comparison response time report OSP 14 with FIX
Created attachment 1502330 [details] docker_image_customize_patch.sh
NFV QE applied the patch, Tests executions reduced, from 250sec to 146~168sec, it not failing NFV CI any more, But compare to 13, same test with all of its API calls takes ~110sec
Verified, - comment13
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045