Bug 1642934
| Summary: | Longer execution time of OS Rest APIS scenarios on OSP14 | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Yariv <yrachman> | ||||||||||
| Component: | python-oslo-service | Assignee: | Hervé Beraud <hberaud> | ||||||||||
| Status: | CLOSED ERRATA | QA Contact: | pkomarov | ||||||||||
| Severity: | high | Docs Contact: | |||||||||||
| Priority: | high | ||||||||||||
| Version: | 14.0 (Rocky) | CC: | aherr, chjones, hberaud, mariel, mkrcmari, oblaut, pkomarov, supadhya, therve, yobshans | ||||||||||
| Target Milestone: | beta | Keywords: | Triaged | ||||||||||
| Target Release: | 14.0 (Rocky) | ||||||||||||
| Hardware: | x86_64 | ||||||||||||
| OS: | Linux | ||||||||||||
| Whiteboard: | |||||||||||||
| Fixed In Version: | python-oslo-service-1.31.5-0.20180927095012.67a8a8c.el7ost | Doc Type: | Bug Fix | ||||||||||
| Doc Text: |
Previously, threading events with eventlet created unnecessary system calls, which reduced performance of the REST API and resulted in timeout failures in Tempest. This fix improves the response time of the REST API calls, and reduces timeout failures in Tempest.
|
Story Points: | --- | ||||||||||
| Clone Of: | Environment: | ||||||||||||
| Last Closed: | 2019-01-11 11:54:26 UTC | Type: | Bug | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Embargoed: | |||||||||||||
| Attachments: |
|
||||||||||||
|
Description
Yariv
2018-10-25 10:37:17 UTC
Created attachment 1497461 [details]
Detailed API comparison response time report
I would like to add detailed excel comparison test report about REST API response time between OSP 13 and OSP 14.
Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3
(In reply to Yuri Obshansky from comment #3) > Created attachment 1497461 [details] > Detailed API comparison response time report > > I would like to add detailed excel comparison test report about REST API > response time between OSP 13 and OSP 14. > Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3 Is this expected? since the increase in api response is nearly doubled. (In reply to Sanjay Upadhyay from comment #4) > (In reply to Yuri Obshansky from comment #3) > > Created attachment 1497461 [details] > > Detailed API comparison response time report > > > > I would like to add detailed excel comparison test report about REST API > > response time between OSP 13 and OSP 14. > > Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3 > > Is this expected? since the increase in api response is nearly doubled. I think, that is not expected. Please take a look at nightly builds: https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-13-3cont_2comp-restapi-perf-test/ https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-14-3cont_2comp-restapi-perf-test/ https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-compare-restapi-perf-test/ Compared puddles: osp-13-2018-10-30.1 vs osp-14-2018-10-30.6 Prometheus Dashboard user/pass (admin/admin) Created attachment 1499534 [details]
patch_apply_in_image
Hi, I've reran REST API performance test on Openstack 14 puddle 2018-10-30.6 with proposed patch. The result is pretty good. Most of APIs response time better. However there is still a degradation compare with Openstack 13. Please, find detailed results here: - Comparison Openstack 14 vs. Openstack 14 + fix https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-compare-restapi-perf-test/3/ http://10.12.68.21//osp-jmeter/result/compare-restapi-perf-test-14-14fix-2018-04-02.3/index.html - Comparison Openstack 13 vs. Openstack 14 + fix https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-compare-restapi-perf-test/2/ http://10.12.68.21//osp-jmeter/result/compare-restapi-perf-test-2018-04-02.3/index.html - REST API Sanity performance test Openstack 14 + fix https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-14-3cont_2comp-restapi-perf-test/9/ http://10.12.68.21//osp-jmeter/result/osp-restapi-perf-test-fix-14-2018-10-30.6/index.html http://10.35.64.12:3000/d/2018/openstack-stats?orgId=1&from=1541113273005&to=1541117189266 Created attachment 1500681 [details]
Detailed API comparison response time report OSP 14 with FIX
Created attachment 1502330 [details]
docker_image_customize_patch.sh
NFV QE applied the patch, Tests executions reduced, from 250sec to 146~168sec, it not failing NFV CI any more, But compare to 13, same test with all of its API calls takes ~110sec Verified, - comment13 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045 |