Bug 1642934 - Longer execution time of OS Rest APIS scenarios on OSP14
Summary: Longer execution time of OS Rest APIS scenarios on OSP14
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-oslo-service
Version: 14.0 (Rocky)
Hardware: x86_64
OS: Linux
high
high
Target Milestone: beta
: 14.0 (Rocky)
Assignee: Hervé Beraud
QA Contact: pkomarov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-10-25 10:37 UTC by Yariv
Modified: 2019-01-11 11:54 UTC (History)
10 users (show)

Fixed In Version: python-oslo-service-1.31.5-0.20180927095012.67a8a8c.el7ost
Doc Type: Bug Fix
Doc Text:
Previously, threading events with eventlet created unnecessary system calls, which reduced performance of the REST API and resulted in timeout failures in Tempest. This fix improves the response time of the REST API calls, and reduces timeout failures in Tempest.
Clone Of:
Environment:
Last Closed: 2019-01-11 11:54:26 UTC
Target Upstream Version:

Attachments (Terms of Use)
Detailed API comparison response time report (30.86 KB, application/zip)
2018-10-25 13:25 UTC, Yuri Obshansky 		no flags Details
patch_apply_in_image (863 bytes, application/x-shellscript)
2018-10-31 17:47 UTC, Yariv 		no flags Details
Detailed API comparison response time report OSP 14 with FIX (44.04 KB, application/zip)
2018-11-02 19:44 UTC, Yuri Obshansky 		no flags Details
docker_image_customize_patch.sh (1.00 KB, application/x-shellscript)
2018-11-06 09:29 UTC, pkomarov 		no flags Details
Show Obsolete (1) Add an attachment (proposed patch, testcase, etc.)


Links
System ID Priority Status Summary Last Updated
Launchpad 1798774 None None None 2018-10-25 10:37:16 UTC
OpenStack gerrit 614489 None MERGED Use eventlet Event for loopingcall events 2020-02-28 12:38:45 UTC
OpenStack gerrit 614590 None MERGED Stop asserting on Eventlet internals 2020-02-28 12:38:45 UTC
OpenStack gerrit 615263 None MERGED Fix stop of loopingcall 2020-02-28 12:38:45 UTC
OpenStack gerrit 616505 None MERGED oslo.service 1.31.6 2020-02-28 12:38:45 UTC
Red Hat Product Errata RHEA-2019:0045 None None None 2019-01-11 11:54:32 UTC

Description Yariv 2018-10-25 10:37:17 UTC 
Description of problem:

API calls, take 1.3 percent the time 10 release.


Version-Release number of selected component (if applicable):
puddle 2018-10-19.3

How reproducible:
Always, through CLI calls, 
This cause tempest scenarios tests to fail while waiting for ACTIVE status in server create rest command.

U/S BZ exist and merged two days ago, this BZ it to track Fix downstream


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 3 Yuri Obshansky 2018-10-25 13:25:52 UTC 
Created attachment 1497461 [details]
Detailed API comparison response time report

I would like to add detailed excel comparison test report about REST API response time between OSP 13 and OSP 14. 
Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3
Comment 4 Sanjay Upadhyay 2018-10-31 11:04:37 UTC 
(In reply to Yuri Obshansky from comment #3)
> Created attachment 1497461 [details]
> Detailed API comparison response time report
> 
> I would like to add detailed excel comparison test report about REST API
> response time between OSP 13 and OSP 14. 
> Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3

Is this expected? since the increase in api response is nearly doubled.
Comment 5 Yuri Obshansky 2018-10-31 12:58:37 UTC 
(In reply to Sanjay Upadhyay from comment #4)
> (In reply to Yuri Obshansky from comment #3)
> > Created attachment 1497461 [details]
> > Detailed API comparison response time report
> > 
> > I would like to add detailed excel comparison test report about REST API
> > response time between OSP 13 and OSP 14. 
> > Puddles: osp-13-p-2018-10-02.1 and osp-14-2018-10-10.3
> 
> Is this expected? since the increase in api response is nearly doubled.

I think, that is not expected. 
Please take a look at nightly builds:
https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-13-3cont_2comp-restapi-perf-test/
https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-14-3cont_2comp-restapi-perf-test/
https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/day2testing/view/ospjmeter/job/DFG-day2testing-ospjmeter-compare-restapi-perf-test/

Compared puddles: osp-13-2018-10-30.1 vs osp-14-2018-10-30.6
Prometheus Dashboard user/pass (admin/admin)
Comment 7 Yariv 2018-10-31 17:47:41 UTC 
Created attachment 1499534 [details]
patch_apply_in_image
Comment 10 Yuri Obshansky 2018-11-02 19:44:55 UTC 
Created attachment 1500681 [details]
Detailed API comparison response time report OSP 14 with FIX
Comment 11 pkomarov 2018-11-06 09:29:08 UTC 
Created attachment 1502330 [details]
docker_image_customize_patch.sh
Comment 13 Yariv 2018-11-08 11:48:37 UTC 
NFV QE applied the patch, 
Tests executions reduced, from 250sec to 146~168sec, it not failing NFV CI any more,
But compare to 13, same test with all of its API calls takes ~110sec
Comment 16 pkomarov 2018-11-13 13:16:47 UTC 
Verified, - comment13
Comment 19 errata-xmlrpc 2019-01-11 11:54:26 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045
Note You need to log in before you can comment on or make changes to this bug.