Bug 1111574 - content host API - json returned by index takes too long to process
Summary: content host API - json returned by index takes too long to process
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: API
Version: 6.0.3
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: Roman Plevka
URL: http://projects.theforeman.org/issues...
Whiteboard:
Depends On:
Blocks: sat61-release-notes
TreeView+ depends on / blocked
 
Reported: 2014-06-20 12:08 UTC by Tom McKay
Modified: 2019-04-01 20:26 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-27 08:42:02 UTC


Attachments (Terms of Use)
listing time scatter polot (16.07 KB, image/png)
2016-05-03 15:55 UTC, Roman Plevka
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1500 normal SHIPPED_LIVE Red Hat Satellite 6.2 Base Libraries 2016-07-27 12:24:38 UTC
Foreman Issue Tracker 6307 None None None 2016-04-22 16:36:44 UTC

Description Tom McKay 2014-06-20 12:08:05 UTC
On a modest data set (~350 hosts, a couple of those hypervisors with several dozen guests) hammer times out when calling index to fetch all 350.

Suggest rabl be split up as is done in the foreman code such that there are layers depending if index or show is being called.

Comment 1 Tom McKay 2014-06-20 12:08:07 UTC
Created from redmine issue http://projects.theforeman.org/issues/6307

Comment 7 Bryan Kearney 2015-08-25 17:24:29 UTC
Upstream bug component is API

Comment 8 Justin Sherrill 2016-03-16 14:05:01 UTC
Note that the /systems api is being removed in favor of the /hosts api.  The /hosts api does not return information from any backend service and should be much faster.

That said 300-350 is a very arbitrary number and the whole point of pagination is to be able to fetch in some amount of chunks.  This bz is arguing that 300 should be fast, but should 3000?  300000? What is the limit?

I'll move to ON_QA to test that 300 can be fetched fairly quickly.

Comment 10 Roman Plevka 2016-05-03 15:54:53 UTC
VERIFIED
I've run the following script to measure time of host listing as they were created (every 10th).
The time for ~300 hosts is fair enough - ~15 seconds but the time seems to exponentially grow (see attached screenshot).

Anyway, the hammer command does not time out even after 70 seconds. As Justin has pointed out, there's the pagination functionality which gives user control over how many records are being fetched by request.

  80 +-+----+------+------+------+------+------+------+------+------+----+-+
     +      +      +      +      +      +      +      +      +      +    AAA
  70 +-+                                     "times.log" using 1:2    AAA+-+
     |                                                              AA     |
  60 +-+                                                         AAA     +-+
     |                                                         AAA         |
     |                                                     AAAA            |
  50 +-+                                               A AA              +-+
     |                                              AAAAA                  |
  40 +-+                                         A A A                   +-+
     |                                      AAAAA A                        |
  30 +-+                                 AAAA                            +-+
     |                               AAAA                                  |
  20 +-+                        AAAAAA                                   +-+
     |                     AAAAAA                                          |
     |                AAAAAA                                               |
  10 +-+       AAAAAAA                                                   +-+
     +AAAAAAAAA    +      +      +      +      +      +      +      +      +
   0 +-+----+------+------+------+------+------+------+------+------+----+-+
     0     100    200    300    400    500    600    700    800    900    1000
                                     hosts #

Comment 11 Roman Plevka 2016-05-03 15:55:49 UTC
Created attachment 1153492 [details]
listing time scatter polot

Comment 12 Roman Plevka 2016-05-03 16:04:20 UTC
Verified with Sat 6.2.0 BETA

Comment 14 errata-xmlrpc 2016-07-27 08:42:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1500


Note You need to log in before you can comment on or make changes to this bug.