Bug 1177609

Summary: Content View index page takes a long time to list
Product: Red Hat Satellite Reporter: Peter Vreman <peter.vreman>
Component: APIAssignee: Walden Raines <walden>
Status: CLOSED ERRATA QA Contact: jaudet
Severity: high Docs Contact:
Priority: high    
Version: 6.0.8CC: bbuckingham, cwelton, hklein, jmontleo, walden
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: http://projects.theforeman.org/issues/9564
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1197722 (view as bug list) Environment:
Last Closed: 2015-08-12 05:21:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1122832, 1197722    
Attachments:
Description Flags
Script for populating server with content. none

Description Peter Vreman 2014-12-29 12:18:54 UTC
Description of problem:
Similar to issue BZ1161636 for the ContentViewVersions, also the index of the Content View page takes a long time. See below results on a VM with 4vCPUs and 16MB that it takes 45 seconds to list only 14 CVs:

# time curl -K /opt/hoici/etc/curl-hoici.conf -H 'Content-Type: application/json' -d '{"per_page":9999}' -XGET https://localhost/katello/api/v2/organizations/4/content_views > p1.json

real    0m44.786s
user    0m0.201s
sys     0m0.075s
# ls -l p1.json
-rw-r--r-- 1 root root 188770 Dec 29 12:02 p1.json
# cat p1.json | jq .results[].id | wc -l
14
# cat p1.json | jq .results[].puppet_modules[].id | wc -l
295

It also contains many (repeated) details about the product sync_plans, which are superfluous information:
# cat p1.json | jq .results[].repositories[].product.sync_plan.id | wc -l
60
# cat p1.json | jq .results[].repositories[].product.sync_plan.products[].id 2> /dev/null | wc -l
258



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create 10 Products (or use RHEL products)
2. Create 1 Sync plan with all 10 Products
3. Create 50 Puppet modules
4. Create 20 CVs with the products and puppet modules
5. List the CVs

Actual results:


Expected results:
List of CVs returned within 5 seconds

Additional info:
There might be more places where too much detailed information is inherited by including the "show" views of the relations in the index part. Maybe create a index-show and a detailed-show view per resource type. The index-show will be included in the index views or relationship includes. The detailed-show will be used only for the show option to display the resource itself.

Comment 1 RHEL Program Management 2014-12-29 12:23:58 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 3 Peter Vreman 2015-02-26 07:13:37 UTC
This also affects the Content View dropdown boxes at the following locations:

- Activation Key - Associate with Content View
- Composite Content View - Add Content View

Comment 4 Peter Vreman 2015-02-26 07:15:20 UTC
Updating version to 6.0.8, as it is still not resolved, even with BZ1161636 (slow Content View Versions listing) fixed

Comment 5 Walden Raines 2015-02-26 15:14:29 UTC
Created redmine issue http://projects.theforeman.org/issues/9564 from this bug

Comment 6 Walden Raines 2015-02-26 15:30:57 UTC
PR: https://github.com/Katello/katello/pull/5054

Comment 7 Bryan Kearney 2015-02-27 15:04:47 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/9564 has been closed
-------------
Walden Raines
Applied in changeset commit:katello|726b9c4f1b34635311074ac9a618d6231633e3b7.

Comment 8 Peter Vreman 2015-03-02 11:36:23 UTC
Confirmed that the patch works:

# time curl -K /opt/hoici/etc/sat6/curl-hoici.conf -H 'Content-Type: application/json' -d '{"per_page":9999}' -XGET https://localhost/katello/api/v2/organizations/4/content_views > p1.json

real    0m2.512s
user    0m0.184s
sys     0m0.051s
# ls -l p1.json
-rw-r--r-- 1 root root 90979 Mar  2 11:33 p1.json
# cat p1.json | jq .results[].id | wc -l
34
# cat p1.json | jq .results[].puppet_modules[].id | wc -l
275
# cat p1.json | jq .results[].repositories[] | grep sync_plan | wc -l
0

Comment 9 Peter Vreman 2015-03-02 14:19:32 UTC
The patch is missing the content_counts for the repositories for the UI to display

--- /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/views/katello/api/v2/content_views/_content_view.json.rabl
+++ /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/views/katello/api/v2/content_views/_content_view.json.rabl
@@ -23,7 +23,12 @@
 end

 child :repositories => :repositories do
-  extends 'katello/api/v2/repositories/show'
+  attributes :id, :name, :label, :content_type
+  node :content_counts do |repo|
+    if repo.respond_to?(:pulp_repo_facts)
+      repo.pulp_repo_facts['content_unit_counts']
+    end
+  end
 end

 child :puppet_modules => :puppet_modules do

Comment 10 Walden Raines 2015-03-02 17:16:23 UTC
(In reply to Peter Vreman from comment #9)
> The patch is missing the content_counts for the repositories for the UI to
> display
> 
> ---
> /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/views/katello/api/
> v2/content_views/_content_view.json.rabl
> +++
> /opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/views/katello/api/
> v2/content_views/_content_view.json.rabl
> @@ -23,7 +23,12 @@
>  end
> 
>  child :repositories => :repositories do
> -  extends 'katello/api/v2/repositories/show'
> +  attributes :id, :name, :label, :content_type
> +  node :content_counts do |repo|
> +    if repo.respond_to?(:pulp_repo_facts)
> +      repo.pulp_repo_facts['content_unit_counts']
> +    end
> +  end
>  end
> 
>  child :puppet_modules => :puppet_modules do

I have created a new bug for this issue: bug #1197836

Comment 15 jaudet 2015-03-13 21:44:06 UTC
Tested against Satellite-6.1.0-RHEL-6-20150311.1. Server had four cores and six gigs of ram. Followed test procedure, except only uploaded seventeen puppet modules instead of fifty. Test result:

$ time curl -k -H 'Content-Type: application/json' --user …:… -d '{"per_page":9999}' -XGET https://localhost/katello/api/v2/organizations/331/content_views > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 74658    0 74658    0    17  91078     20 --:--:-- --:--:-- --:--:--  106k

real    0m0.830s
user    0m0.049s
sys     0m0.067s

Comment 16 jaudet 2015-03-13 21:45:26 UTC
Created attachment 1001508 [details]
Script for populating server with content.

Comment 17 Bryan Kearney 2015-08-11 13:29:52 UTC
This bug is slated to be released with Satellite 6.1.

Comment 18 errata-xmlrpc 2015-08-12 05:21:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:1592