Bug 1271143
| Summary: | Listing a single repositories with 1000 puppet modules takes 10+ minutes | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Peter Vreman <peter.vreman> | ||||
| Component: | Content Management | Assignee: | Justin Sherrill <jsherril> | ||||
| Status: | CLOSED ERRATA | QA Contact: | jcallaha | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 6.1.2 | CC: | bbuckingham, chpeters, hklein, jcallaha, jsherril, sthirugn | ||||
| Target Milestone: | Unspecified | Keywords: | Triaged | ||||
| Target Release: | Unused | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-11-19 15:57:13 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1122832 | ||||||
| Attachments: |
|
||||||
Created attachment 1082304 [details]
json response of repository puppet modules
From the Logging it can be seen that 99% of the time is spend in the rendering of the view.
2015-10-13 08:29:10 [I] Processing by Katello::Api::V2::PuppetModulesController#index as */*
2015-10-13 08:29:10 [I] Parameters: {"per_page"=>9999, "api_version"=>"v2", "repository_id"=>"158", "puppet_module"=>{"per_page"=>9999}}
2015-10-13 08:29:10 [I] Authorized user hoici(hoici )
...
2015-10-13 08:39:38 [I] Rendered /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.67/app/views/katello/api/v2/puppet_modules/index.json.rabl within
katello/api/v2/layouts/collection (616511.4ms)
2015-10-13 08:39:39 [I] Completed 200 OK in 629043ms (Views: 615725.0ms | ActiveRecord: 847.5ms)
Sync plans are not used, as mentioned in the a related BZ1253802 Proposed patch: diff --git a/app/views/katello/api/v2/puppet_modules/show.json.rabl b/app/views/katello/api/v2/puppet_modules/show.json.rabl index 2c4fde0..e72285b 100644 --- a/app/views/katello/api/v2/puppet_modules/show.json.rabl +++ b/app/views/katello/api/v2/puppet_modules/show.json.rabl @@ -15,5 +15,5 @@ attributes :tag_list attributes :repoids child :repositories => :repositories do |_repository| - extends 'katello/api/v2/repositories/show' + attributes :id, :name end Patch confirmed to work, below is the result with the patch applied:
[crash] root@li-lc-1578:~# time curl -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XGET https://localhost/katello/api/v2/repositories/158/puppet_modules > p.json
real 0m21.139s
user 0m0.215s
sys 0m0.150s
[crash] root@li-lc-1578:~# ls -l p.json
-rw-r--r--. 1 root root 395472 Oct 13 11:24 p.json
[crash] root@li-lc-1578:~# hoi-json p.json --get ALL | wc -l
13444
[crash] root@li-lc-1578:~# hoi-json p.json --get ALL | grep -c '\.repositories\.'
1990
2015-10-13 11:24:26 [I] Processing by Katello::Api::V2::PuppetModulesController#index as */*
2015-10-13 11:24:26 [I] Parameters: {"per_page"=>9999, "api_version"=>"v2", "repository_id"=>"158", "puppet_module"=>{"per_page"=>9999}}
2015-10-13 11:24:27 [I] Authorized user hoici(hoici )
2015-10-13 11:24:45 [I] Rendered /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.67/app/views/katello/api/v2/puppet_modules/index.json.rabl within katello/api/v2/layouts/collection (6709.1ms)
2015-10-13 11:24:46 [I] Completed 200 OK in 19596ms (Views: 7322.7ms | ActiveRecord: 61.2ms)
Verified. Used 3,684 puppet modules in a single repository. Response time was 730ms in RHEL 7.1 and 820ms in RHEL 6.7 GET - https://cloud-qe-11.idmqe.lab.eng.bos.redhat.com/katello/api/repositories/4/puppet_modules total modules: 3,684. Response time: 730ms HOSTNAME: cloud-qe-11.idmqe.lab.eng.bos.redhat.com OS: redhat RELEASE: Red Hat Enterprise Linux Server release 7.1 (Maipo) FOREMAN: 1.7.2 SATELLITE: 6.1.4 RUBY: ruby 2.0.0p598 (2014-11-13) [x86_64-linux] PUPPET: 3.6.2 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:2474 |
Description of problem: Listing a repositories with almost 1000 puppet modules takes 10 minutes. The flattening the shows that 80% of the returned data are the repositories results. But this is for the requested query to list a specified repository superfluous data. [crash] root@li-lc-1578:~# time curl -K /opt/hoici/etc/sat6/curl-hoici.conf "-HContent-Type: application/json" "-d{\"per_page\":9999}" -XGET https://localhost/katello/api/v2/repositories/158/puppet_modules > p.json real 10m30.165s user 0m0.271s sys 0m0.080s [crash] root@li-lc-1578:~# ls -l p.json -rw-r--r--. 1 root root 1615271 Oct 13 08:39 p.json [crash] root@li-lc-1578:~# hoi-json p.json --get results.id | wc -l 988 [crash] root@li-lc-1578:~# hoi-json p.json --get ALL | wc -l 57130 [crash] root@li-lc-1578:~# hoi-json p.json --get ALL | grep -c '\.repositories\.' 45770 Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Create a puppet repository with 1000 puppet modules 2. List puppet repository using the API 3. Actual results: Slow repsonse Response contains redundant repositories information Expected results: Response <10seconds Response does not include redundant repository information Additional info: