Bug 747336
| Summary: | Performance problems after uploading manifest | ||
|---|---|---|---|
| Product: | [Retired] Pulp | Reporter: | Jeff Weiss <jweiss> |
| Component: | z_other | Assignee: | James Slagle <jslagle> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Preethi Thomas <pthomas> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 1.0.0 | CC: | bkearney, dajohnso, gkhachik, jason.dobies, jsherril, jslagle, mmccune, tstrachota |
| Target Milestone: | --- | Keywords: | TestBlocker, Triaged |
| Target Release: | Sprint 30 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | --- | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 747354, 757752 | ||
|
Description
Jeff Weiss
2011-10-19 14:24:05 UTC
Revising what pages are slow - looks like Sync Status page, Promotions (esp clicking into packages) . The other pages were slow for me, because they were being loaded at the same time, in another browser- but when loaded when the system is not already under load, it's ok. Specifically we need some bulk APIs to fetch repo status for various repositories in one call. If you have 300+ repos in a product it should not require 300+ REST GET calls to fetch the status of each. Instead of making 300+ calls to: GET /pulp/api/repositories/<repoid>/sync/ we should instead have one rollup call that can fetch the status for all repos in a product. supposedly there is a pulp bug that is addressing this and I'll comment there as well. A bulk call is needed for repo sync history as well as status. A bulk call is also needed for GET /pulp/api/repositories/<id>/distribution/ We need to be able to pull in the distributions for a set of repositories (and know which distributions go with which repositories). Added bulk support for these 3 items. I detailed the API's on a wiki page, since it might be a lot for bugzilla: https://fedorahosted.org/pulp/wiki/RepoBulkAPIEnhancements build: 0.252 moving to verified root@preethi ~]# rpm -q pulp pulp-0.0.255-1.fc15.noarch [root@preethi ~]# [root@preethi ~]# curl -k -u admin:admin -X GET https://localhost/pulp/api/statuses/ |json_reformat % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1836 100 1836 0 0 1036 0 0:00:01 0:00:01 --:--:-- 1036 [ { "progress": { "status": "FINISHED", "num_success": 2941, "size_total": 3369661596, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 2941, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 6, "total_count": 6, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 2935, "total_count": 2935, "items_left": 0, "size_left": 0, "total_size_bytes": 3369661596, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "f15", "traceback": null }, { "progress": { "status": "FINISHED", "num_success": 3113, "size_total": 3455015673, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 3113, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 6, "total_count": 6, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 3107, "total_count": 3107, "items_left": 0, "size_left": 0, "total_size_bytes": 3455015673, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "f16", "traceback": null }, { "progress": { "status": "FINISHED", "num_success": 6298, "size_total": 5467671020, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 6298, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 4, "total_count": 4, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 6294, "total_count": 6294, "items_left": 0, "size_left": 0, "total_size_bytes": 5467671020, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "centos", "traceback": null } ] [root@preethi ~]# curl -k -u admin:admin -X GET https://localhost/pulp/api/statuses/repository/syncs/?repoid=centos |json_reformat % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 614 0 614 0 0 1331 0 --:--:-- --:--:-- --:--:-- 1331 [ { "progress": { "status": "FINISHED", "num_success": 6298, "size_total": 5467671020, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 6298, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 4, "total_count": 4, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 6294, "total_count": 6294, "items_left": 0, "size_left": 0, "total_size_bytes": 5467671020, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "centos", "traceback": null } ] [root@preethi ~]# curl -k -u admin:admin -X GET https://localhost/pulp/api/histories/ |json_reformat |more % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 107k 100 107k 0 0 440k 0 --:--:-- --:--:-- --:--:-- 443k[ { "scheduled_time": "2012-01-04T19:59:22Z", "exception": null, "task_type": "RepoSyncTask", "job_id": null, "args": [ "f16" ], "class_name": null, "_ns": "task_history", "start_time": "2012-01-04T14:59:22-05:00", "traceback": null, "task_string": "Task 989ad270-370e-11e1-b45d-002564a85a58: _sync(f16, skip={ }, max_speed=None, threads=None)", "method_name": "_sync", "finish_time": "2012-01-04T15:05:09-05:00", "state": "finished", "result": true, "kwargs": { "skip": { }, [root@preethi ~]# curl -k -u admin:admin -X GET "https://localhost/pulp/api/statuses/repository/syncs/?repoid=centos&repoid=f15&_union=repoid" |json_reformat % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1225 100 1225 0 0 1522 0 --:--:-- --:--:-- --:--:-- 1525 [ { "progress": { "status": "FINISHED", "num_success": 2941, "size_total": 3369661596, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 2941, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 6, "total_count": 6, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 2935, "total_count": 2935, "items_left": 0, "size_left": 0, "total_size_bytes": 3369661596, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "f15", "traceback": null }, { "progress": { "status": "FINISHED", "num_success": 6298, "size_total": 5467671020, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 6298, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 4, "total_count": 4, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 6294, "total_count": 6294, "items_left": 0, "size_left": 0, "total_size_bytes": 5467671020, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "centos", "traceback": null } ] Pulp v1.0 is released Closed Current Release. Pulp v1.0 is released. |