Description of problem: After uploading a manifest from http://axiom.rdu.redhat.com/git/gitweb.cgi?p=system-engine;a=blob_plain;f=manifests/stageManifest06October2011.zip;hb=HEAD The UI performance tanks in certain areas (not an exhaustive list): Sync (syncing 1 small repo takes way longer than it should) Systems page load System subscriptions pane load Version-Release number of selected component (if applicable): katello-0.1.95-1.git.11.27446a9.el6.noarch How reproducible: always Steps to Reproduce: 1. Upload manifest specified above 2. Click around in UI 3. Actual results: Extreme slowness Expected results: Normal performance as before the manifest was uploaded Additional info: The upload of the manifest itself is also slow
Revising what pages are slow - looks like Sync Status page, Promotions (esp clicking into packages) . The other pages were slow for me, because they were being loaded at the same time, in another browser- but when loaded when the system is not already under load, it's ok.
Specifically we need some bulk APIs to fetch repo status for various repositories in one call. If you have 300+ repos in a product it should not require 300+ REST GET calls to fetch the status of each. Instead of making 300+ calls to: GET /pulp/api/repositories/<repoid>/sync/ we should instead have one rollup call that can fetch the status for all repos in a product. supposedly there is a pulp bug that is addressing this and I'll comment there as well.
A bulk call is needed for repo sync history as well as status.
A bulk call is also needed for GET /pulp/api/repositories/<id>/distribution/ We need to be able to pull in the distributions for a set of repositories (and know which distributions go with which repositories).
Added bulk support for these 3 items. I detailed the API's on a wiki page, since it might be a lot for bugzilla: https://fedorahosted.org/pulp/wiki/RepoBulkAPIEnhancements
build: 0.252
moving to verified root@preethi ~]# rpm -q pulp pulp-0.0.255-1.fc15.noarch [root@preethi ~]# [root@preethi ~]# curl -k -u admin:admin -X GET https://localhost/pulp/api/statuses/ |json_reformat % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1836 100 1836 0 0 1036 0 0:00:01 0:00:01 --:--:-- 1036 [ { "progress": { "status": "FINISHED", "num_success": 2941, "size_total": 3369661596, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 2941, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 6, "total_count": 6, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 2935, "total_count": 2935, "items_left": 0, "size_left": 0, "total_size_bytes": 3369661596, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "f15", "traceback": null }, { "progress": { "status": "FINISHED", "num_success": 3113, "size_total": 3455015673, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 3113, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 6, "total_count": 6, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 3107, "total_count": 3107, "items_left": 0, "size_left": 0, "total_size_bytes": 3455015673, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "f16", "traceback": null }, { "progress": { "status": "FINISHED", "num_success": 6298, "size_total": 5467671020, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 6298, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 4, "total_count": 4, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 6294, "total_count": 6294, "items_left": 0, "size_left": 0, "total_size_bytes": 5467671020, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "centos", "traceback": null } ] [root@preethi ~]# curl -k -u admin:admin -X GET https://localhost/pulp/api/statuses/repository/syncs/?repoid=centos |json_reformat % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 614 0 614 0 0 1331 0 --:--:-- --:--:-- --:--:-- 1331 [ { "progress": { "status": "FINISHED", "num_success": 6298, "size_total": 5467671020, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 6298, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 4, "total_count": 4, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 6294, "total_count": 6294, "items_left": 0, "size_left": 0, "total_size_bytes": 5467671020, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "centos", "traceback": null } ] [root@preethi ~]# curl -k -u admin:admin -X GET https://localhost/pulp/api/histories/ |json_reformat |more % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 107k 100 107k 0 0 440k 0 --:--:-- --:--:-- --:--:-- 443k[ { "scheduled_time": "2012-01-04T19:59:22Z", "exception": null, "task_type": "RepoSyncTask", "job_id": null, "args": [ "f16" ], "class_name": null, "_ns": "task_history", "start_time": "2012-01-04T14:59:22-05:00", "traceback": null, "task_string": "Task 989ad270-370e-11e1-b45d-002564a85a58: _sync(f16, skip={ }, max_speed=None, threads=None)", "method_name": "_sync", "finish_time": "2012-01-04T15:05:09-05:00", "state": "finished", "result": true, "kwargs": { "skip": { }, [root@preethi ~]# curl -k -u admin:admin -X GET "https://localhost/pulp/api/statuses/repository/syncs/?repoid=centos&repoid=f15&_union=repoid" |json_reformat % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1225 100 1225 0 0 1522 0 --:--:-- --:--:-- --:--:-- 1525 [ { "progress": { "status": "FINISHED", "num_success": 2941, "size_total": 3369661596, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 2941, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 6, "total_count": 6, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 2935, "total_count": 2935, "items_left": 0, "size_left": 0, "total_size_bytes": 3369661596, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "f15", "traceback": null }, { "progress": { "status": "FINISHED", "num_success": 6298, "size_total": 5467671020, "num_download": 0, "item_name": null, "items_left": 0, "items_total": 6298, "item_type": "", "step": "Importing data into pulp", "size_left": 0, "details": { "tree_file": { "num_success": 4, "total_count": 4, "items_left": 0, "size_left": 0, "total_size_bytes": 0, "num_error": 0 }, "rpm": { "num_success": 6294, "total_count": 6294, "items_left": 0, "size_left": 0, "total_size_bytes": 5467671020, "num_error": 0 } }, "error_details": [ ], "num_error": 0 }, "state": "finished", "exception": null, "repoid": "centos", "traceback": null } ]
Pulp v1.0 is released Closed Current Release.
Pulp v1.0 is released.