Bug 670532
| Summary: | Performance Issue with re-sync of f14-updates | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Retired] Pulp | Reporter: | John Matthews <jmatthew> | ||||
| Component: | z_other | Assignee: | John Matthews <jmatthew> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Preethi Thomas <pthomas> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | unspecified | CC: | tsanders | ||||
| Target Milestone: | --- | Keywords: | Triaged | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2011-08-16 12:08:25 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 647488 | ||||||
| Attachments: |
|
||||||
|
Description
John Matthews
2011-01-18 15:42:56 UTC
Created attachment 474081 [details]
/var/log/pulp/pulp.log
For sync in comment #0, after download packages was completed, it took 90 minutes to finish sync operations. Looking at resyncs of local channels Example rhel-i386-server-5 resync takes ~20 mins. 20 mins local sync rhel-i386-server-5 - 10 mins md5sum check of ~7100 packages - 5 mins spent determining old packages to remove - 3 mins spent errata processing Same resync as of comment #3 now completes in under 13 minutes. Changes made were to reduce the time for: - Determining old packages and old errata Sync of f14-testing as noted in comment #0 completed in under 7 minutes. QE: When testing this bug I would recommend you sync Fedora 14 Testing, then wait two days and re-sync it. You should see several packages/errata have been updated. Previously it would take roughly 30 seconds to process each package to remove. If you look at the logs you can see how many packages/errata are old and being removed. This is also communicated back to the user through the foreground sync status as text saying: "Removing 95 old packages". [INFO][Thread-5] _sync() @ repo.py:1195 - Sync returned 1118 packages, 397 errata in 346.108876944 seconds [INFO][Thread-5] _sync() @ repo.py:1206 - 95 old packages to process, 92 new packages to process ... ... ... [INFO][Thread-5] _sync() @ repo.py:1220 - Examining 392 errata from repo f14_testing [INFO][Thread-5] _sync() @ repo.py:1224 - Removing 20 old errata from repo f14_testing Fixed in 0.134. 2011-03-21 09:17:08,314 [INFO][Thread-6] _sync() @ repo.py:1423 - Sync returned 7038 packages, 1997 errata in 346.846409082 seconds 2011-03-21 09:17:10,342 [INFO][Thread-6] _sync() @ repo.py:1434 - 209 old packages to process, 410 new packages to process 2011-03-21 09:25:11,362 [INFO][Thread-6] _sync() @ repo.py:1448 - Examining 1935 errata from repo f14-update 2011-03-21 09:25:11,772 [INFO][Thread-6] _sync() @ repo.py:1452 - Removing 47 old errata from repo f14-update 2011-03-21 09:25:14,153 [DEBUG][Thread-6] remove_packages() @ repo.py:907 - remove_packages invoked on f14-update with no packages verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.151-1.fc14.noarch
sync seems to take around 15 minutes
Start time
2011-03-21 09:11:21,466 [INFO][Thread-6] _sync() @ repo.py:1412 - Sync of f14-update starting, skip_dict = {}
End time
2011-03-21 09:25:18,567 [INFO][Thread-6] modify_repo() @ util.py:362 - modifyrepo with /var/lib/pulp//repos/pub/fedora/linux//updates/14/x86_64//updateinfo.xml on /var/lib/pulp//repos/pub/fedora/linux//updates/14/x86_64/repodata finished
2011-03-21 09:25:18,987 [INFO][Thread-6] _sync() @ repo.py:1456 - Adding 109 new errata to repo f14-update
Closing with Community Release 15 pulp-0.0.223-4. Closing with Community Release 15 pulp-0.0.223-4. |