Bug 1264560
| Summary: | As a user, my rpm sync finishes quickly when upstream metadata hasn't changed | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Mike McCune <mmccune> |
| Component: | Pulp | Assignee: | Justin Sherrill <jsherril> |
| Status: | CLOSED ERRATA | QA Contact: | Tazim Kolhar <tkolhar> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | Unspecified | CC: | bbuckingham, bkearney, bmbouter, chpeters, cwelton, daviddavis, dkliban, ggainey, ipanova, jcallaha, mhrivnak, mmccune, pcreech, psuriset, rchan, tkolhar, ttereshc |
| Target Milestone: | Unspecified | Keywords: | Triaged |
| Target Release: | Unused | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| URL: | http://projects.theforeman.org/issues/12470 | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Satellite was performing sync post-processing even if a repository did not change. The code was updated to skip this processing if it is not necessary.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-01-21 07:42:19 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The Pulp upstream bug status is at VERIFIED. Updating the external tracker on this bug. The Pulp upstream bug priority is at High. Updating the external tracker on this bug. *** Bug 1269833 has been marked as a duplicate of this bug. *** Failed QA in Satellite 6.1.4. Sync still takes ~3 minutes with no new packages.
Started at: 2015-11-04 17:00:29 UTC
Ended at: 2015-11-04 17:03:32 UTC
RELEASE: Red Hat Enterprise Linux Server release 7.1 (Maipo)
FOREMAN: 1.7.2
SATELLITE: 6.1.4
RUBY: ruby 2.0.0p598 (2014-11-13) [x86_64-linux]
PUPPET: 3.6.2
Additionally, the repository being re-sync'd was Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server. So, this bug is 2 parts, we got the Pulp part included in 6.1.4 but there still needs to be work done in Katello to detect when no-new metadata was synced so we can skip reindexing and other tasks. There will be improvement in 6.1.4 but not the entire resolution as needed. Going to move this to 6.1.6 and have the Katello side of things addressed. Created redmine issue http://projects.theforeman.org/issues/12470 from this bug [SATELLITE-6.1.0 2b3202a] Fixes #12470 - skip post-sync actions if no new content was synced Author: Justin Sherrill <jsherril> Date: Fri Nov 13 09:43:01 2015 -0500 9 files changed, 86 insertions(+), 22 deletions(-) create mode 100644 app/lib/actions/katello/repository/index_content.rb create mode 100644 app/lib/actions/middleware/execute_if_contents_changed.rb Hi, please provide verification steps. thanks and regards, Tazim This caused a regression: https://bugzilla.redhat.com/show_bug.cgi?id=1297308 FAILEDQA: for sat6.1.6 compose 5 # rpm -qa | grep foreman hp-dl180g6-01.rhts.eng.bos.redhat.com-foreman-proxy-1.0-1.noarch ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch foreman-vmware-1.7.2.50-1.el7sat.noarch rubygem-hammer_cli_foreman_tasks-0.0.3.5-1.el7sat.noarch foreman-selinux-1.7.2.17-1.el7sat.noarch ruby193-rubygem-foreman_bootdisk-4.0.2.14-1.el7sat.noarch foreman-ovirt-1.7.2.50-1.el7sat.noarch foreman-1.7.2.50-1.el7sat.noarch ruby193-rubygem-foreman_docker-1.2.0.24-1.el7sat.noarch ruby193-rubygem-foreman-tasks-0.6.15.7-1.el7sat.noarch rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch rubygem-hammer_cli_foreman_docker-0.0.3.10-1.el7sat.noarch foreman-debug-1.7.2.50-1.el7sat.noarch foreman-proxy-1.7.2.8-1.el7sat.noarch hp-dl180g6-01.rhts.eng.bos.redhat.com-foreman-client-1.0-1.noarch hp-dl180g6-01.rhts.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch foreman-discovery-image-3.0.5-3.el7sat.noarch ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch foreman-libvirt-1.7.2.50-1.el7sat.noarch foreman-gce-1.7.2.50-1.el7sat.noarch rubygem-hammer_cli_foreman-0.1.4.15-1.el7sat.noarch ruby193-rubygem-foreman_discovery-2.0.0.23-1.el7sat.noarch foreman-postgresql-1.7.2.50-1.el7sat.noarch foreman-compute-1.7.2.50-1.el7sat.noarch ruby193-rubygem-foreman-redhat_access-0.2.4-1.el7sat.noarch rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch Tried to sync RHEL6.6 repo with no new packages : but still it took ~3 mins Id: c287bf96-9010-4229-852d-925ddbaea7a8 Label: Actions::Katello::Repository::Sync Name: Synchronize Owner: admin Started at: 2016-01-16 15:55:47 UTC Ended at: 2016-01-16 15:58:21 UTC State: stopped Result: success Params: repository 'rhel_repo'; product 'rhel6'; organization 'Default Organization' Parent task 100.0% Complete 100% Output: No new packages. As per my understanding, it should take less time since no new packages to sync Thanks and Regards, Tazim No new packages. Spoke to the developers, got to the conclusion that this much time is expected to download the metadata Id: c287bf96-9010-4229-852d-925ddbaea7a8 Label: Actions::Katello::Repository::Sync Name: Synchronize Owner: admin Started at: 2016-01-16 15:55:47 UTC Ended at: 2016-01-16 15:58:21 UTC State: stopped Result: success Params: repository 'rhel_repo'; product 'rhel6'; organization 'Default Organization' Parent task 100.0% Complete Issue resolved for preprocess. Post process it takes same amount of time still Other Bz: https://bugzilla.redhat.com/show_bug.cgi?id=1279490 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:0052 The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. |
The repomd.xml file has its own timestamp in the XML, and then each file it references has its own timestamp. The treeinfo file contains its own timestamp, which already gets checked for freshness. We should check the repomd.xml timestamp and skip the rest of the sync if it hasn't changed. This will reduce the sync time for a big repo from possibly 1-2 minutes to a small number of seconds when the upstream repo hasn't changed. This will also help us be smarter about when the distribution files get re-downloaded. Deliverables: rpm sync checks the timestamp in repomd.xml, and skips all unit types but "distribution" if it hasn't changed since the last sync. tell Katello team (specifically jsherrill) about this change once it's in Pulp. They might be able to remove some code on their end once this is in place. release notes