Bug 758916 - Implement rsync delta based plugins
Summary: Implement rsync delta based plugins
Keywords:
Status: CLOSED EOL
Alias: None
Product: PulpDist
Classification: Community
Component: Pulp Plugins
Version: unspecified
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: 0.3.0
Assignee: Nick Coghlan
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-12-01 00:17 UTC by Nick Coghlan
Modified: 2016-09-29 03:04 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-29 03:04:26 UTC


Attachments (Terms of Use)

Description Nick Coghlan 2011-12-01 00:17:57 UTC
The currently implemented importer plugins invoke rsync directly against the remote site. The intent is to provide an additional importer->distributor->importer plugin chain that uses a remote Pulp server to generate the rsync delta files over the LAN, passes the delta file over the WAN, then applies the delta locally at the destination site.

1. This will initially only be supported for "snapshot" tree layouts, since the previous version of the tree is needed to determine the deltas and some marker is needed to indicate when a new version of the tree is ready for synchronisation.
2. The upstream importer invokes the rsync CLI as usual, but adds the "--only-write-batch" flag. The generated delta is then stored in Pulp as a content unit. Initially, a delta will be generated solely for the most recent version of the tree. Longer term, a config option to allow generation and storage of additional delta files to update from older versions of the tree may be added.
3. The upstream distributor then makes these content units available via HTTP.
4. The downstream importer queries the upstream distributor, reporting its latest local version. The upstream distributor then either replies with a link to the appropriate delta file (to be downloaded and applied), or details of the upstream repo for a direct rsync call (i.e. the client is sufficiently out of date that the required delta file has already been purged).

Comment 1 Nick Coghlan 2016-09-29 03:04:26 UTC
Marking all remaining PulpDist issues as CLOSED-EOL

It doesn't make sense to leave these issues open, as PulpDist hasn't been in active development for quite some time, and restarting development would involve a significant modernisation effort that would likely render many of these requests irrelevant.


Note You need to log in before you can comment on or make changes to this bug.