Bug 758914 - Progress reporting for rsync based importers
Summary: Progress reporting for rsync based importers
Keywords:
Status: CLOSED EOL
Alias: None
Product: PulpDist
Classification: Community
Component: Pulp Plugins
Version: unspecified
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: 0.2.0
Assignee: Nick Coghlan
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 758936
TreeView+ depends on / blocked
 
Reported: 2011-11-30 23:56 UTC by Nick Coghlan
Modified: 2016-09-29 03:04 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-29 03:04:24 UTC


Attachments (Terms of Use)

Description Nick Coghlan 2011-11-30 23:56:50 UTC
The PulpDist importer plugins need to be updated to:

- monitor the progress of the underlying rsync call
- report that progress to Pulp

While the plugins are still based on the rsync CLI, this can likely be achieved by switching away from the use of the subprocess convenience APIs and invoking subprocess.Popen directly.

1. Wait for the "x files found" line in the rsync output.
2. Monitor the file counts emitted by rsync and use them to track progress against the original file count.

For multi-tree importers, hazard an initial guess based on the number of files in the first tree multiplied by the number of trees. As trees are updated, update the counts accordingly.

Always add 1% (rounding up) to the overall estimate to allow for the tidying up at the end (i.e. make sure to only report 100% progress when we're actually done)

Comment 1 Nick Coghlan 2016-09-29 03:04:24 UTC
Marking all remaining PulpDist issues as CLOSED-EOL

It doesn't make sense to leave these issues open, as PulpDist hasn't been in active development for quite some time, and restarting development would involve a significant modernisation effort that would likely render many of these requests irrelevant.


Note You need to log in before you can comment on or make changes to this bug.