Bug 758914

Summary: Progress reporting for rsync based importers
Product: [Community] PulpDist Reporter: Nick Coghlan <ncoghlan>
Component: Pulp PluginsAssignee: Nick Coghlan <ncoghlan>
Status: CLOSED EOL QA Contact:
Severity: unspecified Docs Contact:
Priority: medium    
Version: unspecified   
Target Milestone: 0.2.0   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-29 03:04:24 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 758936    

Description Nick Coghlan 2011-11-30 23:56:50 UTC 
The PulpDist importer plugins need to be updated to:

- monitor the progress of the underlying rsync call
- report that progress to Pulp

While the plugins are still based on the rsync CLI, this can likely be achieved by switching away from the use of the subprocess convenience APIs and invoking subprocess.Popen directly.

1. Wait for the "x files found" line in the rsync output.
2. Monitor the file counts emitted by rsync and use them to track progress against the original file count.

For multi-tree importers, hazard an initial guess based on the number of files in the first tree multiplied by the number of trees. As trees are updated, update the counts accordingly.

Always add 1% (rounding up) to the overall estimate to allow for the tidying up at the end (i.e. make sure to only report 100% progress when we're actually done)
Comment 1 Nick Coghlan 2016-09-29 03:04:24 UTC 
Marking all remaining PulpDist issues as CLOSED-EOL

It doesn't make sense to leave these issues open, as PulpDist hasn't been in active development for quite some time, and restarting development would involve a significant modernisation effort that would likely render many of these requests irrelevant.