Bug 525545

Summary:	RFE: better rsync sparse file support.
Product:	[Fedora] Fedora	Reporter:	Eric Sandeen <esandeen>
Component:	rsync	Assignee:	Pavel Šimerda (pavlix) <psimerda>
Status:	CLOSED UPSTREAM	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	low
Version:	rawhide	CC:	matt, mluscon, mzember, ssorce
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-09-30 15:02:11 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	827429

Description Eric Sandeen 2009-09-24 18:52:19 UTC

I know about the -S option, but to my understanding this still does an awful lot of 0-memory-handling; it simply seeks over 0s on the receiving end.

# touch sparsefile
# /root/truncate sparsefile 8589934592
# time /tmp/rsync -S sparsefile sparsefile2
real	0m59.651s
user	1m13.718s
sys	0m34.412s
# ls -lh sparsefile2
-rw-r--r-- 1 root root 8.0G Sep 24 14:03 sparsefile2
# du -h sparsefile2
20K	sparsefile2

so 1 minute to transfer 20k (hm, why not 0k, but anyway)

Would it be possible to do the 0-detection on the sender, and essentially just send seeks to the receiver?  Would skipping checksums for large swaths of 0s speed things up?  Or even, use fiemap to obtain file mapping and only read/transfer instantiated block ranges?

Not sure if this fits into how rsync is designed, but it'd be a very nice feature if possible.  This would probably have a nice usecase for moving guest vm images around ...

Comment 1 Matt McCutchen 2009-11-21 06:04:01 UTC

This is covered by rsync bug (don't link) #5801 except for the fiemap part.

Comment 2 Eric Sandeen 2009-11-21 15:06:15 UTC

Matt, thanks.  I really think this would be hugely useful for transferring filesystem images for xample (whether for analysis, vm shuffling, etc...)

-Eric

Comment 3 Fedora Admin XMLRPC Client 2010-10-05 11:59:49 UTC

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 4 Fedora Admin XMLRPC Client 2012-05-07 09:31:54 UTC

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 5 Matt McCutchen 2012-05-08 04:18:27 UTC

The fiemap part is now filed upstream: https://bugzilla.samba.org/show_bug.cgi?id=8918

Comment 6 Fedora Admin XMLRPC Client 2014-09-30 12:14:02 UTC

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 8 Eric Sandeen 2014-09-30 15:02:11 UTC

5 years later it's pretty clear that asking Fedora to fix this was the wrong way to go.  ;)  No need to have a bug cluttering things up if it's never going to get touched.

Comment 9 Martin Žember 2014-09-30 15:30:29 UTC

What would be a feasible way to proceed?

Comment 10 Eric Sandeen 2014-09-30 18:52:25 UTC

I suppose I should bring it up upstream...

Comment 11 Michal Luscon 2014-10-01 08:08:57 UTC

There are already filled upstream bugzillas for better sparse file support - 
https://bugzilla.samba.org/show_bug.cgi?id=8918
https://bugzilla.samba.org/show_bug.cgi?id=5801
https://bugzilla.samba.org/show_bug.cgi?id=7854