Bug 525545 - RFE: better rsync sparse file support.
Summary: RFE: better rsync sparse file support.
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: rsync
Version: rawhide
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Pavel Šimerda (pavlix)
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 827429
TreeView+ depends on / blocked
 
Reported: 2009-09-24 18:52 UTC by Eric Sandeen
Modified: 2019-02-19 08:38 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-09-30 15:02:11 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Samba Project 5801 None None None Never

Description Eric Sandeen 2009-09-24 18:52:19 UTC
I know about the -S option, but to my understanding this still does an awful lot of 0-memory-handling; it simply seeks over 0s on the receiving end.

# touch sparsefile
# /root/truncate sparsefile 8589934592
# time /tmp/rsync -S sparsefile sparsefile2
real	0m59.651s
user	1m13.718s
sys	0m34.412s
# ls -lh sparsefile2
-rw-r--r-- 1 root root 8.0G Sep 24 14:03 sparsefile2
# du -h sparsefile2
20K	sparsefile2

so 1 minute to transfer 20k (hm, why not 0k, but anyway)

Would it be possible to do the 0-detection on the sender, and essentially just send seeks to the receiver?  Would skipping checksums for large swaths of 0s speed things up?  Or even, use fiemap to obtain file mapping and only read/transfer instantiated block ranges?

Not sure if this fits into how rsync is designed, but it'd be a very nice feature if possible.  This would probably have a nice usecase for moving guest vm images around ...

Comment 1 Matt McCutchen 2009-11-21 06:04:01 UTC
This is covered by rsync bug (don't link) #5801 except for the fiemap part.

Comment 2 Eric Sandeen 2009-11-21 15:06:15 UTC
Matt, thanks.  I really think this would be hugely useful for transferring filesystem images for xample (whether for analysis, vm shuffling, etc...)

-Eric

Comment 3 Fedora Admin XMLRPC Client 2010-10-05 11:59:49 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 4 Fedora Admin XMLRPC Client 2012-05-07 09:31:54 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 5 Matt McCutchen 2012-05-08 04:18:27 UTC
The fiemap part is now filed upstream: https://bugzilla.samba.org/show_bug.cgi?id=8918

Comment 6 Fedora Admin XMLRPC Client 2014-09-30 12:14:02 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 8 Eric Sandeen 2014-09-30 15:02:11 UTC
5 years later it's pretty clear that asking Fedora to fix this was the wrong way to go.  ;)  No need to have a bug cluttering things up if it's never going to get touched.

Comment 9 Martin Žember 2014-09-30 15:30:29 UTC
What would be a feasible way to proceed?

Comment 10 Eric Sandeen 2014-09-30 18:52:25 UTC
I suppose I should bring it up upstream...

Comment 11 Michal Luscon 2014-10-01 08:08:57 UTC
There are already filled upstream bugzillas for better sparse file support - 
https://bugzilla.samba.org/show_bug.cgi?id=8918
https://bugzilla.samba.org/show_bug.cgi?id=5801
https://bugzilla.samba.org/show_bug.cgi?id=7854


Note You need to log in before you can comment on or make changes to this bug.