Bug 2317735 - RFE: extend snapdiff API to enable only transferring changed portions of a file
Summary: RFE: extend snapdiff API to enable only transferring changed portions of a file
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: CephFS
Version: 8.1
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 8.1
Assignee: Venky Shankar
QA Contact: Hemanth Kumar
Rivka Pollack
URL:
Whiteboard:
Depends On: 2308992
Blocks: 2351689
TreeView+ depends on / blocked
 
Reported: 2024-10-10 02:49 UTC by Greg Farnum
Modified: 2025-06-03 19:22 UTC (History)
8 users (show)

Fixed In Version: ceph-19.2.1-75.el9cp
Doc Type: Enhancement
Doc Text:
.`cephfs-mirror` daemon only transfers changed blocks in a file Previously, `cephfs-mirror` daemon would transfer whole files, which is inefficient for large files. With this enhancement, the `cephfs-mirror` daemon uses the blockdiff API in the MDS to only transfer changed blocks in a file. As a result, sync performance is significantly improved in some circumstances, especially for large files.
Clone Of:
Environment:
Last Closed:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 69791 0 None None None 2025-02-04 17:08:05 UTC
Red Hat Issue Tracker RHCEPH-9985 0 None None None 2024-10-10 02:51:10 UTC

Description Greg Farnum 2024-10-10 02:49:56 UTC
libcephfs exposes a "snapdiff" API that lets programmatic users query for the list of files changed between two snapshots. This identifies the changed files, but does not identify the changed regions within those files.

RADOS exposes primitives that let you identify the regions which have changed between two snapshots. We should extend the CephFS snapdiff API to additionally let users query for the file diff, by querying the underlying RADOS objects. This should dramatically reduce the amount of data read and transmitted when trying to maintain consistent copies over time.

Comment 1 Storage PM bot 2024-10-10 02:50:07 UTC
Please specify the severity of this bug. Severity is defined here:
https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.


Note You need to log in before you can comment on or make changes to this bug.