Description of problem:
-----------------------
There hasn't been a significant performance improvement after the snapdiff feature implementation, as the time taken to sync a snapshot after modifying file permissions is still around 9 minutes.


Version-Release number of selected component (if applicable):
-------------------------------------------------------------
ceph version 19.1.0-55.el9cp (eebe02d2c1876620ecf21f31cdbc828dc99ba7a3) squid (rc)


Steps to Reproduce:
-------------------

Created 100 depth directories using Crefi. -

[root@magna029 a97d0061-0ea2-4f00-94d3-f36d62d53c59]# for i in {tar,binary,text,sparse} ; do python3 crefi.py --multi -b 100 -d 100 -n 100 --random --min=1K --max=2K -t $i /mnt/subvolc1_1/ ; done


[root@magna028 26171d20-f1d5-11ee-b6c2-002590fc2a2e]# ceph --admin-daemon ceph-client.cephfs-mirror.magna028.kxaogg.2.93988612716904.asok fs mirror peer status cephfs1@3 131bcf4a-9824-4982-90a7-5bffb3c6724b
{
    "/volumes/_nogroup/subvolc1_1": {
        "state": "idle",
        "last_synced_snap": {
            "id": 20,
            "name": "snap_fs1_sv_1_initial",
            "sync_duration": 23141655,
            "sync_time_stamp": "471172.556049s",
            "sync_bytes": 6612302693
        },
        "snaps_synced": 1,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    },
}

Initial Sync: Took 385.6 to sync to remote cluster

After completing the initial sync, I modified the file permissions of a file located several directory levels deep.

Permission Change : 
—-------------------
[root@magna029 ~ ]# cd
/mnt/subvc1/volumes/_nogroup/subvolc1_1/a97d0061-0ea2-4f00-94d3-f36d62d53c59/level099/level199/level299/level399/level499/level599/level699/level799/level899/level999/level1099/level1199/level1299/level1399

[root@magna029 level1399]# pwd
/mnt/subvc1/volumes/_nogroup/subvolc1_1/a97d0061-0ea2-4f00-94d3-f36d62d53c59/level099/level199/level299/level399/level499/level599/level699/level799/level899/level999/level1099/level1199/level1299/level1399

[root@magna029 level1399]# ls -l 66cf2352%%MTH264TOM1
-rw-r--r--. 1 root root 1673 Aug 28 13:17 66cf2352%%MTH264TOM1

[root@magna029 level1399]# chmod 777 66cf2352%%MTH264TOM1

[root@magna029 level1399]# ls -l 66cf2352%%MTH264TOM1
-rwxrwxrwx. 1 root root 1673 Aug 28 13:17 66cf2352%%MTH264TOM1

[root@magna029 level1399]# ceph fs subvolume snapshot create cephfs1 subvolc1_1 snap_fs1_sv_1_perm_change


[root@magna028 26171d20-f1d5-11ee-b6c2-002590fc2a2e]# ceph --admin-daemon ceph-client.cephfs-mirror.magna028.kxaogg.2.93988612716904.asok fs mirror peer status cephfs1@3 131bcf4a-9824-4982-90a7-5bffb3c6724b
{
    "/volumes/_nogroup/subvolc1_1": {
        "state": "idle",
        "last_synced_snap": {
            "id": 21,
            "name": "snap_fs1_sv_1_perm_change",
            "sync_duration": 547876,
            "sync_time_stamp": "585921.106877s",
            "sync_bytes": 0
        },
        "snaps_synced": 2,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    },
}

The Sync is completed after a few minutes and the sync duration after a file modification is around 547876 [9.13126667 Mins]

I expected a quicker sync for an incremental snapshot with just one file permission modification, but 9 minutes seems excessive.