+++ This bug was initially created as a clone of Bug #1734299 +++ Description of problem: Ctime heals the ctime xattr ("trusted.glusterfs.mdata") in lookup if it's not present. In a multi client scenario, there is a race which results in updating the ctime xattr to older value. e.g. Let c1 and c2 be two clients and file1 be the file which doesn't have the ctime xattr. Let the ctime of file1 be t1. (from backend, ctime heals time attributes from backend when not present). Now following operations are done on mount c1 -> ls -l /mnt1/file1 | c2 -> ls -l /mnt2/file1;echo "append" >> /mnt2/file1; The race is that the both c1 and c2 didn't fetch the ctime xattr in lookup, so both of them tries to heal ctime to time 't1'. If c2 wins the race and appends the file before c1 heals it, it sets the time to 't1' and updates it to 't2' (because of append). Now c1 proceeds to heal and sets it to 't1' which is incorrect. Version-Release number of selected component (if applicable): mainline How reproducible: Always Steps to Reproduce: 1. Create single brick gluster volume and start it 2. Mount at /mnt1 and /mnt2 3. Disable ctime gluster volume set <volname> ctime off 4. Create a file touch /mnt/file1 5. Enable ctime gluster volume set <volname> ctime on 6. Put a breakpoint at gf_utime_set_mdata_lookup_cbk on '/mnt1' 7. ls -l /mnt1/file1 This hits the break point, allow for root gfid and don't continue on stbuf->ia_gfid equals to file1's gfid 8. ls -l /mnt2/file1 9. The ctime xattr is healed from /mnt2. Capture it. getfattr -d -m . -e hex /<brickpath>/file1 | grep mdata 10. echo "append" >> /mnt2/file1 and capture mdata getfattr -d -m . -e hex /<brickpath>/file1 | grep mdata 11. Continue the break point at step 7 and capture the mdata Actual results: mdata xattr at step 11 is equal to step 9 (Went back in time) Expected results: mdata xattr at step 11 should be equal to step 10 Additional info: --- Additional comment from Worker Ant on 2019-07-30 08:14:18 UTC --- REVIEW: https://review.gluster.org/23131 (posix/ctime: Fix race during lookup ctime xattr heal) posted (#1) for review on master by Kotresh HR --- Additional comment from Worker Ant on 2019-08-01 02:59:49 UTC --- REVIEW: https://review.gluster.org/23131 (posix/ctime: Fix race during lookup ctime xattr heal) merged (#2) on master by Amar Tumballi
REVIEW: https://review.gluster.org/23163 (posix/ctime: Fix race during lookup ctime xattr heal) posted (#1) for review on release-6 by Kotresh HR
REVIEW: https://review.gluster.org/23163 (posix/ctime: Fix race during lookup ctime xattr heal) merged (#2) on release-6 by hari gowtham