Bug 1144088
| Summary: | dist-gep-rep: Files on master and slave are not in sync after file renames on master volume. | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | shilpa <smanjara> |
| Component: | geo-replication | Assignee: | Saravanakumar <sarumuga> |
| Status: | CLOSED ERRATA | QA Contact: | Rahul Hinduja <rhinduja> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | rhgs-3.0 | CC: | aavati, annair, avishwan, csaba, nlevinki, rhinduja, vbhat |
| Target Milestone: | --- | ||
| Target Release: | RHGS 3.1.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | rename, consistency | ||
| Fixed In Version: | glusterfs-3.7.0-2.el6rhs | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-07-29 04:36:21 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1202842, 1223636 | ||
Multiple Renames ---------------- If renamed files fall into other bricks, processing changelogs across brick nodes is not sequential. Ordering of execution is not guaranteed when changelogs processed parallel. For example, CREATE f1 RENAME f1 f2 RENAME f2 f1 f1 falls in brick1 and f2 falls in brick2, changelogs are Brick1 Changelog will have, CREATE f1 RENAME f2 f1 Brick2 Changelog will have, RENAME f1 f2 If Brick 1 changelogs executed first and then Brick 2, Slave will have f2 instead of f1. Verified with build: glusterfs-3.7.1-7.el6rhs.x86_64
Master Client1:
===============
[root@wingo master]# for i in {1..100};do touch mb$i;done
[root@wingo master]#
Master Client2:
===============
[root@client master]# for i in {1..100};do mv mb$i nc$i;done
[root@client master]# for i in {1..100};do mv mb$i nc$i;done
[root@client master]# for i in {1..100};do mv mb$i nc$i;done
[root@client master]# for i in {1..100};do mv mb$i nc$i;done
[root@client master]# for i in {1..100};do mv mb$i nc$i;done
[root@client master]# for i in {1..100};do mv mb-1$i nc$i;done
[root@client master]# for i in {1..100};do mv mb-1$i nc-1$i;done
[root@client master]# for i in {1..100};do mv mb-2$i nc-2$i;done
[root@client master]#
Master Client3:
===============
[root@dhcp35-63 master]# for i in {1..100};do mv nc$i mb$i;done
[root@dhcp35-63 master]# for i in {1..100};do mv nc$i mb$i;done
[root@dhcp35-63 master]# for i in {1..100};do mv nc$i mb$i;done
[root@dhcp35-63 master]# for i in {1..100};do mv nc$i mb-1$i;done
[root@dhcp35-63 master]# for i in {1..100};do mv nc$i mb-1$i;done
[root@dhcp35-63 master]# for i in {1..100};do mv nc-1$i mb-2$i;done
[root@dhcp35-63 master]# for i in {1..100};do mv nc-2$i mb-3$i;done
[root@dhcp35-63 master]#
After the files are synced:
============================
[root@wingo master]# ls
mb-31 mb-318 mb-327 mb-336 mb-345 mb-354 mb-363 mb-372 mb-381 mb-390
mb-310 mb-319 mb-328 mb-337 mb-346 mb-355 mb-364 mb-373 mb-382 mb-391
mb-3100 mb-32 mb-329 mb-338 mb-347 mb-356 mb-365 mb-374 mb-383 mb-392
mb-311 mb-320 mb-33 mb-339 mb-348 mb-357 mb-366 mb-375 mb-384 mb-393
mb-312 mb-321 mb-330 mb-34 mb-349 mb-358 mb-367 mb-376 mb-385 mb-394
mb-313 mb-322 mb-331 mb-340 mb-35 mb-359 mb-368 mb-377 mb-386 mb-395
mb-314 mb-323 mb-332 mb-341 mb-350 mb-36 mb-369 mb-378 mb-387 mb-396
mb-315 mb-324 mb-333 mb-342 mb-351 mb-360 mb-37 mb-379 mb-388 mb-397
mb-316 mb-325 mb-334 mb-343 mb-352 mb-361 mb-370 mb-38 mb-389 mb-398
mb-317 mb-326 mb-335 mb-344 mb-353 mb-362 mb-371 mb-380 mb-39 mb-399
[root@wingo master]#
At Slave:
=========
[root@wingo slave]# ls
mb-31 mb-318 mb-327 mb-336 mb-345 mb-354 mb-363 mb-372 mb-381 mb-390
mb-310 mb-319 mb-328 mb-337 mb-346 mb-355 mb-364 mb-373 mb-382 mb-391
mb-3100 mb-32 mb-329 mb-338 mb-347 mb-356 mb-365 mb-374 mb-383 mb-392
mb-311 mb-320 mb-33 mb-339 mb-348 mb-357 mb-366 mb-375 mb-384 mb-393
mb-312 mb-321 mb-330 mb-34 mb-349 mb-358 mb-367 mb-376 mb-385 mb-394
mb-313 mb-322 mb-331 mb-340 mb-35 mb-359 mb-368 mb-377 mb-386 mb-395
mb-314 mb-323 mb-332 mb-341 mb-350 mb-36 mb-369 mb-378 mb-387 mb-396
mb-315 mb-324 mb-333 mb-342 mb-351 mb-360 mb-37 mb-379 mb-388 mb-397
mb-316 mb-325 mb-334 mb-343 mb-352 mb-361 mb-370 mb-38 mb-389 mb-398
mb-317 mb-326 mb-335 mb-344 mb-353 mb-362 mb-371 mb-380 mb-39 mb-399
[root@wingo slave]#
Moving the bug to verified state.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-1495.html |
Description of problem: Renamed files from multiple clients. Did multiple renames from the clients and found that the file names are not consistent on master and slave volumes. Version-Release number of selected component (if applicable): glusterfs-3.6.0.28-1.el6rhs.x86_64 How reproducible: Tried twice Steps to Reproduce: 1.Create a distribute replicate master and slave volumes 2.Create and start geo-rep session 3.Mount the master volume on three different clients. 3.Write data to master volume from one client. for i in {1..100};do touch mb$i;done 4.Execute "for i in {1..100};do mv mb$i nc$i;done" from one client and "for i in {1..100};do mv nc$i mb$i;done" from another client. 5.Do step #4 multiple times in a loop. Tried 3 times. 6.Check the geo-rep session to see if all the files are synced. 7.Once the files are synced, mount the slave volume and check the files inside and compare them with master volume Actual results: Some of the files that were renamed did not propogate to the slave. It still contained the original name. So the file names are inconsistent on master and slave nodes Expected results: The renames should be consistent on both master and slave Additional info: 1.[root@paul master]# for i in {1..100};do touch mb$i;done 2.[root@rick client]# for i in {1..100};do mv mb$i nc$i;done 3.[root@pat client1]# for i in {1..100};do mv nc$i mb$i;done 4.[root@rick client]# for i in {1..100};do mv mb$i nc$i;done 5.[root@pat client1]# for i in {1..100};do mv nc$i mb$i;done Repeat #3 and #4 a couple of times. Wait for the files to sync. [root@paul master]# ls /mnt/slave mb100-1 mb18 mb29 mb39 mb51 mb63 mb73 mb81 mb88 mb95 nc14 nc28 nc40 nc5 nc59 nc71 nc92 mb11 mb21 mb30 mb4 mb56 mb66 mb75 mb83 mb89 nc1 nc19 nc3 nc41 nc50 nc60 nc72 nc94 mb1-1 mb23 mb31 mb42 mb57 mb67 mb76 mb84 mb9 nc10 nc2 nc33 nc45 nc52 nc61 nc74 nc96 mb15 mb24 mb32 mb43 mb58 mb69 mb77 mb85 mb90 nc100 nc20 nc34 nc46 nc53 nc64 nc79 nc97 mb16 mb26 mb35 mb44 mb6 mb7 mb78 mb86 mb91 nc12 nc22 nc36 nc47 nc54 nc65 nc80 nc98 mb17 mb27 mb37 mb49 mb62 mb70 mb8 mb87 mb93 nc13 nc25 nc38 nc48 nc55 nc68 nc82 nc99 [root@paul master]# ls /mnt/master mb1 mb12 mb18 mb23 mb29 mb34 mb4 mb45 mb50 mb56 mb61 mb67 mb72 mb78 mb83 mb89 mb94 mb10 mb13 mb19 mb24 mb3 mb35 mb40 mb46 mb51 mb57 mb62 mb68 mb73 mb79 mb84 mb9 mb95 mb100 mb14 mb2 mb25 mb30 mb36 mb41 mb47 mb52 mb58 mb63 mb69 mb74 mb8 mb85 mb90 mb96 mb100-1 mb15 mb20 mb26 mb31 mb37 mb42 mb48 mb53 mb59 mb64 mb7 mb75 mb80 mb86 mb91 mb97 mb11 mb16 mb21 mb27 mb32 mb38 mb43 mb49 mb54 mb6 mb65 mb70 mb76 mb81 mb87 mb92 mb98 mb1-1 mb17 mb22 mb28 mb33 mb39 mb44 mb5 mb55 mb60 mb66 mb71 mb77 mb82 mb88 mb93 nc99 Taking mb1 for example to point out the difference: [root@paul master]# ls /mnt/master | grep mb1 mb1 mb10 mb100 mb100-1 mb11 mb1-1 mb12 mb13 mb14 mb15 mb16 mb17 mb18 mb19 [root@paul master]# ls /mnt/slave | grep mb1 mb100-1 mb11 mb1-1 mb15 mb16 mb17 mb18 [root@paul master]# ls /mnt/slave | grep nc10 nc10 nc100 [root@paul master]# ls /mnt/master | grep nc10 [root@paul master]# [root@paul geo-automation]# ls /mnt/slave | grep nc1 nc1 nc10 nc100 nc12 nc13 nc14 nc19 [root@paul geo-automation]# ls /mnt/master | grep nc1 [root@paul geo-automation]# Some files have been renamed and some have not been.