| Summary: | dist-geo-rep : hardlink sync through xsync crawl resulted in duplicate files in the same directory. | ||
|---|---|---|---|
| Product: | Red Hat Gluster Storage | Reporter: | Vijaykumar Koppad <vkoppad> |
| Component: | geo-replication | Assignee: | Bug Updates Notification Mailing List <rhs-bugs> |
| Status: | CLOSED EOL | QA Contact: | storage-qa-internal <storage-qa-internal> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 2.1 | CC: | avishwan, chrisw, csaba, david.macdonald, khiremat |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-11-25 08:48:56 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Description
Vijaykumar Koppad
2013-12-02 12:45:39 UTC
This I was able to hit it again with the same build. But this time is didn't happen in /mnt/slave/level00/hardlink_to_files/ but it happened in /mnt/slave/level00/ which is usual.
The script I ran,
for i in {1..20}
do
./crefi.py -n 100 --multi -b 10 -d 10 --random --max=2K --min=1K /mnt/master/
sleep 100
./crefi.py -n 100 --multi -b 10 -d 10 --random --max=2K --min=1K --fop=chmod /mnt/master/
sleep 100
./crefi.py -n 100 --multi -b 10 -d 10 --random --max=2K --min=1K --fop=chown /mnt/master/
sleep 100
./crefi.py -n 100 --multi -b 10 -d 10 --random --max=2K --min=1K --fop=chgrp /mnt/master/
sleep 100
./crefi.py -n 100 --multi -b 10 -d 10 --random --max=2K --min=1K --fop=symlink /mnt/master/
sleep 100
./crefi.py -n 100 --multi -b 10 -d 10 --random --max=2K --min=1K --fop=hardlink /mnt/master/
sleep 100
./crefi.py -n 100 --multi -b 10 -d 10 --random --max=2K --min=1K --fop=rename /mnt/master/
sleep 500
done
which would incrimentaly create files and do operations on all of the files at that moment. This took some 24 hours to complete, in between there were many disconnections, which resulted in xsync crawls. This might have triggered hardlink syncing through hybrid crawl. But if that was the case the duplicate should have been in hardlinks_to_files directories, it happened in /mnt/slave/level00/. Since it is the issue with duplicate files, I have updated it here.
Vijay, If a hardlink created to a file falls into the subvolume other than on which file resides, dht linkto file gets created on the hashed subvolume and hardlink is created on the cached subvolume. Since the geo-rep daemon runs per brick, the xync changelog is created for the linkto file and also the original hardlink file. Both gets processed by the sync daemon resulting in race to form duplicate files of same name. The patch of ignoring dht linkto files will solve this problem from geo-rep side. https://code.engineering.redhat.com/gerrit/17336 This will fix the issue from geo-rep side, but it seems there is a larger issue of race outside geo-rep where it is allowing to process the file creation of same name in same location. -Kotresh H R This has happened again on the build glusterfs-3.4.0.58rhs-1. As per the Comment 3, link-to files are ignored. and with the following patch, Hardlink is handled as link in XSync crawl. http://review.gluster.org/#/c/5934/ This bug can be closed once verified as below, 1. Create data and Hardlinks in Master Volume. 2. Geo-replication Create and Start. 3. All Regular files and Hardlinks checksums should match. Closing this bug since RHGS 2.1 release reached EOL. Required bugs are cloned to RHGS 3.1. Please re-open this issue if found again. Closing this bug since RHGS 2.1 release reached EOL. Required bugs are cloned to RHGS 3.1. Please re-open this issue if found again. |