Bug 1760713 - Remove-brick : linkto file is missing for certain files from hash subvol and hence cannot access those files from client node.
Summary: Remove-brick : linkto file is missing for certain files from hash subvol and ...
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: distribute
Version: rhgs-3.5
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: ---
Assignee: Susant Kumar Palai
QA Contact: Sayalee
Depends On:
Blocks: 1696815
TreeView+ depends on / blocked
Reported: 2019-10-11 07:29 UTC by Sayalee
Modified: 2020-01-16 08:57 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
When the cluster.lookup-optimize option is enabled, running a rename operation after a rebalance can lead to some files being inaccessible from the client. Workaround: Disable cluster.lookup-optimize on the volume. # gluster volume set VOLNAME cluster.lookup-optimize off
Clone Of:
Last Closed: 2020-01-16 07:46:46 UTC
Target Upstream Version:

Attachments (Terms of Use)

Description Sayalee 2019-10-11 07:29:59 UTC
Description of problem:
Performed remove brick on a volume, with rename of files and remove of files running in parallel as IO, there are remove-brick failures due to ENOENTs. 
Also, the linkto-file of some files go missing from the backend bricks and so those files become inaccessible from the client node.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Create a 5X3 volume. 
2. The volume is mounted at :
   mnt1 --> RHEL 7.7 mahcine, FUSE mount
   mnt2 --> RHEL 8.1 machine, GNFS mount
3. On the client node, created 5 dirs:
   dir1 : contains 500 block files
   dir2 : contains 500 device file 
   dir3 : contained an untar of linux kernel, started removing all the files and directories under dir3 using rm-rf
   dir4 - contains 500 pipe files
   dir5 - created 100 files, started renaming 100 files such that each file will be renamed 2000 times.
4. While the rename of files and rm -rf was still in progress, started remove-brick operation on the volume to   convert it into 4X3.
5. Once the remove-brick migration was completed, committed the remove-brick process.

Actual results:
* Remove-brick migration had failures due to ENOENTs.
* The number of block, device and pipe files was same, before and after remove-brick.
* No issues w.r.t. rm -rf of files under dir3.
* All files under dir5 were renamed, but some files became inaccessible. For these files, the linkto-file was missing from the hash subvol.

Expected results:
* All files should be accessible from the client node.

Additional info:
sos-reports will be shared.

Note You need to log in before you can comment on or make changes to this bug.