Bug 1259750

Summary: DHT: Few files are missing after remove-brick operation
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: RajeshReddy <rmekala>
Component: distributeAssignee: Susant Kumar Palai <spalai>
Status: CLOSED ERRATA QA Contact: RajeshReddy <rmekala>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rhgs-3.1CC: byarlaga, mzywusko, rcyriac, spalai, vagarwal
Target Milestone: ---Keywords: ZStream
Target Release: RHGS 3.1.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.7.1-15 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1260051 (view as bug list) Environment:
Last Closed: 2015-10-05 07:25:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1260051, 1262197    

Description RajeshReddy 2015-09-03 13:23:28 UTC 
Description of problem:
=====================
Few files are missing after removal of brick, even those files are not exist in the decommissioned brick and this is data loss 

Version-Release number of selected component (if applicable):
===============
glusterfs-server-3.7.1-12

How reproducible:


Steps to Reproduce:
===================
1. Create a distributed volume with three bricks, and mount it on client using FUSE and create 2k files
2. Make sure one of the brick is at least 90% full (fill it from the back end)  
3. Delete all files from the mount point
4. Create 2k files with the same name and it result creation of few files with -----T attribute on the 90% full brick 
5. Create hard links to the -----T files from the mount point 
6. Remove the brick which holds both data and link files 

Actual results:
==============
Remove brick operation succeeded but deleted data file 

Expected Result:
============
All files should be migrated properly from the decommissioned brick to remaining bricks 

Info
=======
[root@rhs-client39 data]# gluster vol status dht10
Status of volume: dht10
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick rhs-client9.lab.eng.blr.redhat.com:/r
hs/brick1/dht10                             49180     0          Y       957  
Brick rhs-client39.lab.eng.blr.redhat.com:/
rhs/brick10/dht10                           49170     0          Y       5902 
Brick rhs-client39.lab.eng.blr.redhat.com:/
rhs/brick3/dht10                            49171     0          Y       5920 
NFS Server on localhost                     2049      0          Y       5939 
NFS Server on rhs-client9.lab.eng.blr.redha
t.com                                       2049      0          Y       1183 
 
Task Status of Volume dht10
------------------------------------------------------------------------------
Task                 : Remove brick        
ID                   : 0dcc1704-0a0f-4eb0-ad15-de7235271da1
Removed bricks:     
rhs-client39.lab.eng.blr.redhat.com:/rhs/brick3/dht10
Status               : completed
Comment 3 Susant Kumar Palai 2015-09-11 06:45:49 UTC 
downstream patch: https://code.engineering.redhat.com/gerrit/#/c/57566/
Comment 4 Susant Kumar Palai 2015-09-11 12:14:27 UTC 
We are targeting one more patch for this bug. 
https://code.engineering.redhat.com/gerrit/#/c/57617/

Hence, moving this back to POST.
Comment 6 Susant Kumar Palai 2015-09-14 06:20:43 UTC 
downstream patches: https://code.engineering.redhat.com/gerrit/#/c/57617/
                    https://code.engineering.redhat.com/gerrit/#/c/57566/
Comment 7 RajeshReddy 2015-09-16 10:43:55 UTC 
Tested with build glusterfs-3.7.1-15, and ran fallowing tests and all tests passed so marking this bug as verified 

1. Removed the brick which has both data and Hard link files 
2. Removed the birck which has hard link files 
3. Removed the brick which has data files
Comment 9 errata-xmlrpc 2015-10-05 07:25:38 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1845.html