Bug 1065653 - Rebalance operation is stuck on "In progress status" on a distributed volume and link files are created..
Summary: Rebalance operation is stuck on "In progress status" on a distributed volume ...
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: GlusterFS
Classification: Community
Component: distribute
Version: mainline
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Nithya Balachandran
QA Contact:
URL:
Whiteboard: dht-rebalance-file
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-15 12:30 UTC by Jiffin Tony Thottan
Modified: 2018-03-15 06:59 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2018-03-15 06:59:51 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)
sosreport (884 bytes, application/x-xz)
2014-02-15 12:37 UTC, Jiffin Tony Thottan
no flags Details
sosreport (888 bytes, application/x-xz)
2014-02-15 12:37 UTC, Jiffin Tony Thottan
no flags Details

Description Jiffin Tony Thottan 2014-02-15 12:30:06 UTC
Description of problem:

In a distributed volume when rebalance is performed on hole files, the mount point contained the link files and rebalance operation was not completed.. 



Volume Name: dishol1
Type: Distribute
Volume ID: ac82e12e-a047-410a-bde0-bd919dc9aca6
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: 192.168.0.5:/brick1/dishol1-brick1
Brick2: 192.168.0.76:/brick1/dishol1-brick2
Brick3: 192.168.0.5:/brick2/dishol1-brick3
Brick4: 192.168.0.76:/brick2/dishol1-brick4


Before rebalance:

[root@localhost dis]# du -a
216	./file3
1944	./file4
1012	./file1
300	./file2
3472	.

[root@localhost dis]# ls -l
total 3472
-rw-r--r-- 1 root root 109951163811579 Feb 15 17:22 file1
-rw-r--r-- 1 root root 109951163084713 Feb 15 17:23 file2
-rw-r--r-- 1 root root 109951162995861 Feb 15 17:23 file3
-rw-r--r-- 1 root root 109951164767288 Feb 15 17:23 file4

After rebalance:

[root@localhost dis]# ls -l
total 3472
-rw-r-Sr-T 1 root root 109951163811579 Feb 15 17:22 file1
-rw-r--r-- 1 root root 109951163084713 Feb 15 17:23 file2
-rw-r--r-- 1 root root 109951162995861 Feb 15 17:23 file3
-rw-r--r-- 1 root root 109951164767288 Feb 15 17:23 file4



Also, the extended attributes of file is shown in the mount point.

[root@localhost dis]# mount |grep mnt
192.168.0.5:/dishol1 on /mnt/dis type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
[root@localhost dis]# 


[root@localhost dis]# getfattr -d -m. file1
# file: file1
trusted.glusterfs.dht.linkto="dishol1-client-2"



[root@localhost dis]# attr -l file1
Attribute "glusterfs.dht.linkto" has a 17 byte value for file1
[root@localhost dis]# 


Version-Release number of selected component (if applicable):

glusterfs-libs-3.5.0-0.5.beta3.fc20.x86_64
glusterfs-fuse-3.5.0-0.5.beta3.fc20.x86_64
glusterfs-cli-3.5.0-0.5.beta3.fc20.x86_64
glusterfs-3.5.0-0.5.beta3.fc20.x86_64
glusterfs-devel-3.5.0-0.5.beta3.fc20.x86_64
glusterfs-server-3.5.0-0.5.beta3.fc20.x86_64


How reproducible:

N/A

Steps to Reproduce:

As mentioned above

Actual results:

The rebalance operation was never completed and stuck on 'In progress' status.

Expected results:

The rebalance should be completed..

Comment 1 Jiffin Tony Thottan 2014-02-15 12:37:01 UTC
Created attachment 863577 [details]
sosreport

Sosreport attached..

Comment 2 Jiffin Tony Thottan 2014-02-15 12:37:46 UTC
Created attachment 863578 [details]
sosreport

Comment 3 Manikandan 2016-01-05 12:23:31 UTC
Hi Jiffin,

Could you check if it is reproducible on the latest build?

Thanks,
Manikandan Selvaganesh.

Comment 4 Sakshi 2016-02-04 08:45:08 UTC
This is because rebalance first sets the xattrs on the destination file and then removes the linkto xattr. Hence there is a small window in which you will see the linkto xattr from the mount point. Also what is the size of the file with holes? Are you sure it had completely migrated?

Comment 6 Nithya Balachandran 2017-09-04 14:28:39 UTC
Jiffin,

By link file do you mean this file?

-rw-r-Sr-T 1 root root 109951163811579 Feb 15 17:22 file1

Thanks,
Nithya

Comment 7 Jiffin 2018-03-15 06:59:51 UTC
Actually I don't recollect much about this issue. Hence closing the bug as insufficient data


Note You need to log in before you can comment on or make changes to this bug.