Bug 763810 (GLUSTER-2078)

Summary: Volume Migration is not working
Product: [Community] GlusterFS Reporter: Timothy Asir <timothyasir>
Component: glusterdAssignee: Vijay Bellur <vbellur>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: urgent    
Version: 3.1.1CC: gluster-bugs, rabhat, vijay
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: RTP Mount Type: ---
Documentation: DNR CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Timothy Asir 2010-11-10 09:05:33 EST
Volume migration and volume migration status are not working.

Reproduce:

I followed the below steps:

[root@aserver1 ~]# mkdir /tmp/a1
[root@aserver1 ~]# mkdir /tmp/a2
[root@aserver1 ~]# mkdir /tmp/aa
[root@aserver1 ~]# gluster volume create aa aserver1:/tmp/a1
Creation of volume aa has been successful
[root@aserver1 ~]# gluster volume start aa
Starting volume aa has been successful
[root@aserver1 ~]# mount -t nfs -o nolock aserver1:aa /mnt/
[root@aserver1 ~]# touch /mnt/{1..100}
[root@aserver1 ~]# umount /mnt/
[root@aserver1 ~]# gluster volume replace-brick aa aserver1:/tmp/a1 aserver1:/tmp/a2 start
replace-brick started successfully
[root@aserver1 ~]# gluster volume replace-brick aa aserver1:/tmp/a1 aserver1:/tmp/a2 status
Number of files migrated = 0       Current file=
[root@aserver1 ~]# echo $?
0
[root@aserver1 ~]# ls /tmp/a2
[root@aserver1 ~]# ls /tmp/a1/
1    12  16  2   23  27  30  34  38  41  45  49  52  56  6   63  67  70  74  78  81  85  89  92  96
10   13  17  20  24  28  31  35  39  42  46  5   53  57  60  64  68  71  75  79  82  86  9   93  97
100  14  18  21  25  29  32  36  4   43  47  50  54  58  61  65  69  72  76  8   83  87  90  94  98
11   15  19  22  26  3   33  37  40  44  48  51  55  59  62  66  7   73  77  80  84  88  91  95  99

[root@aserver1 ~]# glusterfs --version
glusterfs 3.1.1qa4 built on Nov  2 2010 23:15:22
Repository revision: v3.1.0-70-g8ca9673
Copyright (c) 2006-2010 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU Affero General Public License.
Comment 1 Vijay Bellur 2010-11-10 22:40:46 EST
root@deepthought:~/workspace/sep10/glusterfs.git# mkdir /tmp/a{1..3}
root@deepthought:~/workspace/sep10/glusterfs.git# glusterd
root@deepthought:~/workspace/sep10/glusterfs.git# gluster volume create aa deepthought:/tmp/a1
Creation of volume aa has been successful
root@deepthought:~/workspace/sep10/glusterfs.git# gluster volume start aa
Starting volume aa has been successful
root@deepthought:~/workspace/sep10/glusterfs.git# mount -t nfs -o nolock deepthought:/aa /mnt/client1
root@deepthought:~/workspace/sep10/glusterfs.git# touch /mnt/client1/{1..100}
root@deepthought:~/workspace/sep10/glusterfs.git# umount /mnt/client1 
root@deepthought:~/workspace/sep10/glusterfs.git# gluster volume replace-brick aa deepthought:/tmp/a1 deepthought:/tmp/a2
Usage: volume replace-brick <VOLNAME> <BRICK> <NEW-BRICK> start|pause|abort|commit|status
Replacing brick from Volume aa failed
root@deepthought:~/workspace/sep10/glusterfs.git# gluster volume replace-brick aa deepthought:/tmp/a1 deepthought:/tmp/a2 start
replace-brick started successfully
root@deepthought:~/workspace/sep10/glusterfs.git# ls /tmp/a2
1    11  14  17  2   22  25  28  30  33  36  39  41  44  47  5   52  55  58  60  63  66  69  71  74  77  8   82  85  88  90  93  96  99
10   12  15  18  20  23  26  29  31  34  37  4   42  45  48  50  53  56  59  61  64  67  7   72  75  78  80  83  86  89  91  94  97
100  13  16  19  21  24  27  3   32  35  38  40  43  46  49  51  54  57  6   62  65  68  70  73  76  79  81  84  87  9   92  95  98
root@deepthought:~/workspace/sep10/glusterfs.git# ls /tmp/a1
1    11  14  17  2   22  25  28  30  33  36  39  41  44  47  5   52  55  58  60  63  66  69  71  74  77  8   82  85  88  90  93  96  99
10   12  15  18  20  23  26  29  31  34  37  4   42  45  48  50  53  56  59  61  64  67  7   72  75  78  80  83  86  89  91  94  97
100  13  16  19  21  24  27  3   32  35  38  40  43  46  49  51  54  57  6   62  65  68  70  73  76  79  81  84  87  9   92  95  98
root@deepthought:~/workspace/sep10/glusterfs.git# gluster volume replace-brick aa deepthought:/tmp/a1 deepthought:/tmp/a2 status
Number of files migrated = 100        Migration complete 
root@deepthought:~/workspace/sep10/glusterfs.git# gluster volume replace-brick aa deepthought:/tmp/a1 deepthought:/tmp/a2 commit
replace-brick commit successful
root@deepthought:~/workspace/sep10/glusterfs.git# gluster volume info

Volume Name: repl
Type: Replicate
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: deepthought:/data/export8
Brick2: deepthought:/data/export9

Volume Name: aa
Type: Distribute
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: deepthought:/tmp/a2
root@deepthought:~/workspace/sep10/glusterfs.git#
Comment 2 Timothy Asir 2010-11-11 22:11:59 EST
Unfortunately migration is not working in platform setup.
Please test with platform iso image.
Comment 3 Timothy Asir 2010-11-16 03:16:22 EST
Behavior is appearing in "glusterfs 3.1.1qa7" also.
Comment 4 Anand Avati 2010-11-16 06:35:08 EST
PATCH: http://patches.gluster.com/patch/5719 in master (rpc-clnt: move rpc_clnt_reconnect() call to rpc_clnt_init)
Comment 5 Anand Avati 2010-11-16 06:35:13 EST
PATCH: http://patches.gluster.com/patch/5720 in master (rpc-clnt: introduce new way of creating rpc-clnt objects)
Comment 6 Anand Avati 2010-11-16 06:35:17 EST
PATCH: http://patches.gluster.com/patch/5721 in master (protocol/client: use the new rpc_clnt initialization methods)
Comment 7 Timothy Asir 2010-11-17 07:01:00 EST
Behavior is appearing in "glusterfs 3.1.1qa8" also.

I followed the below steps in platform server:

Creation of volume test has been successful
[root@t1 ~]# gluster volume start test
Starting volume test has been successful
[root@t1 ~]# mount -t nfs -o nolock t1:/test /tmp/test
[root@t1 ~]# touch /tmp/test/file{1..10}
[root@t1 ~]# umount /tmp/test
[root@t1 ~]# ls /tmp/test1 
file1  file10  file2  file3  file4  file5  file6  file7  file8  file9
[root@t1 ~]# gluster volume replace-brick test t1:/tmp/test1 t1:/tmp/test2 start
replace-brick started successfully
[root@t1 ~]# gluster volume replace-brick test t1:/tmp/test1 t1:/tmp/test2 status
replace-brick status unknown
Replacing brick from Volume test failed
[root@t1 ~]# ls /tmp/test2 
[root@t1 ~]# gluster volume replace-brick test t1:/tmp/test1 t1:/tmp/test2 status
replace-brick status unknown
Replacing brick from Volume test failed
[root@t1 ~]# glusterd --version
glusterfs 3.1.1qa8 built on Nov 17 2010 06:06:26
Repository revision: v3.1.0-131-gce9f328
Copyright (c) 2006-2010 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU Affero General Public License.

[root@t1 ~]# uname -mrsn
Linux t1 2.6.30.10-105.2.4.fc11.x86_64 x86_64
[root@t1 ~]# cat /proc/version 
Linux version 2.6.30.10-105.2.4.fc11.x86_64 (mockbuild@x86-06.phx2.fedoraproject.org) (gcc version 4.4.1 20090725 (Red Hat 4.4.1-2) (GCC) ) #1 SMP Tue Jan 19 22:46:59 UTC 2010
Comment 8 Raghavendra Bhat 2011-02-20 21:49:37 EST
chacked with glusterfs-3.1.1, glusterfs-3.1.2 and latest git (26cedae57d5b7cb8d50ed077ce29c92e30d6e260). Migration is working fine.