Description of problem: I can't add another peer to a cluster after a restart, when the "Number of Bricks" shown by "gluster volume info" is messed up. ------------------------------------------------------- Version-Release number of selected component (if applicable): 3.3.2-2 ------------------------------------------------------- How reproducible: every time ------------------------------------------------------- Steps to Reproduce: I don't know if there is an easier way to encounter this error. That's just what i've done. 1. Get a clean installation of five equal configured servers. My servers (N = 1-5): ---- serverN> cat /etc/system-release CentOS release 6.4 (Final) ---- serverN> uname -r 2.6.32-358.14.1.el6.x86_64 ---- serverN> yum list installed | grep gluster glusterfs.x86_64 3.3.2-2.el6 @glusterfs-3-3-epel glusterfs-fuse.x86_64 3.3.2-2.el6 @glusterfs-3-3-epel glusterfs-server.x86_64 3.3.2-2.el6 @glusterfs-3-3-epel ---- Gluster isn't configured yet, so let's probe all server. ---- server3> gluster peer probe server1 Probe successful server3> gluster peer probe server2 Probe successful server3> gluster peer probe server4 Probe successful server1> gluster peer probe server3 Probe successful 2. Setup a replica two set on two of them. Setup a volume with a replica two set, using server3 as the main configuration server. ---- server3> gluster volume create glusterFS replica 2 server3:/mnt/gluster/glusterFS server4:/mnt/gluster/glusterFS Creation of volume glusterFS has been successful. Please start the volume to access data. ---- Start the volume ---- server3> gluster volume start glusterFS Starting volume glusterFS has been successful 3. Expand the now existing volume with the replica two set to a replica four set. Let's expand the volume to four servers. We do this on our new main configuration server - server1. ---- server1> gluster volume add-brick glusterFS replica 4 server1:/mnt/gluster/glusterFS server2:/mnt/gluster/glusterFS Add Brick successful 4. Now let's shrink this volume back to a replica two set with only two servers, server1 and server2 remaining. Let's start the shrinking process. ---- server1> gluster volume remove-brick glusterFS replica 2 server3:/mnt/gluster/glusterFS server4:/mnt/gluster/glusterFS start Remove Brick start successful ---- Commit if ready. ---- server1> gluster volume remove-brick glusterFS replica 2 server3:/mnt/gluster/glusterFS server4:/mnt/gluster/glusterFS commit Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y Remove Brick commit successful ---- 5. Let's see what "gluster volume info" shows ---- server1> gluster volume info Volume Name: glusterFS Type: Replicate Volume ID: 71b83e52-b5ee-41ee-9012-141d35da76b1 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: server1:/mnt/gluster/glusterFS Brick2: server2:/mnt/gluster/glusterFS ---- This seems fine. Until we restart gluster. Let's do that. ---- server1> /etc/init.d/glusterd restart server2> /etc/init.d/glusterd restart server3> /etc/init.d/glusterd restart server4> /etc/init.d/glusterd restart server5> /etc/init.d/glusterd restart ---- server1> gluster volume info Volume Name: glusterFS Type: Replicate Volume ID: 71b83e52-b5ee-41ee-9012-141d35da76b1 Status: Started Number of Bricks: 0 x 4 = 2 Transport-type: tcp Bricks: Brick1: server1:/mnt/gluster/glusterFS Brick2: server2:/mnt/gluster/glusterFS ---- And there it goes. "Number of Bricks: 0 x 4 = 2" 6. Let's try to probe server5 as a new server to the cluster. ---- server1> gluster peer probe server5 ---- This takes a minute or so and doesn't show any result in the console. ---- server1> gluster peer status Number of Peers: 4 Hostname: server2 Uuid: 0e697f01-96c8-4933-803b-1b65fb7180aa State: Peer in Cluster (Connected) Hostname: server3 Uuid: b282aad7-348c-47b6-922b-8bb255802a7a State: Peer in Cluster (Connected) Hostname: server4 Uuid: 6bf4e23c-24b5-4cb5-bc85-6bdfd9d6a390 State: Peer in Cluster (Connected) Hostname: server5 Uuid: 9bc0593f-8214-46c9-b913-107b8303ee45 State: Probe Sent to Peer (Connected) ---- "State: Probe Sent to Peer (Connected)" on server5. That's all and it doesn't change. ---- server5 > gluster peer status Number of Peers: 1 Hostname: server1 Uuid: 0ad8d07e-f8d5-4ce0-992c-56a6c3e0b13d State: Connected to Peer (Connected) ---- Even a restart doesn't change a thing. ------------------------------------------------------- Actual results: "Number of Bricks: 0 x 4 = 2" Unable to add a server to the cluster. ------------------------------------------------------- Expected results: "Number of Bricks: 1 x 2 = 2" Being able to add a server to the cluster. ------------------------------------------------------- Additional info: none -------------------------------------------------------
Any progress on this?
Try the latest QA release if your playing with gluster as its much better than the 3.2 or 3.3 release. http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.4.1qa1/
The version that this bug has been reported against, does not get any updates from the Gluster Community anymore. Please verify if this report is still valid against a current (3.4, 3.5 or 3.6) release and update the version, or close this bug. If there has been no update before 9 December 2014, this bug will get automatocally closed.