Description of problem: ======================= Attaching another node to a cluster which has tiered volumes times out. Version-Release number of selected component (if applicable): ============================================================= glusterfs 3.7.0beta1 built on May 1 2015 How reproducible: ================ 3/3 Steps to Reproduce: ================== 1.Create a 4 node cluster 2.Create a tiered volume. Fuse and NFS mount the volume and create some IO 3.Attach another node to the cluster gluster peer probe snapshot11.lab.eng.blr.redhat.com Error : Request timed out -----Part of glusterd log from snapshot11.lab.eng.blr.redhat.com------------ [2015-05-04 12:42:45.846735] E [glusterd-svc-mgmt.c:29:glusterd_svc_create_rundir] 0-management: Unable to create rundir /var/lib/glusterd/vols/gluster_shared_storage/run [2015-05-04 12:42:45.846797] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600 [2015-05-04 12:42:45.849728] E [glusterd-utils.c:4273:glusterd_brick_start] 0-management: Could not find peer on which brick rhs-arch-srv2.lab.eng.blr.redhat.com:/rhs/brick8/b8 resides [2015-05-04 12:42:45.849774] E [glusterd-op-sm.c:1840:glusterd_start_bricks] 0-management: Failed to start rhs-arch-srv2.lab.eng.blr.redhat.com:/rhs/brick8/b8 for gluster_shared_storage [2015-05-04 12:42:46.259992] E [glusterd-svc-mgmt.c:29:glusterd_svc_create_rundir] 0-management: Unable to create rundir /var/lib/glusterd/vols/vol0/run [2015-05-04 12:42:46.260124] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600 [2015-05-04 12:42:46.262278] E [glusterd-utils.c:4273:glusterd_brick_start] 0-management: Could not find peer on which brick rhs-arch-srv4.lab.eng.blr.redhat.com:/rhs/brick4/b4 resides [2015-05-04 12:42:46.262363] E [glusterd-op-sm.c:1840:glusterd_start_bricks] 0-management: Failed to start rhs-arch-srv4.lab.eng.blr.redhat.com:/rhs/brick4/b4 for vol0 [2015-05-04 12:42:46.262588] E [glusterd-volgen.c:959:volgen_write_volfile] 0-management: No such file or directory [2015-05-04 12:42:46.262616] E [glusterd-volgen.c:1003:volgen_write_volfile] 0-management: failed to create volfile /var/lib/glusterd/vols/vol0/vol0-snapd.vol [2015-05-04 12:42:46.262638] E [glusterd-snapd-svc.c:244:glusterd_snapdsvc_start] 0-management: Couldn't create snapd volfile for volume: vol0 [2015-05-04 12:42:47.493470] E [glusterd-volgen.c:4976:glusterd_create_volfiles] 0-management: Could not generate trusted client volfiles [2015-05-04 12:42:47.493497] E [glusterd-sm.c:1224:glusterd_friend_sm] 0-glusterd: handler returned: -1 -------------------------------------------------------------------- [root@snapshot11 ~]# gluster peer status Number of Peers: 1 Hostname: 10.70.34.50 Uuid: 1c1b2314-8bf5-4b64-9b58-7a5fd1eca21c State: Connected to Peer (Connected) [root@inception ~]# gluster peer status Number of Peers: 4 Hostname: rhs-arch-srv2.lab.eng.blr.redhat.com Uuid: 54cda762-d199-4ee4-be57-50a0b3313bcf State: Peer in Cluster (Connected) Hostname: rhs-arch-srv3.lab.eng.blr.redhat.com Uuid: 419a4609-e9e8-4cc6-b1c5-58a5efaf3ecf State: Peer in Cluster (Connected) Hostname: rhs-arch-srv4.lab.eng.blr.redhat.com Uuid: 961b6d3a-9534-4d28-a4c7-9c1d3296cd60 State: Peer in Cluster (Connected) Hostname: snapshot11.lab.eng.blr.redhat.com Uuid: 429914f4-486d-4eb7-ac27-593335d76592 State: Probe Sent to Peer (Connected) Actual results: Expected results: Additional info:
Scaling up the cluster is not successful when tiered volumes are present. Proposing this bug as a blocker
Fixed as part of the following changes, http://review.gluster.org/#/c/10339/ http://review.gluster.org/#/c/10449/
http://review.gluster.org/#/c/10339/ merged http://review.gluster.org/#/c/10449/ needs review
3.7 bug 1219846, patch review.gluster.org/10449
This bug was ON_QA status, and on GlusterFS product in bugzilla, we don't have that as a valid status. We are closing it as 'CURRENT RELEASE ' to indicate the availability of the fix, please reopen if found again.