Description of problem: On Tiered volume, gluster volume status shows error "Commit failed failed on localhost" glusterd logs ========================================= [2015-06-22 09:05:53.710783] I [glusterd-handler.c:4205:__glusterd_handle_status_volume] 0-management: Received status volume req for volume vol0 [2015-06-22 09:05:53.721641] E [MSGID: 106062] [glusterd-op-sm.c:2816:_add_remove_bricks_to_dict] 0-management: Unable to get brick3 [2015-06-22 09:05:53.721691] E [MSGID: 106397] [glusterd-op-sm.c:2863:_add_task_to_dict] 0-management: Failed to add remove bricks to dict [2015-06-22 09:05:53.721715] E [MSGID: 106060] [glusterd-op-sm.c:2934:glusterd_aggregate_task_status] 0-management: Failed to add task details to dict [2015-06-22 09:05:53.721740] E [MSGID: 106123] [glusterd-syncop.c:1413:gd_commit_op_phase] 0-management: Commit of operation 'Volume Status' failed on localhost [2015-06-22 09:05:53.721740] E [MSGID: 106123] [glusterd-syncop.c:1413:gd_commit_op_phase] 0-management: Commit of operation 'Volume Status' failed on localhost Version-Release number of selected component (if applicable): How reproducible: 1/1 Steps to Reproduce: 1. Create 2*2 distribute-replicate volume 2. add 1*2 replica tier 3. set vol0 cluster.tier-demote-frequency 30 4. set vol0 features.ctr-enabled 1 5. Enable uss 6. detach tier 7 Didn't commit 8 Create snapshot 9 run gluster v status <volname> Actual results: gluster v status command fails with error "Commit failed on localhost" glusterd logs: ======================================== [2015-06-22 09:05:53.710783] I [glusterd-handler.c:4205:__glusterd_handle_status_volume] 0-management: Received status volume req for volume vol0 [2015-06-22 09:05:53.721641] E [MSGID: 106062] [glusterd-op-sm.c:2816:_add_remove_bricks_to_dict] 0-management: Unable to get brick3 [2015-06-22 09:05:53.721691] E [MSGID: 106397] [glusterd-op-sm.c:2863:_add_task_to_dict] 0-management: Failed to add remove bricks to dict [2015-06-22 09:05:53.721715] E [MSGID: 106060] [glusterd-op-sm.c:2934:glusterd_aggregate_task_status] 0-management: Failed to add task details to dict [2015-06-22 09:05:53.721740] E [MSGID: 106123] [glusterd-syncop.c:1413:gd_commit_op_phase] 0-management: Commit of operation 'Volume Status' failed on localhost [2015-06-22 09:05:53.721740] E [MSGID: 106123] [glusterd-syncop.c:1413:gd_commit_op_phase] 0-management: Commit of operation 'Volume Status' failed on localhost Expected results: Additional info: [root@darkknight ~]# gluster v info vol0 Volume Name: vol0 Type: Tier Volume ID: b9b586fa-4cf7-49e4-aace-cd09a805281f Status: Started Number of Bricks: 6 Transport-type: tcp Hot Tier : Hot Tier Type : Replicate Number of Bricks: 1 x 2 = 2 Brick1: 10.70.33.219:/rhs/brick1/SS2 Brick2: 10.70.33.214:/rhs/brick1/SS1 Cold Tier: Cold Tier Type : Distributed-Replicate Number of Bricks: 2 x 2 = 4 Brick3: 10.70.33.214:/rhs/brick1/b1 Brick4: 10.70.33.219:/rhs/brick1/b2 Brick5: 10.70.33.225:/rhs/brick1/b3 Brick6: 10.70.44.13:/rhs/brick1/b4 Options Reconfigured: cluster.entry-self-heal: off cluster.metadata-self-heal: off cluster.data-self-heal: off features.barrier: disable server.allow-insecure: on features.show-snapshot-directory: enable features.ctr-enabled: 1 cluster.tier-demote-frequency: 10 features.uss: enable
logs uploaded @ http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1234295/