Bug 858275 - Gluster volume status doesn't show disconnected peers
Gluster volume status doesn't show disconnected peers
Product: GlusterFS
Classification: Community
Component: cli (Show other bugs)
Unspecified Linux
medium Severity medium
: ---
: ---
Assigned To: Kaushal
Depends On:
  Show dependency treegraph
Reported: 2012-09-18 10:13 EDT by Filip Pytloun
Modified: 2014-12-14 14:40 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-12-14 14:40:29 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Filip Pytloun 2012-09-18 10:13:59 EDT
Description of problem:
Gluster volume status shows only connected peers, not disconnected ones.

Steps to Reproduce:
1. create gluster of multiple nodes
2. shutdown glusterd on one of those nodes
3. run gluster volume status on online node
Actual results:
No disconnected/offline nodes

Expected results:
I would expect display of offline nodes and their online status as false
Comment 1 Joe Julian 2012-09-18 10:33:29 EDT
Shutting down glusterd does not shut down glusterfsd process(es). This leaves the brick(s) up and, thus, online. This is by design.
Comment 2 mailbox 2012-10-10 06:19:49 EDT
Even shutting down glusterfsd, I wasn't able to see an "Online = N" node as
output of "gluster volume status": the line corresponding to the node was
just missing from the list.
Comment 3 Pierre-Francois Laquerre 2012-12-10 11:23:30 EST
Same issue here on 3.3.1. "gluster volume status" can mislead you into thinking that everything is fine (all bricks are up!), when you are actually missing a server or three. Below is an example from this morning.

"gluster volume info" correctly shows my 40 bricks:

Volume Name: bigdata
Type: Distributed-Replicate
Volume ID: 56498956-7b4b-4ee3-9d2b-4c8cfce26051
Status: Started
Number of Bricks: 20 x 2 = 40
Transport-type: tcp
Brick1: ml43:/mnt/donottouch/localb
Brick2: ml44:/mnt/donottouch/localb
Brick3: ml43:/mnt/donottouch/localc
Brick4: ml44:/mnt/donottouch/localc
Brick5: ml45:/mnt/donottouch/localb
Brick6: ml46:/mnt/donottouch/localb
Brick7: ml45:/mnt/donottouch/localc
Brick8: ml46:/mnt/donottouch/localc
Brick9: ml47:/mnt/donottouch/localb
Brick10: ml48:/mnt/donottouch/localb
Brick11: ml47:/mnt/donottouch/localc
Brick12: ml48:/mnt/donottouch/localc
Brick13: ml45:/mnt/donottouch/locald
Brick14: ml46:/mnt/donottouch/locald
Brick15: ml47:/mnt/donottouch/locald
Brick16: ml48:/mnt/donottouch/locald
Brick17: ml51:/mnt/donottouch/localb
Brick18: ml52:/mnt/donottouch/localb
Brick19: ml51:/mnt/donottouch/localc
Brick20: ml52:/mnt/donottouch/localc
Brick21: ml51:/mnt/donottouch/locald
Brick22: ml52:/mnt/donottouch/locald
Brick23: ml53:/mnt/donottouch/locald
Brick24: ml54:/mnt/donottouch/locald
Brick25: ml53:/mnt/donottouch/localc
Brick26: ml54:/mnt/donottouch/localc
Brick27: ml53:/mnt/donottouch/localb
Brick28: ml54:/mnt/donottouch/localb
Brick29: ml55:/mnt/donottouch/localb
Brick30: ml29:/mnt/donottouch/localb
Brick31: ml55:/mnt/donottouch/localc
Brick32: ml29:/mnt/donottouch/localc
Brick33: ml30:/mnt/donottouch/localc
Brick34: ml31:/mnt/donottouch/localc
Brick35: ml30:/mnt/donottouch/localb
Brick36: ml31:/mnt/donottouch/localb
Brick37: ml40:/mnt/donottouch/localb
Brick38: ml41:/mnt/donottouch/localb
Brick39: ml40:/mnt/donottouch/localc
Brick40: ml41:/mnt/donottouch/localc
Options Reconfigured:
nfs.register-with-portmap: OFF
nfs.disable: on
performance.quick-read: on

"gluster peer status" shows that two of my servers are offline:
Number of Peers: 15

Hostname: ml52
Uuid: 4de42f67-4cca-4d28-8600-9018172563ba
State: Peer in Cluster (Connected)

Hostname: ml41
Uuid: b404851f-dfd5-4746-a3bd-81bb0d888009
State: Peer in Cluster (Connected)

Hostname: ml46
Uuid: af74d39b-09d6-47ba-9c3b-72d993dca4ce
State: Peer in Cluster (Connected)

Hostname: ml54
Uuid: c55580fa-2c9d-493d-b9d1-3bce016c8b29
State: Peer in Cluster (Connected)

Hostname: ml51
Uuid: 5491b6dc-0f96-43d9-95d9-a41018a8542c
State: Peer Rejected (Connected)

Hostname: ml48
Uuid: efd79145-bfd9-4eea-b7a7-50be18d9ffe0
State: Peer in Cluster (Connected)

Hostname: ml43
Uuid: a9044e9a-39e1-4907-8921-43da870b7f31
State: Peer in Cluster (Connected)

Hostname: ml45
Uuid: 0eebbceb-8f62-4c55-8160-41348f90e191
State: Peer in Cluster (Connected)

Hostname: ml47
Uuid: e831092d-b196-46ec-947d-a5635e8fbd1e
State: Peer in Cluster (Connected)

Hostname: ml30
Uuid: e56b4c57-a058-4464-a1e6-c4676ebf00cc
State: Peer in Cluster (Connected)

Hostname: ml40
Uuid: ffcc06ae-100a-4fa2-888e-803a41ae946c
State: Peer in Cluster (Connected)

Hostname: ml55
Uuid: 366339ed-52e5-4722-a1b3-e3bb1c49ea4f
State: Peer in Cluster (Disconnected)

Hostname: ml31
Uuid: 699019f6-2f4a-45cb-bfa4-f209745f8a6d
State: Peer in Cluster (Disconnected)

Hostname: ml29
Uuid: 58aa8a16-5d2b-4c06-8f06-2fd0f7fc5a37
State: Peer in Cluster (Connected)

Hostname: ml53
Uuid: 1dc6ee08-c606-4755-8756-b553f66efa88
State: Peer in Cluster (Connected)

but "gluster volume status" doesn't have anything about ml31 and ml55:
Status of volume: bigdata
Gluster process                                         Port    Online  Pid
Brick ml43:/mnt/donottouch/localb                       24014   Y       9366
Brick ml44:/mnt/donottouch/localb                       24016   Y       22097
Brick ml43:/mnt/donottouch/localc                       24015   Y       9371
Brick ml44:/mnt/donottouch/localc                       24015   Y       22102
Brick ml45:/mnt/donottouch/localb                       24015   Y       10122
Brick ml46:/mnt/donottouch/localb                       24015   Y       7911
Brick ml45:/mnt/donottouch/localc                       24016   Y       10127
Brick ml46:/mnt/donottouch/localc                       24016   Y       7916
Brick ml47:/mnt/donottouch/localb                       24015   Y       9636
Brick ml48:/mnt/donottouch/localb                       24015   Y       10414
Brick ml47:/mnt/donottouch/localc                       24016   Y       9641
Brick ml48:/mnt/donottouch/localc                       24016   Y       10419
Brick ml45:/mnt/donottouch/locald                       24017   Y       10132
Brick ml46:/mnt/donottouch/locald                       24017   Y       7921
Brick ml47:/mnt/donottouch/locald                       24017   Y       9646
Brick ml48:/mnt/donottouch/locald                       24017   Y       10424
Brick ml52:/mnt/donottouch/localb                       24015   Y       17587
Brick ml52:/mnt/donottouch/localc                       24016   Y       17592
Brick ml52:/mnt/donottouch/locald                       24017   Y       17597
Brick ml53:/mnt/donottouch/locald                       24017   Y       20125
Brick ml54:/mnt/donottouch/locald                       24017   Y       18658
Brick ml53:/mnt/donottouch/localc                       24016   Y       20130
Brick ml54:/mnt/donottouch/localc                       24016   Y       18663
Brick ml53:/mnt/donottouch/localb                       24015   Y       20135
Brick ml54:/mnt/donottouch/localb                       24015   Y       18668
Brick ml29:/mnt/donottouch/localb                       24014   Y       30882
Brick ml29:/mnt/donottouch/localc                       24015   Y       30887
Brick ml30:/mnt/donottouch/localc                       24015   Y       30411
Brick ml30:/mnt/donottouch/localb                       24014   Y       30416
Brick ml40:/mnt/donottouch/localb                       24014   Y       8465
Brick ml41:/mnt/donottouch/localb                       24014   Y       7889
Brick ml40:/mnt/donottouch/localc                       24015   Y       8470
Brick ml41:/mnt/donottouch/localc                       24015   Y       7894
Self-heal Daemon on localhost                           N/A     Y       22115
Self-heal Daemon on ml30                                N/A     Y       30429
Self-heal Daemon on ml29                                N/A     Y       30900
Self-heal Daemon on ml48                                N/A     Y       10438
Self-heal Daemon on ml54                                N/A     Y       14959
Self-heal Daemon on ml52                                N/A     Y       17611
Self-heal Daemon on ml45                                N/A     Y       10146
Self-heal Daemon on ml41                                N/A     Y       7907
Self-heal Daemon on ml40                                N/A     Y       8483
Self-heal Daemon on ml43                                N/A     Y       9384
Self-heal Daemon on ml46                                N/A     Y       7935
Self-heal Daemon on ml53                                N/A     Y       20149
Self-heal Daemon on ml47                                N/A     Y       9660

I would have expected a "Brick ml55:/mnt/donottouch/... N/A N N/A" for ml55 and ml31's bricks.
Comment 4 Niels de Vos 2014-11-27 09:53:53 EST
The version that this bug has been reported against, does not get any updates from the Gluster Community anymore. Please verify if this report is still valid against a current (3.4, 3.5 or 3.6) release and update the version, or close this bug.

If there has been no update before 9 December 2014, this bug will get automatocally closed.

Note You need to log in before you can comment on or make changes to this bug.