Bug 858275

Summary:	Gluster volume status doesn't show disconnected peers
Product:	[Community] GlusterFS	Reporter:	Filip Pytloun <filip.pytloun>
Component:	cli	Assignee:	Kaushal <kaushal>
Status:	CLOSED DEFERRED	QA Contact:
Severity:	medium	Docs Contact:
Priority:	medium
Version:	3.3.0	CC:	bugs, dave, gluster-bugs, joe, mailbox, pierre.francois
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-12-14 19:40:29 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Filip Pytloun 2012-09-18 14:13:59 UTC

Description of problem:
Gluster volume status shows only connected peers, not disconnected ones.

Steps to Reproduce:
1. create gluster of multiple nodes
2. shutdown glusterd on one of those nodes
3. run gluster volume status on online node
  
Actual results:
No disconnected/offline nodes

Expected results:
I would expect display of offline nodes and their online status as false

Comment 1 Joe Julian 2012-09-18 14:33:29 UTC

Shutting down glusterd does not shut down glusterfsd process(es). This leaves the brick(s) up and, thus, online. This is by design.

Comment 2 mailbox 2012-10-10 10:19:49 UTC

Even shutting down glusterfsd, I wasn't able to see an "Online = N" node as
output of "gluster volume status": the line corresponding to the node was
just missing from the list.

Comment 3 Pierre-Francois Laquerre 2012-12-10 16:23:30 UTC

Same issue here on 3.3.1. "gluster volume status" can mislead you into thinking that everything is fine (all bricks are up!), when you are actually missing a server or three. Below is an example from this morning.

"gluster volume info" correctly shows my 40 bricks:

Volume Name: bigdata
Type: Distributed-Replicate
Volume ID: 56498956-7b4b-4ee3-9d2b-4c8cfce26051
Status: Started
Number of Bricks: 20 x 2 = 40
Transport-type: tcp
Bricks:
Brick1: ml43:/mnt/donottouch/localb
Brick2: ml44:/mnt/donottouch/localb
Brick3: ml43:/mnt/donottouch/localc
Brick4: ml44:/mnt/donottouch/localc
Brick5: ml45:/mnt/donottouch/localb
Brick6: ml46:/mnt/donottouch/localb
Brick7: ml45:/mnt/donottouch/localc
Brick8: ml46:/mnt/donottouch/localc
Brick9: ml47:/mnt/donottouch/localb
Brick10: ml48:/mnt/donottouch/localb
Brick11: ml47:/mnt/donottouch/localc
Brick12: ml48:/mnt/donottouch/localc
Brick13: ml45:/mnt/donottouch/locald
Brick14: ml46:/mnt/donottouch/locald
Brick15: ml47:/mnt/donottouch/locald
Brick16: ml48:/mnt/donottouch/locald
Brick17: ml51:/mnt/donottouch/localb
Brick18: ml52:/mnt/donottouch/localb
Brick19: ml51:/mnt/donottouch/localc
Brick20: ml52:/mnt/donottouch/localc
Brick21: ml51:/mnt/donottouch/locald
Brick22: ml52:/mnt/donottouch/locald
Brick23: ml53:/mnt/donottouch/locald
Brick24: ml54:/mnt/donottouch/locald
Brick25: ml53:/mnt/donottouch/localc
Brick26: ml54:/mnt/donottouch/localc
Brick27: ml53:/mnt/donottouch/localb
Brick28: ml54:/mnt/donottouch/localb
Brick29: ml55:/mnt/donottouch/localb
Brick30: ml29:/mnt/donottouch/localb
Brick31: ml55:/mnt/donottouch/localc
Brick32: ml29:/mnt/donottouch/localc
Brick33: ml30:/mnt/donottouch/localc
Brick34: ml31:/mnt/donottouch/localc
Brick35: ml30:/mnt/donottouch/localb
Brick36: ml31:/mnt/donottouch/localb
Brick37: ml40:/mnt/donottouch/localb
Brick38: ml41:/mnt/donottouch/localb
Brick39: ml40:/mnt/donottouch/localc
Brick40: ml41:/mnt/donottouch/localc
Options Reconfigured:
nfs.register-with-portmap: OFF
nfs.disable: on
performance.quick-read: on

"gluster peer status" shows that two of my servers are offline:
Number of Peers: 15

Hostname: ml52
Uuid: 4de42f67-4cca-4d28-8600-9018172563ba
State: Peer in Cluster (Connected)

Hostname: ml41
Uuid: b404851f-dfd5-4746-a3bd-81bb0d888009
State: Peer in Cluster (Connected)

Hostname: ml46
Uuid: af74d39b-09d6-47ba-9c3b-72d993dca4ce
State: Peer in Cluster (Connected)

Hostname: ml54
Uuid: c55580fa-2c9d-493d-b9d1-3bce016c8b29
State: Peer in Cluster (Connected)

Hostname: ml51
Uuid: 5491b6dc-0f96-43d9-95d9-a41018a8542c
State: Peer Rejected (Connected)

Hostname: ml48
Uuid: efd79145-bfd9-4eea-b7a7-50be18d9ffe0
State: Peer in Cluster (Connected)

Hostname: ml43
Uuid: a9044e9a-39e1-4907-8921-43da870b7f31
State: Peer in Cluster (Connected)

Hostname: ml45
Uuid: 0eebbceb-8f62-4c55-8160-41348f90e191
State: Peer in Cluster (Connected)

Hostname: ml47
Uuid: e831092d-b196-46ec-947d-a5635e8fbd1e
State: Peer in Cluster (Connected)

Hostname: ml30
Uuid: e56b4c57-a058-4464-a1e6-c4676ebf00cc
State: Peer in Cluster (Connected)

Hostname: ml40
Uuid: ffcc06ae-100a-4fa2-888e-803a41ae946c
State: Peer in Cluster (Connected)

Hostname: ml55
Uuid: 366339ed-52e5-4722-a1b3-e3bb1c49ea4f
State: Peer in Cluster (Disconnected)

Hostname: ml31
Uuid: 699019f6-2f4a-45cb-bfa4-f209745f8a6d
State: Peer in Cluster (Disconnected)

Hostname: ml29
Uuid: 58aa8a16-5d2b-4c06-8f06-2fd0f7fc5a37
State: Peer in Cluster (Connected)

Hostname: ml53
Uuid: 1dc6ee08-c606-4755-8756-b553f66efa88
State: Peer in Cluster (Connected)


but "gluster volume status" doesn't have anything about ml31 and ml55:
Status of volume: bigdata
Gluster process                                         Port    Online  Pid
------------------------------------------------------------------------------
Brick ml43:/mnt/donottouch/localb                       24014   Y       9366
Brick ml44:/mnt/donottouch/localb                       24016   Y       22097
Brick ml43:/mnt/donottouch/localc                       24015   Y       9371
Brick ml44:/mnt/donottouch/localc                       24015   Y       22102
Brick ml45:/mnt/donottouch/localb                       24015   Y       10122
Brick ml46:/mnt/donottouch/localb                       24015   Y       7911
Brick ml45:/mnt/donottouch/localc                       24016   Y       10127
Brick ml46:/mnt/donottouch/localc                       24016   Y       7916
Brick ml47:/mnt/donottouch/localb                       24015   Y       9636
Brick ml48:/mnt/donottouch/localb                       24015   Y       10414
Brick ml47:/mnt/donottouch/localc                       24016   Y       9641
Brick ml48:/mnt/donottouch/localc                       24016   Y       10419
Brick ml45:/mnt/donottouch/locald                       24017   Y       10132
Brick ml46:/mnt/donottouch/locald                       24017   Y       7921
Brick ml47:/mnt/donottouch/locald                       24017   Y       9646
Brick ml48:/mnt/donottouch/locald                       24017   Y       10424
Brick ml52:/mnt/donottouch/localb                       24015   Y       17587
Brick ml52:/mnt/donottouch/localc                       24016   Y       17592
Brick ml52:/mnt/donottouch/locald                       24017   Y       17597
Brick ml53:/mnt/donottouch/locald                       24017   Y       20125
Brick ml54:/mnt/donottouch/locald                       24017   Y       18658
Brick ml53:/mnt/donottouch/localc                       24016   Y       20130
Brick ml54:/mnt/donottouch/localc                       24016   Y       18663
Brick ml53:/mnt/donottouch/localb                       24015   Y       20135
Brick ml54:/mnt/donottouch/localb                       24015   Y       18668
Brick ml29:/mnt/donottouch/localb                       24014   Y       30882
Brick ml29:/mnt/donottouch/localc                       24015   Y       30887
Brick ml30:/mnt/donottouch/localc                       24015   Y       30411
Brick ml30:/mnt/donottouch/localb                       24014   Y       30416
Brick ml40:/mnt/donottouch/localb                       24014   Y       8465
Brick ml41:/mnt/donottouch/localb                       24014   Y       7889
Brick ml40:/mnt/donottouch/localc                       24015   Y       8470
Brick ml41:/mnt/donottouch/localc                       24015   Y       7894
Self-heal Daemon on localhost                           N/A     Y       22115
Self-heal Daemon on ml30                                N/A     Y       30429
Self-heal Daemon on ml29                                N/A     Y       30900
Self-heal Daemon on ml48                                N/A     Y       10438
Self-heal Daemon on ml54                                N/A     Y       14959
Self-heal Daemon on ml52                                N/A     Y       17611
Self-heal Daemon on ml45                                N/A     Y       10146
Self-heal Daemon on ml41                                N/A     Y       7907
Self-heal Daemon on ml40                                N/A     Y       8483
Self-heal Daemon on ml43                                N/A     Y       9384
Self-heal Daemon on ml46                                N/A     Y       7935
Self-heal Daemon on ml53                                N/A     Y       20149
Self-heal Daemon on ml47                                N/A     Y       9660

I would have expected a "Brick ml55:/mnt/donottouch/... N/A N N/A" for ml55 and ml31's bricks.

Comment 4 Niels de Vos 2014-11-27 14:53:53 UTC

The version that this bug has been reported against, does not get any updates from the Gluster Community anymore. Please verify if this report is still valid against a current (3.4, 3.5 or 3.6) release and update the version, or close this bug.

If there has been no update before 9 December 2014, this bug will get automatocally closed.