1637445 – Brick process not coming up after force starting volume due to stale port map entries

Bug 1637445 - Brick process not coming up after force starting volume due to stale port map entries

Summary: Brick process not coming up after force starting volume due to stale port map...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Gluster Storage
Classification:	Red Hat Storage
Component:	glusterd
Sub Component:
Version:	rhgs-3.4
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	medium
Target Milestone:	---
Target Release:	RHGS 3.4.z Batch Update 3
Assignee:	Mohit Agrawal
QA Contact:	Upasana
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2018-10-09 09:45 UTC by Upasana
Modified:	2019-02-04 07:41 UTC (History)
CC List:	7 users (show)
Fixed In Version:	glusterfs-3.12.2-33
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2019-02-04 07:41:25 UTC
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2019:0263	0	None	None	None	2019-02-04 07:41:38 UTC

Description Upasana 2018-10-09 09:45:35 UTC

Description of problem:
==========================
Hit Bug 1637379 - "Portmap entries showing stale brick entries when bricks are down" after which if i start the volume giving force brick not coming up 

Version-Release number of selected component (if applicable):
===========================================================
glusterfs-server-3.12.2-18.1.el7rhgs.x86_64

How reproducible:
=================
1/1

Steps to Reproduce:
===================
1.After hitting Bug 1637379 force started a volume,volume start shows successful
2.Check vol status , brick not up
3.took a glusterd statedump and checked that the brick is assigned to both new and old port

Actual results:
===============
root@dhcp35-36 gluster]# gluster vol status vol_-9-1
Status of volume: vol_-9-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-9-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-9-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-9-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-9-1
------------------------------------------------------------------------------
There are no active volume tasks
 
[root@dhcp35-36 gluster]# gluster vol start vol_-9-1 force
volume start: vol_-9-1: success

[root@dhcp35-36 gluster]# gluster vol status vol_-9-1
Status of volume: vol_-9-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-9-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-9-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-9-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       24278
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       25647
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       27768
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       27912
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       31111
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       20758
 
Task Status of Volume vol_-9-1
------------------------------------------------------------------------------
There are no active volume tasks

But is assigned a new pmap brickname 
[root@dhcp35-36 gluster]# cat glusterdump.5096.dump.1539071920 |grep pmap
glusterd.pmap_port=49152
glusterd.pmap[49152].type=4
glusterd.pmap[49152].brickname=/gluster/brick1/v2 /gluster/brick2/vol3 /gluster/brick2/vol4 /gluster/brick2/vol5 /gluster/brick2/vol6 /gluster/brick1/vol_-1-1 /gluster/brick2/vol_-1-2 /gluster/brick3/vol_-1-3 /gluster/brick1/vol_-10-1 /gluster/brick2/vol_-10-2 /gluster/brick3/vol_-10-3 /gluster/brick1/vol_-2-1 /gluster/brick2/vol_-2-2 /gluster/brick3/vol_-2-3 /gluster/brick1/vol_-3-1 /gluster/brick2/vol_-3-2 /gluster/brick3/vol_-3-3 /gluster/brick1/vol_-4-1 /gluster/brick2/vol_-4-2 /gluster/brick3/vol_-4-3 /gluster/brick1/vol_-5-1 /gluster/brick2/vol_-5-2 /gluster/brick3/vol_-5-3 /gluster/brick1/vol_-6-1 /gluster/brick2/vol_-6-2 /gluster/brick3/vol_-6-3 /gluster/brick1/vol_-7-1 /gluster/brick2/vol_-7-2 /gluster/brick3/vol_-7-3 /gluster/brick1/vol_-8-1 /gluster/brick2/vol_-8-2 /gluster/brick3/vol_-8-3 /gluster/brick1/vol_-9-1 /gluster/brick2/vol_-9-2
glusterd.pmap_port=49153
glusterd.pmap[49153].type=0
glusterd.pmap[49153].brickname=(null)
glusterd.pmap_port=49154
glusterd.pmap[49154].type=4
glusterd.pmap[49154].brickname=/gluster/brick1/v21 /gluster/brick2/vol_-9-2 /gluster/brick1/vol_-9-1
[root@dhcp35-36 gluster]# 

Expected results:
=================
There should not be stale entires on the pmap port and brick should come up successfully after volume force start 


Additional info:

[root@dhcp35-36 gluster]# gluster vol info
 
Volume Name: replica-vol10
Type: Replicate
Volume ID: 3dc07b81-b26a-4600-9ea4-8a1aa3412bd8
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.36:/gluster/brick1/v2
Brick2: 10.70.35.78:/gluster/brick1/v2
Brick3: 10.70.35.192:/gluster/brick1/v2
Options Reconfigured:
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: replica-vol11
Type: Replicate
Volume ID: 03dcf4c6-9418-4a0b-89d7-d730cafe4c43
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.36:/gluster/brick1/v21
Brick2: 10.70.35.78:/gluster/brick1/v21
Brick3: 10.70.35.192:/gluster/brick1/v21
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: replica-vol3
Type: Replicate
Volume ID: dc41a775-d65b-40d7-a9d5-c30a6405d648
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.36:/gluster/brick2/vol3
Brick2: 10.70.35.78:/gluster/brick2/vol3
Brick3: 10.70.35.192:/gluster/brick2/vol3
Options Reconfigured:
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: replica-vol4
Type: Replicate
Volume ID: 2762fa7a-21df-4e94-9f83-dfd0ec96967a
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.36:/gluster/brick2/vol4
Brick2: 10.70.35.78:/gluster/brick2/vol4
Brick3: 10.70.35.192:/gluster/brick2/vol4
Options Reconfigured:
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: replica-vol5
Type: Replicate
Volume ID: 3fe31981-2701-42bc-b174-40f26a028994
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.36:/gluster/brick2/vol5
Brick2: 10.70.35.78:/gluster/brick2/vol5
Brick3: 10.70.35.192:/gluster/brick2/vol5
Options Reconfigured:
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: replica-vol6
Type: Replicate
Volume ID: dded6e4f-9240-41fd-bb1f-55d32404ec97
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.36:/gluster/brick2/vol6
Brick2: 10.70.35.78:/gluster/brick2/vol6
Brick3: 10.70.35.192:/gluster/brick2/vol6
Options Reconfigured:
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-1-1
Type: Replicate
Volume ID: c8642846-f469-4890-aa44-836ad113f8f6
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-1-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-1-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-1-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-1-2
Type: Replicate
Volume ID: ff010649-eb46-4f6b-97ef-f1b5fa8fcf45
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-1-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-1-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-1-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-1-3
Type: Replicate
Volume ID: e1d8f22b-5f95-4c0d-aa72-a6f0198c3e9f
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-1-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-1-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-1-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-10-1
Type: Replicate
Volume ID: c36876a9-8a97-4062-9ed6-7ae01b9ff8a8
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-10-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-10-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-10-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-10-2
Type: Replicate
Volume ID: 1832ed72-6287-4a09-a45e-1b500c655b7e
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-10-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-10-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-10-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-10-3
Type: Replicate
Volume ID: 231f5121-fa0f-484e-9152-f2f95485ff3e
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-10-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-10-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-10-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-2-1
Type: Replicate
Volume ID: b5ea7e9c-af42-4d6a-b3d5-6f3f648c5631
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-2-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-2-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-2-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-2-2
Type: Replicate
Volume ID: ee9ed353-ce96-459b-a01d-dfabfa2fb9e9
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-2-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-2-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-2-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-2-3
Type: Replicate
Volume ID: 10792991-8754-4e44-97ca-2fea440bce9d
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-2-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-2-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-2-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-3-1
Type: Replicate
Volume ID: 9d2e378b-6128-472b-8763-790198e5dba8
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-3-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-3-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-3-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-3-2
Type: Replicate
Volume ID: 07745afc-bdca-4dda-a26a-49c59ce720ec
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-3-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-3-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-3-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-3-3
Type: Replicate
Volume ID: 99d59c09-e223-448d-96e4-07d0e389e2f8
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-3-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-3-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-3-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-4-1
Type: Replicate
Volume ID: 5cf57641-29a0-4619-a46e-48db7d222ea0
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-4-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-4-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-4-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-4-2
Type: Replicate
Volume ID: 3774d248-aea3-4fe5-97a8-623511390125
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-4-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-4-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-4-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-4-3
Type: Replicate
Volume ID: 8e43775d-dbe9-42b8-8891-70786585b7f9
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-4-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-4-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-4-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-5-1
Type: Replicate
Volume ID: d3702d74-4acc-4136-ba6f-eaf0060bbc42
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-5-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-5-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-5-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-5-2
Type: Replicate
Volume ID: 2d534e2d-e640-4b27-941a-687150666b1d
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-5-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-5-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-5-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-5-3
Type: Replicate
Volume ID: 7ff6c489-1861-4d8a-a7ae-335d40ed1412
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-5-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-5-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-5-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-6-1
Type: Replicate
Volume ID: ee03b122-ad01-43ac-8db8-f9aa221b5b1f
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-6-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-6-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-6-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-6-2
Type: Replicate
Volume ID: 858a87ef-6556-4f9b-96ec-0e9632339ab7
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-6-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-6-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-6-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-6-3
Type: Replicate
Volume ID: 8205d65e-0418-4126-86c4-53399c449e2d
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-6-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-6-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-6-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-7-1
Type: Replicate
Volume ID: aa5bebec-1a08-4df0-bdd2-a5c8c20d1648
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-7-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-7-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-7-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-7-2
Type: Replicate
Volume ID: b1306d40-0c40-4054-93ae-dbe457991a52
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-7-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-7-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-7-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-7-3
Type: Replicate
Volume ID: 28b0fae0-2bb2-423b-9bae-326499ffc056
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-7-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-7-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-7-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-8-1
Type: Replicate
Volume ID: bf70d466-40df-4a22-8fb0-bc962be14d8e
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-8-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-8-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-8-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-8-2
Type: Replicate
Volume ID: 33d3e0c5-2a3c-47c9-8781-88f7578d8529
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-8-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-8-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-8-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-8-3
Type: Replicate
Volume ID: 5a51b1b0-98fa-41bc-818f-2cacbb60ddf7
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick3/vol_-8-3
Brick2: 10.70.35.78:/gluster/brick3/vol_-8-3
Brick3: 10.70.35.36:/gluster/brick3/vol_-8-3
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-9-1
Type: Replicate
Volume ID: b510a570-dd5b-4eb3-bd8a-fb9ba75aa060
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick1/vol_-9-1
Brick2: 10.70.35.78:/gluster/brick1/vol_-9-1
Brick3: 10.70.35.36:/gluster/brick1/vol_-9-1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
 
Volume Name: vol_-9-2
Type: Replicate
Volume ID: ba250ecc-ca4d-4697-a1cf-6917662baf0c
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.70.35.224:/gluster/brick2/vol_-9-2
Brick2: 10.70.35.78:/gluster/brick2/vol_-9-2
Brick3: 10.70.35.36:/gluster/brick2/vol_-9-2
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.max-bricks-per-process: 0
cluster.brick-multiplex: true
[root@dhcp35-36 gluster]# 




[root@dhcp35-36 gluster]# gluster vol status
Status of volume: replica-vol10
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.36:/gluster/brick1/v2        N/A       N/A        N       N/A  
Brick 10.70.35.78:/gluster/brick1/v2        49153     0          Y       24313
Brick 10.70.35.192:/gluster/brick1/v2       49152     0          Y       20214
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume replica-vol10
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: replica-vol11
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.36:/gluster/brick1/v21       49154     0          Y       6750 
Brick 10.70.35.78:/gluster/brick1/v21       49153     0          Y       24313
Brick 10.70.35.192:/gluster/brick1/v21      49152     0          Y       20214
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume replica-vol11
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: replica-vol3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.36:/gluster/brick2/vol3      N/A       N/A        N       N/A  
Brick 10.70.35.78:/gluster/brick2/vol3      49153     0          Y       24313
Brick 10.70.35.192:/gluster/brick2/vol3     49152     0          Y       20214
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume replica-vol3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: replica-vol4
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.36:/gluster/brick2/vol4      N/A       N/A        N       N/A  
Brick 10.70.35.78:/gluster/brick2/vol4      49153     0          Y       24313
Brick 10.70.35.192:/gluster/brick2/vol4     49152     0          Y       20214
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume replica-vol4
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: replica-vol5
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.36:/gluster/brick2/vol5      N/A       N/A        N       N/A  
Brick 10.70.35.78:/gluster/brick2/vol5      49153     0          Y       24313
Brick 10.70.35.192:/gluster/brick2/vol5     49152     0          Y       20214
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
 
Task Status of Volume replica-vol5
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: replica-vol6
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.36:/gluster/brick2/vol6      N/A       N/A        N       N/A  
Brick 10.70.35.78:/gluster/brick2/vol6      49153     0          Y       24313
Brick 10.70.35.192:/gluster/brick2/vol6     49152     0          Y       20214
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume replica-vol6
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-1-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-1-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-1-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-1-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-1-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-1-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-1-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-1-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-1-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-1-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-1-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-1-3 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-1-3  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-1-3  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-1-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-10-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-10-
1                                           49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-10-1 49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-10-1 N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-10-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-10-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-10-
2                                           49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-10-2 49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-10-2 N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-10-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-10-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-10-
3                                           49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-10-3 49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-10-3 N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-10-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-2-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-2-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-2-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-2-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume vol_-2-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-2-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-2-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-2-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-2-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-2-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-2-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-2-3 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-2-3  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-2-3  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-2-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-3-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-3-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-3-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-3-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-3-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-3-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-3-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-3-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-3-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-3-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-3-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-3-3 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-3-3  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-3-3  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-3-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-4-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-4-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-4-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-4-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-4-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-4-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-4-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-4-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-4-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-4-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-4-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-4-3 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-4-3  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-4-3  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-4-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-5-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-5-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-5-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-5-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-5-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-5-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-5-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-5-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-5-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-5-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-5-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-5-3 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-5-3  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-5-3  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-5-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-6-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-6-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-6-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-6-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-6-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-6-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-6-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-6-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-6-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-6-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-6-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-6-3 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-6-3  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-6-3  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume vol_-6-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-7-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-7-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-7-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-7-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-7-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-7-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-7-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-7-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-7-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-7-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-7-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-7-3 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-7-3  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-7-3  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-7-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-8-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-8-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-8-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-8-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
 
Task Status of Volume vol_-8-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-8-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-8-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-8-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-8-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume vol_-8-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-8-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick3/vol_-8-3 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick3/vol_-8-3  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick3/vol_-8-3  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume vol_-8-3
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-9-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick1/vol_-9-1 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick1/vol_-9-1  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick1/vol_-9-1  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
 
Task Status of Volume vol_-9-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-9-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.224:/gluster/brick2/vol_-9-2 49152     0          Y       23450
Brick 10.70.35.78:/gluster/brick2/vol_-9-2  49153     0          Y       24313
Brick 10.70.35.36:/gluster/brick2/vol_-9-2  N/A       N/A        N       N/A  
Self-heal Daemon on localhost               N/A       N/A        Y       6922 
Self-heal Daemon on 10.70.35.192            N/A       N/A        Y       6697 
Self-heal Daemon on 10.70.35.159            N/A       N/A        Y       8862 
Self-heal Daemon on 10.70.35.23             N/A       N/A        Y       8956 
Self-heal Daemon on 10.70.35.224            N/A       N/A        Y       1670 
Self-heal Daemon on 10.70.35.78             N/A       N/A        Y       12130
 
Task Status of Volume vol_-9-2
------------------------------------------------------------------------------
There are no active volume tasks
 
[root@dhcp35-36 gluster]#

Comment 3 Atin Mukherjee 2018-10-09 14:58:26 UTC

I'll get this bug picked up once I look at BZ 1637379 as that seems to be the originating source of the problem in the setup.

Comment 4 Atin Mukherjee 2018-10-10 04:10:25 UTC

Do you still have the set up? If yes, I'd like you to get me the output of following:

Considering Brick 10.70.35.36:/gluster/brick1/vol_-9-1 is showing N/A, please take output of

cat /var/run/gluster/vols/vol_-9-1/<pidfile> . If you see the file and a number inside it, do a ps aux | grep <pid number>.

On a different note, I couldn't find out this volume is being force started from cmd_history.log file.

Also can you/Nag justify the reason of marking this as severity urgent?

Comment 18 Atin Mukherjee 2018-11-09 11:27:05 UTC

upstream patch : https://review.gluster.org/#/c/21568/

Comment 26 errata-xmlrpc 2019-02-04 07:41:25 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0263

Note You need to log in before you can comment on or make changes to this bug.