1655875 – Brick Multiplexing: Brick process shows as online in vol status even when brick is brought down using umounting the LV

Bug 1655875 - Brick Multiplexing: Brick process shows as online in vol status even when brick is brought down using umounting the LV

Summary: Brick Multiplexing: Brick process shows as online in vol status even when bri...

Keywords:
Status:	CLOSED CANTFIX
Alias:	None
Product:	Red Hat Gluster Storage
Classification:	Red Hat Storage
Component:	core
Sub Component:
Version:	rhgs-3.4
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	medium
Target Milestone:	---
Target Release:	---
Assignee:	Sanju
QA Contact:	Rahul Hinduja
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2018-12-04 07:13 UTC by Upasana
Modified:	2019-11-27 05:16 UTC (History)
CC List:	5 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2019-11-27 05:16:25 UTC
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Description Upasana 2018-12-04 07:13:39 UTC

Description of problem:
========================

With brick mux enabled,if i bring down the bricks by umounting the LV some of the volumes of still show brick as online and heal info shows the brick online instead of transport end point error

Version-Release number of selected component (if applicable):
=============================================================
glusterfs-server-3.12.2-29.el7rhgs.x86_64

How reproducible:
=================
1/1

Steps to Reproduce:
1.Enabled brick mux and created 9 volumes (3 volumes using 1 LV )
2.now bring down bricks by using umount -l of the lv mount
3.The vol status still shows brick as online for 2 volumes and ofline for one volume 

Actual results:
==============
 
[root@dhcp35-56 ~]# umount -l /gluster/brick3/


Status of volume: vol_-3-1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.56:/gluster/brick3/vol_-3-1  N/A       N/A        N       2774 
Brick 10.70.35.227:/gluster/brick3/vol_-3-1 49152     0          Y       3051 
Brick 10.70.35.11:/gluster/brick3/vol_-3-1  49152     0          Y       11694
Self-heal Daemon on localhost               N/A       N/A        Y       3750 
Self-heal Daemon on dhcp35-227.lab.eng.blr.
redhat.com                                  N/A       N/A        Y       3827 
Self-heal Daemon on dhcp35-11.lab.eng.blr.r
edhat.com                                   N/A       N/A        Y       12935
 
Task Status of Volume vol_-3-1
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-3-2
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.56:/gluster/brick3/vol_-3-2  N/A       N/A        N       2774 
Brick 10.70.35.227:/gluster/brick3/vol_-3-2 49152     0          Y       3051 
Brick 10.70.35.11:/gluster/brick3/vol_-3-2  49152     0          Y       11694
Self-heal Daemon on localhost               N/A       N/A        Y       3750 
Self-heal Daemon on dhcp35-227.lab.eng.blr.
redhat.com                                  N/A       N/A        Y       3827 
Self-heal Daemon on dhcp35-11.lab.eng.blr.r
edhat.com                                   N/A       N/A        Y       12935
 
Task Status of Volume vol_-3-2
------------------------------------------------------------------------------
There are no active volume tasks
 
Status of volume: vol_-3-3
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.70.35.56:/gluster/brick3/vol_-3-3  N/A       N/A        N       N/A  
Brick 10.70.35.227:/gluster/brick3/vol_-3-3 49152     0          Y       3051 
Brick 10.70.35.11:/gluster/brick3/vol_-3-3  49152     0          Y       11694
Self-heal Daemon on localhost               N/A       N/A        Y       3750 
Self-heal Daemon on dhcp35-11.lab.eng.blr.r
edhat.com                                   N/A       N/A        Y       12935
Self-heal Daemon on dhcp35-227.lab.eng.blr.
redhat.com                                  N/A       N/A        Y       3827 
 
Task Status of Volume vol_-3-3
------------------------------------------------------------------------------
There are no active volume tasks
 


Expected results:
==================
The brick should be offline for all 3 volumes

Additional info:
=================
[root@dhcp35-56 ~]# df -h
Filesystem                           Size  Used Avail Use% Mounted on
/dev/mapper/rhgs-root                 26G  3.0G   24G  12% /
devtmpfs                             3.9G     0  3.9G   0% /dev
tmpfs                                3.9G     0  3.9G   0% /dev/shm
tmpfs                                3.9G   65M  3.8G   2% /run
tmpfs                                3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/GLUSTER_vg1-GLUSTER_lv1   50G  1.6G   49G   4% /gluster/brick1
/dev/mapper/GLUSTER_vg2-GLUSTER_lv2   50G   34M   50G   1% /gluster/brick2
/dev/sda1                           1014M  163M  852M  17% /boot
tmpfs                                783M     0  783M   0% /run/user/0
[root@dhcp35-56 ~]# 



The heal info also shows brick as online instead of showing transport end point error on the 2 volumes and on the 3rd it is fine

[root@dhcp35-56 ~]# gluster v heal vol_-3-1 info
Brick 10.70.35.56:/gluster/brick3/vol_-3-1
Status: Connected
Number of entries: 0

Brick 10.70.35.227:/gluster/brick3/vol_-3-1
/linux-4.19.6/Documentation/features/sched/membarrier-sync-core/arch-support.txt 
/linux-4.19.6/arch/arm/mach-dove/include/mach/bridge-regs.h 
/linux-4.19.6/arch/arm/mach-dove/include/mach/dove.h 
/linux-4.19.6/Documentation/features/sched/numa-balancing/arch-support.txt 
/linux-4.19.6/Documentation/w1/masters/ds2482 
/linux-4.19.6/Documentation/w1/masters/ds2490 
/linux-4.19.6/Documentation/w1/masters/00-INDEX 
/linux-4.19.6/Documentation/w1/masters/w1-gpio 
/linux-4.19.6/Documentation/features/seccomp/seccomp-filter/arch-support.txt 
/linux-4.19.6/Documentation/features/vm/ELF-ASLR/a



[root@dhcp35-56 ~]# gluster v heal vol_-3-2 info
Brick 10.70.35.56:/gluster/brick3/vol_-3-2
Status: Connected
Number of entries: 0

Brick 10.70.35.227:/gluster/brick3/vol_-3-2
/linux-4.19.6/arch/arm/boot/compressed/.gitignore 
/linux-4.19.6/arch/x86/include/asm/uv/uv_bau.h 
/linux-4.19.6/arch/arc/plat-eznps/include/plat 
/linux-4.19.6/arch/x86/include/asm/uv/uv_hub.h 
/linux-4.19.6/arch/x86/include/asm/uv/uv_irq.h 
/linux-4.19.6/arch/x86/include/asm/uv/uv_mmrs.h 
/linux-4.19.6/Documentation/aoe/autoload.sh 
/linux-4.19.6/arch/x86/include/asm/vdso.h 
/linux-4.19.6/arch/arm/mach-pxa/palmtc.c 
/linux-4.19.6/Documentation/DMA-API.txt 

[root@dhcp35-56 ~]# gluster v heal vol_-3-3 info
Brick 10.70.35.56:/gluster/brick3/vol_-3-3
Status: Transport endpoint is not connected
Number of entries: -

Brick 10.70.35.227:/gluster/brick3/vol_-3-3
Status: Connected
Number of entries: 0

Brick 10.70.35.11:/gluster/brick3/vol_-3-3
Status: Connected
Number of entries: 0


Very similar to Bug 1450806 - Brick Multiplexing: Brick process shows as online in vol status even when brick is offline and Bug 1450813 - Brick Multiplexing: heal info shows brick as online even when it is brought down both both the bugs have been closed.

Note You need to log in before you can comment on or make changes to this bug.