Description of problem:
=======================
On a default volume there is a volume option 'transport.address-family: inet' which vanishes on resetting a volume and because of this change in volume options ,brick mux feature breaks


Version-Release number of selected component (if applicable):
============================================================
Have been seeing this from a long time
Current testing build - glusterfs-server-6.0-24.el7rhgs.x86_64

How reproducible:
=================
Everytime

Steps to Reproduce:
1.Enable brickmux and create 2-3 volumes , start the volume , all will have same PID and volume option transport.address-family: inet
2.Reset a volume, this volume option is no longer present
3.Start - Stop the volume which was reset 

Actual results:
===============
The volume which was reset and start-stopped , comes up with a different PID


Expected results:
=================
If on a new volume option transport.address-family: inet is present it should not disappear after volume reset


Additional info:
=================
[root@rhs-gp-srv1 glusterd]# gluster v info
 
Volume Name: disperse-vol
Type: Distributed-Disperse
Volume ID: e4741d05-6bb0-4389-8e24-1d556cc01ae8
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x (4 + 2) = 12
Transport-type: tcp
Bricks:
Brick1: 10.70.36.8:/gluster/brick1/ec1
Brick2: 10.70.36.5:/gluster/brick1/ec1
Brick3: 10.70.36.6:/gluster/brick1/ec1
Brick4: 10.70.36.9:/gluster/brick1/ec1
Brick5: 10.70.36.17:/gluster/brick1/ec1
Brick6: 10.70.36.15:/gluster/brick1/ec1
Brick7: 10.70.36.8:/gluster/brick2/ec2
Brick8: 10.70.36.5:/gluster/brick2/ec2
Brick9: 10.70.36.6:/gluster/brick2/ec2
Brick10: 10.70.36.9:/gluster/brick2/ec2
Brick11: 10.70.36.17:/gluster/brick2/ec2
Brick12: 10.70.36.15:/gluster/brick2/ec2
Options Reconfigured:
storage.fips-mode-rchecksum: on
nfs.disable: on
cluster.brick-multiplex: enable
 
Volume Name: ec-vol
Type: Disperse
Volume ID: a6c976c6-a705-4782-8684-c2e60b805b78
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x (4 + 2) = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.36.8:/gluster/brick3/ec1
Brick2: 10.70.36.5:/gluster/brick3/ec1
Brick3: 10.70.36.6:/gluster/brick3/ec1
Brick4: 10.70.36.9:/gluster/brick3/ec1
Brick5: 10.70.36.17:/gluster/brick3/ec1
Brick6: 10.70.36.15:/gluster/brick3/ec1
Options Reconfigured:
transport.address-family: inet
storage.fips-mode-rchecksum: on
nfs.disable: on
cluster.brick-multiplex: enable
 
Volume Name: ec-vol1
Type: Disperse
Volume ID: 6c7f409c-6fca-4d08-8aac-da7fb06d6ec4
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x (4 + 2) = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.36.8:/gluster/brick4/ec1
Brick2: 10.70.36.5:/gluster/brick4/ec1
Brick3: 10.70.36.6:/gluster/brick4/ec1
Brick4: 10.70.36.9:/gluster/brick4/ec1
Brick5: 10.70.36.17:/gluster/brick4/ec1
Brick6: 10.70.36.15:/gluster/brick4/ec1
Options Reconfigured:
transport.address-family: inet
storage.fips-mode-rchecksum: on
nfs.disable: on
cluster.brick-multiplex: enable


Volume disperse-vol has been reset


[root@rhs-gp-srv1 glusterd]# gluster v get disperse-vol all|grep 'transport'
transport.keepalive                     1                                       
transport.listen-backlog                1024                                    
[root@rhs-gp-srv1 glusterd]# 

[root@rhs-gp-srv1 glusterd]# gluster v get ec-vol1 all |grep transport
transport.keepalive                     1                                       
transport.listen-backlog                1024                                    
transport.address-family                inet

--- Additional comment from Upasana on 2019-12-10 18:44:07 IST ---

As this is easily reproducible, not uploading sos-reports.
Please let me know if any more information or data is required

--- Additional comment from Sanju on 2019-12-11 16:42:13 IST ---

Here, the brick-mux feature failing because of the option change. When different options are set for volumes, the volumes will not be compatible with each other and we will have different PID's for bricks.

The actual issue here is: the option <transport.address-family> should not get disappear after volume reset. I will look into it.