Bug 1491756

Summary: [GSS] read-only mount allows writes
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Bipin Kunal <bkunal>
Component: fuseAssignee: Amar Tumballi <atumball>
Status: CLOSED ERRATA QA Contact: Rochelle <rallan>
Severity: medium Docs Contact:
Priority: high    
Version: rhgs-3.2CC: amukherj, atumball, jervine, jorobins, ndevos, psony, rcyriac, rhs-bugs, storage-qa-internal
Target Milestone: ---Keywords: Regression, ZStream
Target Release: RHGS 3.3.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.8.4-51 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-29 03:30:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1507006    
Bug Blocks: 1475688    
Attachments:
Description Flags
trace mount log with read only parameter
none
trace mount log without read-only parameter none

Description Bipin Kunal 2017-09-14 15:16:01 UTC 
Description of problem: Writes are allowed even when client is mounted in read-only mode


Version-Release number of selected component (if applicable):
# rpm -qa | grep glusterfs-server
glusterfs-server-3.8.4-18.6.el7rhgs.x86_64

rpm -qa | grep glusterfs-fuse
glusterfs-fuse-3.8.4-18.6.el7rhgs.x86_64

# cat /var/lib/glusterd/glusterd.info 
UUID=555b4445-7a87-45f5-a52f-0e61ea0c9aab
operating-version=31001

How reproducible:
Always

Steps to Reproduce:
1. create a gluster volume
   # gluster vol info
 
 Volume Name: replicate
 Type: Replicate
 Volume ID: ade9ec77-5fe0-43d4-9912-eee890a87fdb
 Status: Started
 Snapshot Count: 0
 Number of Bricks: 1 x 3 = 3
 Transport-type: tcp
 Bricks:
 Brick1: Server1:/bipin/1
 Brick2: Server1:/bipin/2
 Brick3: Server1:/bipin/3
 Options Reconfigured:
 transport.address-family: inet
 nfs.disable: off
 features.read-only: off
 performance.readdir-ahead: on

2. mount gluster volume with "read-only" mount option

  # mount -t glusterfs  -o ro Server1:/replicate /mnt

3. create files on the mount point

Actual results: Able to write on read-only mounted FS


Expected results: Write should not be allowed on read-only FS


Additional info: In further comments
Comment 2 Bipin Kunal 2017-09-14 15:20:37 UTC 
# ps -ax | grep mnt
30149 pts/1    S+     0:00 tailf /var/log/glusterfs/mnt.log
30418 ?        Ssl    0:00 /usr/sbin/glusterfs --log-level=TRACE --volfile-server=Server1 --volfile-id=/replicate /mnt
31377 pts/0    S+     0:00 grep --color=auto mnt

# umount /mnt
# /usr/sbin/glusterfs --log-level=TRACE  --read-only --volfile-server=Server1 --volfile-id=/replicate /mnt

# cd /mnt
# ls
1.txt  2.txt  3.txt  4.txt
# touch 5.txt
# echo ":khgdscgx" > 5.txt
# cat 5.txt
:khgdscgx

# cd ..
# umount /mnt

# ps -ax | grep mnt
30149 pts/1    S+     0:00 tailf /var/log/glusterfs/mnt.log
31485 pts/0    S+     0:00 grep --color=auto mnt
# mount -a

# cat /etc/fstab | grep mnt
Server1:/replicate 	/mnt	glusterfs	ro	0 	0

# ps -ax | grep mnt
31544 ?        Ssl    0:00 /usr/sbin/glusterfs --read-only --volfile-server=Server1 --volfile-id=/replicate /mnt
31574 pts/0    R+     0:00 grep --color=auto mnt

# cd /mnt
# ls
1.txt  2.txt  3.txt  4.txt  5.txt
# echo "kagdkc" >> 5.txt 

#  rpm -qa | grep glusterfs-fuse
glusterfs-fuse-3.8.4-18.6.el7rhgs.x86_64
Comment 3 Bipin Kunal 2017-09-14 15:25:47 UTC 
TRACE MOUNT LOG after command :
========================================================================

/usr/sbin/glusterfs --log-level=TRACE --read-only --volfile-server=Server1 --volfile-id=/replicate /mnt

========================================================================

[2017-09-14 15:23:21.033853] I [MSGID: 100030] [glusterfsd.c:2412:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.8.4 (args: /usr/sbin/glusterfs --log-level=TRACE --read-only --volfile-server=Server1 --volfile-id=/replicate /mnt)
[2017-09-14 15:23:21.033984] T [MSGID: 0] [xlator.c:193:xlator_dynload] 0-xlator: attempt to load file /usr/lib64/glusterfs/3.8.4/xlator/mount/fuse.so
[2017-09-14 15:23:21.034409] T [MSGID: 0] [xlator.c:244:xlator_dynload] 0-xlator: dlsym(reconfigure) on /usr/lib64/glusterfs/3.8.4/xlator/mount/fuse.so: undefined symbol: reconfigure -- neglecting
[2017-09-14 15:23:21.034450] D [MSGID: 0] [glusterfsd.c:439:set_fuse_mount_options] 0-glusterfsd: fopen-keep-cache mode 2
[2017-09-14 15:23:21.034458] D [MSGID: 0] [glusterfsd.c:503:set_fuse_mount_options] 0-glusterfsd: fuse direct io type 2
[2017-09-14 15:23:21.034464] D [MSGID: 0] [glusterfsd.c:527:set_fuse_mount_options] 0-glusterfsd: fuse no-root-squash mode 0
.
.
.
.
.
.
.
[2017-09-14 15:23:21.034722] D [MSGID: 0] [options.c:1219:xlator_option_init_bool] 0-fuse: option read-only using set value on
.
.
.

========================================================================
Complete log attached.
========================================================================
# gluster vol get replicate features.read-only
Option                                  Value                                   
------                                  -----                                   
features.read-only                      off                                     
========================================================================
Comment 4 Bipin Kunal 2017-09-14 15:27:52 UTC 
Created attachment 1326114 [details]
trace mount log with read only parameter
Comment 5 Bipin Kunal 2017-09-14 15:31:39 UTC 
TRACE MOUNT LOG after command :(Without read-only parameter)
========================================================================

/usr/sbin/glusterfs --log-level=TRACE  --volfile-server=Server1 --volfile-id=/replicate /mnt

========================================================================
[2017-09-14 15:30:11.421566] I [MSGID: 100030] [glusterfsd.c:2412:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.8.4 (args: /usr/sbin/glusterfs --log-level=TRACE --volfile-server=Server1 --volfile-id=/replicate /mnt)
.
.
.
.
[2017-09-14 15:30:11.422885] T [MSGID: 0] [options.c:1219:xlator_option_init_bool] 0-fuse: option read-only not set
.
.
.
========================================================================
Complete mount log attached.
========================================================================
# gluster vol get replicate features.read-only
Option                                  Value                                   
------                                  -----                                   
features.read-only                      off                                     
========================================================================
Comment 6 Bipin Kunal 2017-09-14 15:32:31 UTC 
Created attachment 1326115 [details]
trace mount log without read-only parameter
Comment 17 Amar Tumballi 2017-10-31 09:49:47 UTC 
https://code.engineering.redhat.com/gerrit/#/c/121852/2
Comment 21 Rochelle 2017-11-06 08:34:49 UTC 
Verified with build : glusterfs-3.8.4-51.el7rhgs.x86_64
Works for both FUSE and NFS mounts.

Not able to write into a read-only mount. 

Moving this bug to verified.


Case 1 : No options set + Fuse read only mount
==============================================
[root@dhcp43-14]# gluster volume info read_only
 
Volume Name: read_only
Type: Distributed-Replicate
Volume ID: d9658750-dc8e-406c-b2d7-37aba37abc1d
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.43.14:/rhs/brick1/b1
Brick2: 10.70.43.183:/rhs/brick1/b2
Brick3: 10.70.42.213:/rhs/brick1/b3
Brick4: 10.70.43.14:/rhs/brick2/b4
Brick5: 10.70.43.183:/rhs/brick2/b5
Brick6: 10.70.42.213:/rhs/brick2/b6
Options Reconfigured:
nfs.disable: on
cluster.enable-shared-storage: enable

[root@dhcp37-115 ~]# mount -t glusterfs -o ro 10.70.43.14:/read_only /mnt/ro


[root@dhcp37-115 ~]# mount | grep read_only
10.70.43.14:/read_only on /mnt/ro type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)

[root@dhcp37-115 ~]# cd /mnt/ro
 [root@dhcp37-115 ro]# touch a
touch: cannot touch ‘a’: Read-only file system



--------------------------------------------------------

Case 2: features.read-only: off + Fuse read-only mount
========================================================

[root@dhcp43-14]# gluster volume info 
 
Volume Name: read_only
Type: Distributed-Replicate
Volume ID: d9658750-dc8e-406c-b2d7-37aba37abc1d
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.43.14:/rhs/brick1/b1
Brick2: 10.70.43.183:/rhs/brick1/b2
Brick3: 10.70.42.213:/rhs/brick1/b3
Brick4: 10.70.43.14:/rhs/brick2/b4
Brick5: 10.70.43.183:/rhs/brick2/b5
Brick6: 10.70.42.213:/rhs/brick2/b6
Options Reconfigured:
features.read-only: off
nfs.disable: on
cluster.enable-shared-storage: enable

[root@dhcp37-115 ~]# mount -t glusterfs -o ro 10.70.43.14:/read_only /mnt/ro1

[root@dhcp37-115 ~]# mount | grep read_only
10.70.43.14:/read_only on /mnt/ro type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro1 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
[root@dhcp37-115 ~]# 

root@dhcp37-115 ~]# cd /mnt/ro1
[root@dhcp37-115 ro1]# touch b
touch: cannot touch ‘b’: Read-only file system


------------------------------------------------------------------

case 3: features.read-only: on + Fuse read-only mount
=====================================================

[root@dhcp43-14]# gluster volume info 
 
Volume Name: read_only
Type: Distributed-Replicate
Volume ID: d9658750-dc8e-406c-b2d7-37aba37abc1d
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.43.14:/rhs/brick1/b1
Brick2: 10.70.43.183:/rhs/brick1/b2
Brick3: 10.70.42.213:/rhs/brick1/b3
Brick4: 10.70.43.14:/rhs/brick2/b4
Brick5: 10.70.43.183:/rhs/brick2/b5
Brick6: 10.70.42.213:/rhs/brick2/b6
Options Reconfigured:
features.read-only: on
nfs.disable: on
cluster.enable-shared-storage: enable

[root@dhcp37-115 ~]# mount -t glusterfs -o ro 10.70.43.14:/read_only /mnt/ro2

[root@dhcp37-115 ~]# mount | grep read_only
10.70.43.14:/read_only on /mnt/ro type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro1 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro2 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)

[root@dhcp37-115 ~]# cd /mnt/ro2
[root@dhcp37-115 ro2]# touch c
touch: cannot touch ‘c’: Read-only file system

------------------------------------------------------------------------

case 4: No options set + nfs read-only mount
============================================

[root@dhcp43-14]# gluster volume info 
 
Volume Name: read_only
Type: Distributed-Replicate
Volume ID: d9658750-dc8e-406c-b2d7-37aba37abc1d
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.43.14:/rhs/brick1/b1
Brick2: 10.70.43.183:/rhs/brick1/b2
Brick3: 10.70.42.213:/rhs/brick1/b3
Brick4: 10.70.43.14:/rhs/brick2/b4
Brick5: 10.70.43.183:/rhs/brick2/b5
Brick6: 10.70.42.213:/rhs/brick2/b6
Options Reconfigured:
nfs.disable: off
cluster.enable-shared-storage: enable


[root@dhcp37-115 ~]# mount -t nfs -o vers=3,ro 10.70.43.14:/read_only /mnt/nfs-ro/

[root@dhcp37-115 ~]# mount | grep read_only
10.70.43.14:/read_only on /mnt/ro type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro1 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro2 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/nfs-ro type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.70.43.14,mountvers=3,mountport=38465,mountproto=tcp,local_lock=none,addr=10.70.43.14)

[root@dhcp37-115 ~]# cd /mnt/nfs-ro/

[root@dhcp37-115 ~]# cd /mnt/nfs-ro/
[root@dhcp37-115 nfs-ro]# 
[root@dhcp37-115 nfs-ro]# touch d
touch: cannot touch ‘d’: Read-only file system


------------------------------------------------------------------------------------

case 5: features.read-only: off + nfs read-only mount 
====================================================

[root@dhcp43-14]# gluster volume info 
 
Volume Name: read_only
Type: Distributed-Replicate
Volume ID: d9658750-dc8e-406c-b2d7-37aba37abc1d
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.43.14:/rhs/brick1/b1
Brick2: 10.70.43.183:/rhs/brick1/b2
Brick3: 10.70.42.213:/rhs/brick1/b3
Brick4: 10.70.43.14:/rhs/brick2/b4
Brick5: 10.70.43.183:/rhs/brick2/b5
Brick6: 10.70.42.213:/rhs/brick2/b6
Options Reconfigured:
features.read-only: off
nfs.disable: off
cluster.enable-shared-storage: enable


[root@dhcp37-115 ~]# mount -t nfs -o vers=3,ro 10.70.43.14:/read_only /mnt/nfs-ro1/

[root@dhcp37-115 ~]# mount | grep read_only
10.70.43.14:/read_only on /mnt/ro type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro1 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro2 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/nfs-ro type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.70.43.14,mountvers=3,mountport=38465,mountproto=tcp,local_lock=none,addr=10.70.43.14)
10.70.43.14:/read_only on /mnt/nfs-ro1 type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.70.43.14,mountvers=3,mountport=38465,mountproto=tcp,local_lock=none,addr=10.70.43.14)

[root@dhcp37-115 ~]# cd /mnt/nfs-ro1

[root@dhcp37-115 nfs-ro1]# touch e
touch: cannot touch ‘e’: Read-only file system

------------------------------------------------------------------------------------

case 6: features.read-only: on + nfs read-only mount
====================================================
[root@dhcp43-14]# gluster volume info 
 
Volume Name: read_only
Type: Distributed-Replicate
Volume ID: d9658750-dc8e-406c-b2d7-37aba37abc1d
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.43.14:/rhs/brick1/b1
Brick2: 10.70.43.183:/rhs/brick1/b2
Brick3: 10.70.42.213:/rhs/brick1/b3
Brick4: 10.70.43.14:/rhs/brick2/b4
Brick5: 10.70.43.183:/rhs/brick2/b5
Brick6: 10.70.42.213:/rhs/brick2/b6
Options Reconfigured:
features.read-only: on
nfs.disable: off
cluster.enable-shared-storage: enable


[root@dhcp37-115 ~]# mount -t nfs -o vers=3,ro 10.70.43.14:/read_only /mnt/nfs-ro2/


[root@dhcp37-115 ~]# mount | grep read_only
10.70.43.14:/read_only on /mnt/ro type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro1 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/ro2 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.70.43.14:/read_only on /mnt/nfs-ro type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.70.43.14,mountvers=3,mountport=38465,mountproto=tcp,local_lock=none,addr=10.70.43.14)
10.70.43.14:/read_only on /mnt/nfs-ro1 type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.70.43.14,mountvers=3,mountport=38465,mountproto=tcp,local_lock=none,addr=10.70.43.14)
10.70.43.14:/read_only on /mnt/nfs-ro2 type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.70.43.14,mountvers=3,mountport=38465,mountproto=tcp,local_lock=none,addr=10.70.43.14)

[root@dhcp37-115 ~]# cd /mnt/nfs-ro2/

[root@dhcp37-115 nfs-ro2]# touch f
touch: cannot touch ‘f’: Read-only file system
Comment 24 errata-xmlrpc 2017-11-29 03:30:36 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3276