Bug 1666326

Description of problem:
When using trough NFS a fuse-mounted glusterfs volume, an error is found if the owner of a file (that is not root) tries to write data in its own file when it has read-only permissions.

The problem does not exist if the user is root.

Version-Release number of selected component (if applicable):
All versions tested (glusterfs-3.7.{4-16}, glusterfs-3.8.{4-5}), glusterfs-5.2

How reproducible:
Always

Steps to Reproduce:

Our architecture has 3 layers: the glusterfs servers layer, an intermediate server for reexporting it via NFS, and the final nfs client at the lab.

1. Mount with nfs (at the client) a fuse mounted glusterfs volume exported via nfs (at the intermediate server) to be the home of the user.

cd to home dir and check that the volume is mounted and the file that we will use for testing does not exist:
user@computer:~$ ls -l dest.txt
ls: cannot access dest.txt: No such file or directory
user@computer:~$

2a. Simple test with few logs:

Create a readonly file and try to write data in it. You can execute the following multi-command (error comes at echo, not before). Rm is there only for repetition purposes:

user@computer:~$ rm -f 444.txt; touch 444.txt; chmod 444 444.txt; echo test > 444.txt
-bash: 444.txt: Permission denied

This command fails but does not render information in the glusterfs client log of the NFS intermediate server. Only in the glusterfs server log:

The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 14 times between [2019-01-15 13:48:17.999280] and [2019-01-15 13:49:45.530578]

2b. Test with more logs available (error shows up when executing cp):

user@computer:~$ rm -f 444.txt dest.txt; echo test > 444.txt; chmod 444 444.txt; cp -p 444.txt dest.txt;
cp: failed to close 'dest.txt': Permission denied

Messages at the glusterfs client log of the NFS intermediate server:
[2019-01-15 13:50:05.955944] I [io-stats.c:1667:io_stats_dump_fd] 0-distribuido3468: --- fd stats ---
[2019-01-15 13:50:05.957628] I [io-stats.c:1670:io_stats_dump_fd] 0-distribuido3468:       Filename : /home/user/444.txt
[2019-01-15 13:50:05.959198] I [io-stats.c:1684:io_stats_dump_fd] 0-distribuido3468:   BytesWritten : 5 bytes
[2019-01-15 13:50:05.959217] I [io-stats.c:1698:io_stats_dump_fd] 0-distribuido3468: Write 000004b+ : 1
[2019-01-15 13:50:05.984823] E [MSGID: 114031] [client-rpc-fops_v2.c:281:client4_0_open_cbk] 0-distribuido3468-client-1: remote operation failed. Path: /home/user/dest.txt (25be0793-520e-4a87-9b34-f1b9f633af42) [Permission denied]
[2019-01-15 13:50:05.989937] W [fuse-bridge.c:1124:fuse_fd_cbk] 0-glusterfs-fuse: 1481: OPEN() /home/user/dest.txt => -1 (Permission denied)

Messages at the glusterfs server:
[2019-01-15 13:50:05.931054] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-01-15 13:50:05.956175] I [io-stats.c:1667:io_stats_dump_fd] 0-distribuido3468-io-stats: --- fd stats ---
[2019-01-15 13:50:05.958417] I [io-stats.c:1670:io_stats_dump_fd] 0-distribuido3468-io-stats:       Filename : /home/user/444.txt
[2019-01-15 13:50:05.961269] I [io-stats.c:1684:io_stats_dump_fd] 0-distribuido3468-io-stats:   BytesWritten : 5 bytes
[2019-01-15 13:50:05.961372] I [io-stats.c:1698:io_stats_dump_fd] 0-distribuido3468-io-stats: Write 000004b+ : 1
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 2 times between [2019-01-15 13:50:05.931054] and [2019-01-15 13:50:05.942111]


3. Check result: desired file is empty.
user@computer:~$ ls -l 444.txt
-r--r--r-- 1 user user 0 dic 15  2016 444.txt
user@computer:~$ rm -f 444.txt
user@computer:~$

4. Check that the problem dissapears with root privileges:
user@computer:~$ sudo su
root@computer:/home/user# rm -f 444.txt; touch 444.txt; chmod 444 444.txt; echo test > 444.txt
root@computer:/home/user# ls -l 444.txt
-r--r--r-- 1 root root 5 dic 15  2016 444.txt
root@computer:/home/user# rm -f 444.txt
root@computer:/home/user#

Actual results:
A user who owns a file that is read-only cannot write into it under the described circumstances.

Expected results:
A user who owns a file that is read-only should be able to write into it even if it is read-only.

Additional info:
Special thanks to Bruce Fields and other components of nfs-kernel-server team because initially we thought it was their fault. The same to Soumya Koduri and Miklos Szeredi, that helped to confirm the problem.

See full previous thread on nfs-kernel-server list with history about the issue:
http://marc.info/?t=144801803400001 (for posterity's sake, permanent link: http://marc.info/?i=862ef8b0-61ee-8b7e-a81c-fcdc5a726e37@dit.upm.es).

This bug is reopened because the team thought that was fixed in glusterfs-5.0. See previous bug report at: https://bugzilla.redhat.com/show_bug.cgi?id=1405147.

Presentation (in Spanish, 2015) about the lab config and some tests:
http://www.rediris.es/jt/jt2015/ponencias/?id=jt2015-jt-ses_4b_seg_red_camp_2-a17b2c1.pdf