Bug 850873

Summary: glusterfsd tries to set extended attribute "umask" when files are created
Product: [Community] GlusterFS Reporter: Tobias Sandhaas <tobias.sandhaas>
Component: coreAssignee: Amar Tumballi <amarts>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: low Docs Contact:
Priority: unspecified    
Version: 3.3.0CC: gluster-bugs, tobias.sandhaas, vraman
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.4.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-07-24 18:01:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
strace of glusterfsd during file creation none

Description Tobias Sandhaas 2012-08-22 15:22:24 UTC
Created attachment 606311 [details]
strace of glusterfsd during file creation

Hi,

Description of problem:
For every file creation event there is an error log entry in the
brick log. In our environment this log fills up very fast!


Version-Release number of selected component (if applicable):
OS: RHEL6.3 (latest updates)
Client:
glusterfs-fuse-3.3.0-1.el6.x86_64
fuse-libs-2.8.3-4.el6.x86_64
fuse-2.8.3-4.el6.x86_64

Server:
glusterfs-fuse-3.3.0-1.el6.x86_64
glusterfs-3.3.0-1.el6.x86_64
glusterfs-server-3.3.0-1.el6.x86_64

xfs filesystem:
$ mkfs.xfs -i size=512 -b size=4096 -d sunit=128,swidth=512,agcount=56 -n size=8k /dev/mapper/vg_gluster01-lv_space -f
$ /etc/fstab: /dev/vg_gluster01/lv_space  /space  xfs     noatime,nodiratime,attr2,nobarrier,logbufs=8,logbsize=256k,inode64,delaylog 1 2


How reproducible:
Just create a new file/directory and check the brick log.

Steps to Reproduce:
1. $ touch /space2/testvolume2/fileA
2. Check brick log "space2-n1b1.log"
  E [posix.c:1754:posix_create] 0-testvolume2-posix: setting xattrs on
     /space2/n1b1/fileA failed (Operation not supported)
 and additionally sometimes
  W [posix-helpers.c:665:posix_handle_pair] 0-testvolume2-posix: Extended
     attributes not supported (try remounting brick with 'user_xattr' flag)


Additional info:

On server:
# ls -lad /space2/n1b1/fileA
-rw-r--r-- 2 root root 0 Aug 22 17:09 /space2/n1b1/fileA
# getfattr -d /space2/n1b1/fileA
# attr -l /space2/n1b1/fileA
Attribute "gfid" has a 16 byte value for /space2/n1b1/fileA

On client:
# umask
0022

strace shows following lines:
26592 lstat("/space2/n1b1/10", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
26592 lgetxattr("/space2/n1b1/10", "trusted.gfid", 0x7fe37df0f9d0, 16) = -1 ENODATA (No data available)
26592 lsetxattr("/space2/n1b1/10", "trusted.gfid", "\x10\x0d6\xb4\xec\x0dG\xcd\xa5Hf0\xb4\xa5\x81\xd5", 16, XATTR_CREATE) = 0
<...>
26592 chown("/space2/n1b1/10", 0, 0)    = 0
26592 lstat("/space2/n1b1/10", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
26592 lsetxattr("/space2/n1b1/10", "umask", "18", 3, XATTR_CREATE) = -1 EOPNOTSUPP (Operation not supported)
26592 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2309, ...}) = 0
26592 write(4, "[2012-08-22 16:24:57.961552] E [posix.c:1754:posix_create] 0-testvolume2-posix: setting xattrs on /space2/n1b1/10 failed (Operation not supported)\n", 147) = 147


Possible fix (?):
I assume it's not valid to just use "umask" as value in lsetxattr(). I
do not know why glusterfs tries to set this extended attribute "umask".
Perhaps you should add a more generic attribute filter in
posix-helpers.c:posix_entry_create_xattr_set()?


Additional info:
Extended attributes are working, as can be seen:
$ echo "18" | attr -s umask /space2/n1b1/10 |grep lsetxattr
 lsetxattr("/space2/n1b1/10", "user.umask", "18\x0a", 3, 0) = 0


Regards
Tobias Sandhaas

Comment 1 Amar Tumballi 2012-08-22 16:11:24 UTC
i see the problem...

can you try the patch below on top of the source?

diff --git a/xlators/system/posix-acl/src/posix-acl.c b/xlators/system/posix-acl/src/posix-acl.c
index f54e20a..15ee46a 100644
--- a/xlators/system/posix-acl/src/posix-acl.c
+++ b/xlators/system/posix-acl/src/posix-acl.c
@@ -559,9 +559,12 @@ posix_acl_inherit (xlator_t *this, loc_t *loc, dict_t *params, mode_t mode,
         ret = dict_get_int16 (params, "umask", &tmp_mode);
         if (ret == 0) {
                 client_umask = (mode_t)tmp_mode;
+                dict_del (params, "umask");
                 ret = dict_get_int16 (params, "mode", &tmp_mode);
                 if (ret == 0) {
                         retmode = (mode_t)tmp_mode;
+                        dict_del (params, "mode");
+
                 } else {
                         gf_log (this->name, GF_LOG_ERROR,
                                 "client sent umask, but not the original mode");
------
This should fix the issue for now..

Comment 2 Amar Tumballi 2012-08-22 17:17:37 UTC
patch posted to fix the issue - http://review.gluster.org/3843... if accepted, the fix will go with 3.3.2 or higher releases.

Comment 3 Vijay Bellur 2012-08-23 08:02:57 UTC
CHANGE: http://review.gluster.org/3843 (posix-acl: make use of 'umask' flag passed as argument in fop) merged in master by Anand Avati (avati)