Created attachment 606311 [details] strace of glusterfsd during file creation Hi, Description of problem: For every file creation event there is an error log entry in the brick log. In our environment this log fills up very fast! Version-Release number of selected component (if applicable): OS: RHEL6.3 (latest updates) Client: glusterfs-fuse-3.3.0-1.el6.x86_64 fuse-libs-2.8.3-4.el6.x86_64 fuse-2.8.3-4.el6.x86_64 Server: glusterfs-fuse-3.3.0-1.el6.x86_64 glusterfs-3.3.0-1.el6.x86_64 glusterfs-server-3.3.0-1.el6.x86_64 xfs filesystem: $ mkfs.xfs -i size=512 -b size=4096 -d sunit=128,swidth=512,agcount=56 -n size=8k /dev/mapper/vg_gluster01-lv_space -f $ /etc/fstab: /dev/vg_gluster01/lv_space /space xfs noatime,nodiratime,attr2,nobarrier,logbufs=8,logbsize=256k,inode64,delaylog 1 2 How reproducible: Just create a new file/directory and check the brick log. Steps to Reproduce: 1. $ touch /space2/testvolume2/fileA 2. Check brick log "space2-n1b1.log" E [posix.c:1754:posix_create] 0-testvolume2-posix: setting xattrs on /space2/n1b1/fileA failed (Operation not supported) and additionally sometimes W [posix-helpers.c:665:posix_handle_pair] 0-testvolume2-posix: Extended attributes not supported (try remounting brick with 'user_xattr' flag) Additional info: On server: # ls -lad /space2/n1b1/fileA -rw-r--r-- 2 root root 0 Aug 22 17:09 /space2/n1b1/fileA # getfattr -d /space2/n1b1/fileA # attr -l /space2/n1b1/fileA Attribute "gfid" has a 16 byte value for /space2/n1b1/fileA On client: # umask 0022 strace shows following lines: 26592 lstat("/space2/n1b1/10", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 26592 lgetxattr("/space2/n1b1/10", "trusted.gfid", 0x7fe37df0f9d0, 16) = -1 ENODATA (No data available) 26592 lsetxattr("/space2/n1b1/10", "trusted.gfid", "\x10\x0d6\xb4\xec\x0dG\xcd\xa5Hf0\xb4\xa5\x81\xd5", 16, XATTR_CREATE) = 0 <...> 26592 chown("/space2/n1b1/10", 0, 0) = 0 26592 lstat("/space2/n1b1/10", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 26592 lsetxattr("/space2/n1b1/10", "umask", "18", 3, XATTR_CREATE) = -1 EOPNOTSUPP (Operation not supported) 26592 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2309, ...}) = 0 26592 write(4, "[2012-08-22 16:24:57.961552] E [posix.c:1754:posix_create] 0-testvolume2-posix: setting xattrs on /space2/n1b1/10 failed (Operation not supported)\n", 147) = 147 Possible fix (?): I assume it's not valid to just use "umask" as value in lsetxattr(). I do not know why glusterfs tries to set this extended attribute "umask". Perhaps you should add a more generic attribute filter in posix-helpers.c:posix_entry_create_xattr_set()? Additional info: Extended attributes are working, as can be seen: $ echo "18" | attr -s umask /space2/n1b1/10 |grep lsetxattr lsetxattr("/space2/n1b1/10", "user.umask", "18\x0a", 3, 0) = 0 Regards Tobias Sandhaas
i see the problem... can you try the patch below on top of the source? diff --git a/xlators/system/posix-acl/src/posix-acl.c b/xlators/system/posix-acl/src/posix-acl.c index f54e20a..15ee46a 100644 --- a/xlators/system/posix-acl/src/posix-acl.c +++ b/xlators/system/posix-acl/src/posix-acl.c @@ -559,9 +559,12 @@ posix_acl_inherit (xlator_t *this, loc_t *loc, dict_t *params, mode_t mode, ret = dict_get_int16 (params, "umask", &tmp_mode); if (ret == 0) { client_umask = (mode_t)tmp_mode; + dict_del (params, "umask"); ret = dict_get_int16 (params, "mode", &tmp_mode); if (ret == 0) { retmode = (mode_t)tmp_mode; + dict_del (params, "mode"); + } else { gf_log (this->name, GF_LOG_ERROR, "client sent umask, but not the original mode"); ------ This should fix the issue for now..
patch posted to fix the issue - http://review.gluster.org/3843... if accepted, the fix will go with 3.3.2 or higher releases.
CHANGE: http://review.gluster.org/3843 (posix-acl: make use of 'umask' flag passed as argument in fop) merged in master by Anand Avati (avati)