Md-cache has option "cache-posix-acl" that controls caching of posix ACLs ("system.posix_acl_access"/"system.posix_acl_default") and virtual glusterfs ACLs ("glusterfs.posix.acl"/"glusterfs.posix.default_acl") now.

But, _posix_xattr_get_set does not fill virtual glusterfs ACLs when lookup requests.
So, md-cache caches bad virtual glusterfs ACLs.

After I turn on "cache-posix-acl" option to cache ACLs at md-cache, nfs client gets many EIO errors.

https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/427305

There are two chooses for cache virtual glusterfs ACLs in md-cache,
1. Cache it separately as posix ACLs (a new option maybe "cache-glusterfs-acl" is added);
   And make sure _posix_xattr_get_set fills them when lookup requests.

2. Does not cache it, only cache posix ACLs;
   If gfapi request it, md-cache lookup according posix ACL at cache,
   if exist, make the virtual glusterfs ACL locally and return to gfapi;
   otherwise, send the request to glusterfsd.

Virtual glusterfs ACLs are another format of posix ACLs, there are larger than posix ACLs, 
and always exist no matter the really posix ACL exist or not.

So, I'd prefer #2.

REVIEW: https://review.gluster.org/21410 (md-cache: remove xattr setting after new file created) posted (#2) for review on master by Kinglong Mee

REVIEW: https://review.gluster.org/21412 (md-cache: extends xa_time when previous attributes equals cached attributes) posted (#2) for review on master by Kinglong Mee

REVIEW: https://review.gluster.org/21411 (md-cache: removexattr must be send to glusterfsd) posted (#2) for review on master by Kinglong Mee

REVIEW: https://review.gluster.org/21413 (md-cache: request cached xattrs at stat/fstat) posted (#2) for review on master by Kinglong Mee

REVIEW: https://review.gluster.org/21414 (md-cache: request GF_POSIX_ACL_* xattrs when mode/uid/gid changing) posted (#2) for review on master by Kinglong Mee

REVIEW: https://review.gluster.org/21415 (md-cache: request cached xattrs at getxattr/fgetxattr) posted (#2) for review on master by Kinglong Mee

REVIEW: https://review.gluster.org/21417 (posix: return more xattrs at getxattr/fgetxattr when client request) posted (#2) for review on master by Kinglong Mee

REVIEW: https://review.gluster.org/21416 (posix: fill glusterfs.posix.* acl xattrs at dict) posted (#2) for review on master by Kinglong Mee

REVIEW: https://review.gluster.org/21418 (md-cache: new option \"cache-glusterfs-acl\" for virtual glusterfs ACLs) posted (#2) for review on master by Kinglong Mee

COMMIT: https://review.gluster.org/21417 committed in master by "Amar Tumballi" <amarts> with a commit message- posix: return more xattrs at getxattr/fgetxattr when client request

Change-Id: I37ac6186b3631979d2503d1b185a61b8094dbd0d
Updates: bz#1634220
Signed-off-by: Kinglong Mee <mijinlong>

REVIEW: https://review.gluster.org/21449 (md-cache: fix dict leak of cached xattr) posted (#1) for review on master by Kinglong Mee

COMMIT: https://review.gluster.org/21449 committed in master by "Amar Tumballi" <amarts> with a commit message- md-cache: fix dict leak of cached xattr

Change-Id: I52f8e13e68528ba9679537ffdddf58ec08f9fd0c
Updates: bz#1634220
Signed-off-by: Kinglong Mee <mijinlong>

COMMIT: https://review.gluster.org/21416 committed in master by "Amar Tumballi" <amarts> with a commit message- posix: fill glusterfs.posix.* acl xattrs at dict

Change-Id: I0730a037f96c4386c72ecf2f61c71ec17ffbc1b0
Updates: bz#1634220
Signed-off-by: Kinglong Mee <mijinlong>

COMMIT: https://review.gluster.org/21413 committed in master by "Poornima G" <pgurusid> with a commit message- md-cache: request cached xattrs at stat/fstat

Ganesha always operate file by filehandle, and translates to
glusterfs's stat/fstat many time.

Change-Id: Idd0dc33c31131331ac948754c8b7f898777c31d3
Updates: bz#1634220
Signed-off-by: Kinglong Mee <mijinlong>

REVIEW: https://review.gluster.org/21411 (md-cache: removexattr must be send to glusterfsd) posted (#10) for review on master by Amar Tumballi

REVIEW: https://review.gluster.org/21410 (md-cache: remove xattr setting after new file created) posted (#10) for review on master by Raghavendra G

REVIEW: https://review.gluster.org/21412 (md-cache: extends xa_time when previous attributes equals cached attributes) posted (#10) for review on master by Raghavendra G

REVIEW: https://review.gluster.org/21415 (md-cache: request cached xattrs at getxattr/fgetxattr) posted (#11) for review on master by Raghavendra G

REVIEW: https://review.gluster.org/21418 (md-cache: new option \"cache-glusterfs-acl\" for virtual glusterfs ACLs) posted (#10) for review on master by Raghavendra G

REVIEW: https://review.gluster.org/21414 (md-cache: request ACLs related xattrs when mode change) posted (#10) for review on master by Amar Tumballi

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-6.0, please open a new bug report.

glusterfs-6.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://lists.gluster.org/pipermail/announce/2019-March/000120.html
[2] https://www.gluster.org/pipermail/gluster-users/