Description of problem:
Can not set ACLs on an NFSv4 mounted filesystem when the client is running a
vanilla RHEL or Fedora Core system.  Other Linux distros probably ship with the
same limitation.  The setfacl call fails with EOPTNOTSUPP.

Version-Release number of selected component (if applicable):
- Tested on RHEL 4.4, RHEL 5 Beta 2, and FC6.  FC6 kernel details:
    2.6.19-1.2911.fc6
    libacl-2.2.39-1.1
    acl-2.2.39-1.1

How reproducible:
- Every time

Steps to Reproduce:

> mount -t nfs4 proto-u38:/vol/vol0 /mnt
> cd /mnt
> touch foo
> getfacl foo
# file: foo
# owner: nobody
# group: nobody
user::rw-
group::r--
other::r--

> setfacl -m user:user1:rwx foo
setfacl: foo: Operation not supported

  
Actual results:
Fails with error: EOPNOTSUPP

Expected results:
Should succeed and set an Access Control Entry on file 'foo' for 'user1' giving
read/write/execute permission.

Additional info:
Tested against NetApp filer with NFSv4 enabled, and NFS ACL support enabled. 
NetApp filer is running ONTAP 7.2.1.  Any NFsv4 server will produce the same
behavior, since the call errs on the client without calling the server.

strace shows that getfacl("foo", ...) uses getxattr(2) to obtain the ACL 
extended attribute.  The default POSIX-style ACL tool expects the
name of the ACL extended attribute to be "system.posix_acl_access".

getxattr("foo", "system.posix_acl_access", 0xbff77d50, 132) =
-1 EOPNOTSUPP (Operation not supported)

This eventually traps into the kernel calling:
sys_getattr()->getxattr()->vfs_getattr()->nfs4_getxattr().

nfs4_getxattr() (in fs/nfs/nfs4proc.c) expects a different extended attribute 
name: "system.nfs4_acl", which differs from what is passed in, making it return
error: EOPNOTSUPP

fs/nfs/nfs4proc.c:
...
#define XATTR_NAME_NFSV4_ACL "system.nfs4_acl"
...
ssize_t nfs4_getxattr(struct dentry *dentry, const char *key, void *buf,
                size_t buflen)
{
    ...
        if (strcmp(key, XATTR_NAME_NFSV4_ACL) != 0)
                return -EOPNOTSUPP;
    ....
}

Citi has a set of patches that modify the library behavior to better map
POSIX ACLs to NFSv4 ACLs.  One of the many things it does is use 
"system.nfs4_acl".  It's not a perfect mapping, so they
recommend to use instead the nfs4-acl-tools which use NFSv4 ACLs
directly.  

Customers that require use of ACLs on NFSv4 filesystems, are forced to compile
these tools themselves (this after generating customer calls to their respective
support providers).

The CITI patches and libraries can be found at:
   http://www.citi.umich.edu/projects/nfsv4/linux/

RHEL 4.6 bug describes the same problem under bugzilla bug: 231118