Bug 1625955

Operations done on the server:

# tcpdump -v -w mount_export_wait.pcap port 2049

-----------------------------------------------------------------------------

Operations done on the client:

[root@vm-nfs7-client ~]# mount -o vers=4.2 vm-nfs7-server:/export /mnt
[root@vm-nfs7-client ~]# date; getfattr --absolute-names -n security.selinux /mnt
Thu Sep  6 10:14:51 CEST 2018
# file: /mnt
security.selinux="system_u:object_r:unlabeled_t:s0"

[root@vm-nfs7-client ~]# date; getfattr --absolute-names -n security.selinux /mnt
Thu Sep  6 10:15:10 CEST 2018
# file: /mnt
security.selinux="system_u:object_r:usr_t:s0"

-----------------------------------------------------------------------------

Analysis of tcpdump (mount_export_wait.pcap):

- Mount was done at 10:14:36.887
- Packet 46+47 at 10:14:36.915 show expected label "usr_t" being transmitted
- *But* first getfattr from client, which happens at 10:14:51 shows "unlabeled_t"; no NFS traffic at that time
- second getfattr from client, which happens at 10:15:10 shows "usr_t"; Packet 49+50 show expected label "usr_t" being transmitted

Looks like to me there is some "caching" made on the client side, with non-existent label first.
Later, once cache expires, getfattr produces NFS traffic, and result seems also cached somehow for some time.

How about 4.1 or 4.0 ?

(In reply to Xiong Murphy Zhou from comment #3)
> How about 4.1 or 4.0 ?

The "NFSv4 Security Labels" is introduced by NFS version 4.2 and IMO this feature doesn't apply for 4.1 or 4.0 for now.

I'll generate some more tests about the labeled-nfs later.

I cannot reproduce this problem using BZ 1624848

[06:07:31 root@ ~~]# mount -o vers=4.2 nfs_server:/exportdir-1624848 /mnt/nfsmp1-1624848
[06:07:31 root@ ~~]# ls -lZ /mnt/nfsmp1-1624848/testfile
-rw-r--r--. root root unconfined_u:object_r:usr_t:s0   /mnt/nfsmp1-1624848/testfile
[06:07:31 root@ ~~]# getfattr --absolute-names -n security.selinux /mnt/nfsmp1-1624848/testfile
# file: /mnt/nfsmp1-1624848/testfile
security.selinux="unconfined_u:object_r:usr_t:s0"
                                       ^^^^^^^
So I'm thinking that, in your env, the "ls -lZ" is listing the wrong security label either, right?

"unlabeled_t" is seen for top dir only ("/mnt/nfsmp1-1624848").
"ls -lZ" shows wrong label of course, but I used "getfattr" to read "raw" content, just in case.

(In reply to Renaud Métrich from comment #6)
> "unlabeled_t" is seen for top dir only ("/mnt/nfsmp1-1624848").
> "ls -lZ" shows wrong label of course, but I used "getfattr" to read "raw"
> content, just in case.

Hi Renaud, thanks for your hint.

Now I'm checking the top dir and finding it's "default_t" using "getfattr".
Curiously, it shows correctly "user_t" when using "ls -lZ".
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[04:02:05 root@ ~~]# mount -o vers=4.2 hp-dl385pg8-03.rhts.eng.pek2.redhat.com:/exportdir-1624848 /mnt/nfsmp1-1624848
[04:02:05 root@ ~~]# ls -lZ /mnt/nfsmp1-1624848
-rw-r--r--. root root unconfined_u:object_r:usr_t:s0   testfile
                                           ^^^^^^^
[04:02:05 root@ ~~]# getfattr --absolute-names -n security.selinux /mnt/nfsmp1-1624848
# file: /mnt/nfsmp1-1624848
security.selinux="unconfined_u:object_r:default_t:s0"
                                       ^^^^^^^^^^^

Hoever, if it't not the top dir, the 2 outputs are the same:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[04:02:05 root@ ~~]# ls -lZ /mnt/nfsmp1-1624848/testfile
-rw-r--r--. root root unconfined_u:object_r:usr_t:s0   /mnt/nfsmp1-1624848/testfile
[04:02:05 root@ ~~]# ls -lZ /mnt/nfsmp1-1624848/testfile | grep -q usr_t
[04:02:05 root@ ~~]# getfattr --absolute-names -n security.selinux /mnt/nfsmp1-1624848/testfile
# file: /mnt/nfsmp1-1624848/testfile
security.selinux="unconfined_u:object_r:usr_t:s0"


Can you still be able to reproduce this problem reliably? Or is there something wrong with my steps? Thanks!

I can reproduce.

NFS server:

[root@vm-nfs7-server ~]# rpm -qa nfs-utils
nfs-utils-1.3.0-0.54.el7.x86_64
[root@vm-nfs7-server ~]# uname -r
3.10.0-862.11.6.el7.x86_64

[root@vm-nfs7-server ~]# cat /etc/exports
/export *(rw,no_root_squash,no_wdelay,sync,security_label)

[root@vm-nfs7-server ~]# grep RPCNFSDARGS /etc/sysconfig/nfs 
RPCNFSDARGS="-V 4.2"


NFS client:

[root@vm-nfs7-client ~]# rpm -qa nfs-utils
nfs-utils-1.3.0-0.54.el7.x86_64
[root@vm-nfs7-client ~]# uname -r
3.10.0-862.11.6.el7.x86_64

[root@vm-nfs7-client ~]# ls -Zd /mnt
drwxr-xr-x. root root system_u:object_r:mnt_t:s0       /mnt

[root@vm-nfs7-client ~]# mount -o vers=4.2 vm-nfs7-server:/export /mnt
[root@vm-nfs7-client ~]# date; getfattr --absolute-names -n security.selinux /mnt
Thu Sep 13 10:42:33 CEST 2018
# file: /mnt
security.selinux="system_u:object_r:unlabeled_t:s0"

... WAIT ...

[root@vm-nfs7-client ~]# date; getfattr --absolute-names -n security.selinux /mnt
Thu Sep 13 10:43:03 CEST 2018
# file: /mnt
security.selinux="system_u:object_r:usr_t:s0"

(In reply to Renaud Métrich from comment #8)

Hi Renaud, yes I can reproduce it now!

I tried it setting nfs server and client in the same machine before. And looks like this need to use 2 separate machines.

Thanks for your help.

Indeed, all the BZs have been filed when using 2 systems.

Test kernel from BZ #1624848 (https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=20475583) DOESN'T fix this issue:

# uname -a
Linux vm-nfsclient7 3.10.0-1014.el7.jbf.e48008f02c26.x86_64 #1 SMP Wed Mar 6 12:14:02 EST 2019 x86_64 x86_64 x86_64 GNU/Linux

# mount -o vers=4.2 vm-nfsserver7:/export /mnt && date && getfattr --absolute-names -n security.selinux /mnt
Thu Mar  7 09:36:30 CET 2019
# file: /mnt
security.selinux="system_u:object_r:unlabeled_t:s0"

# date && getfattr --absolute-names -n security.selinux /mnt
Thu Mar  7 09:36:36 CET 2019
# file: /mnt
security.selinux="system_u:object_r:unlabeled_t:s0"

... WAIT ...

# date && getfattr --absolute-names -n security.selinux /mnt
Thu Mar  7 09:37:04 CET 2019
# file: /mnt
security.selinux="unconfined_u:object_r:usr_t:s0"

Have you checked whether this is still reproduceable when /export is on a separate filesystem?

Yes, no change.

Argh, and I can reproduce this one upstream as well.  Thanks for your persistence.

Unlike other similar bugs, the nfs server caps are set correctly:

# grep cap /proc/self/mountstats
	caps:	caps=0x3ffffff,wtmult=512,dtsize=32768,bsize=0,namlen=255

Labels on files other than the mountpoint are correct from the start.

Right, only the mount point is affected

Please see RHEL8 version of this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1660798

RHEL8 bug isn't open so can't view or comment there.  Anyway, I asked the following question on the mailing list discussion of this bug and never saw an answer:

When does NFS set the security label for the root inode / mounted directory (this would be done via nfs_setsecurity() in the labeled NFS case)?  This needs to happen before it is passed to any permission calls or exposed to userspace.

For a local filesystem using xattrs (SECURITY_FS_USE_XATTR in SELinux), the initialization of the security label of the root inode happens during security_sb_set_mnt_opts() -> selinux_set_mnt_opts() -> sb_finish_set_opts() -> inode_doinit_with_dentry(root_inode, root).  In that situation, SELinux calls __vfs_getxattr() to fetch the security.selinux attribute, map it to a SID, and set it in the incore inode security structure.  However, for labeled NFS, which uses "native labeling" aka SECURITY_FS_USE_NATIVE within SELinux aka SECURITY_LSM_NATIVE_LABELS, we need NFS to pass the label into SELinux rather than having SELinux fetch an xattr since MAC labels are directly supported in the protocol and not merely extended or named attributes. This would normally happen via nfs_setsecurity() -> security_inode_notifysecctx() -> selinux_inode_notifysecctx().  However, it isn't clear to me that this occurs for the root inode prior to any further use of it by NFS.