Hide Forgot
Description of problem: From customer: On any of our diskless systems that are running RHEL 7.9, with kernel "kernel-3.10.0-1160.11.1.el7.x86_64", Any share mounted via NFS V4.2 is broken. Our diskless nodes have SE Linux enabled, and are exporting SE Linux permissions as part of these NFS shares, which is why we need V4.2. When we run an "ls" command on a mounted NFS share, there are no results, so things like shell command tab completion do not work. However, if we run an ls command and specify a direct file on a mounted NFS share, it does show. If we mount our NFS shares with an older NFS version, we are able to see our files. However, older NFS versions do not support SE Linux, therefore this is not a solution for us. We believe that the change in the kernel for CVE-2020-25212 is the culprit here. We reverted back to kernel-3.1.0-1160.6.1 and do not see this issue occur. We investigated further, and rolled back the changes to the two files listed in this CVE, and recompiled kernel-3.10.0-1160.11.1, and no longer see the issue. Version-Release number of selected component (if applicable): RHEL 7.9 with kernel-3.10.0-1160.11.1.el7.x86_64 NFS4.2 with selinux / labelled NFS enabled How reproducible: Everytime Steps to Reproduce: TBD bug see customer description above Actual results: directory listings no longer work Expected results: directory listings work Additional info: When does the behavior occur? Frequency? Repeatedly? At certain times? This issue occurs with any NFS V4.2 share on kernel-3.10.0-1160.11.1.el7.x86_64. What information can you provide around timeframes and the business impact? Reverting the changes to CVE-2020-25212 resolves our issues, but in theory reopens our systems to exposure to CVE-2020-25212.
Probably we need the below patch. Suggest creating a test kernel with customer's version plus this patch and see if the problem is resolved. If it is this is probably going to be a mess since we have other RHEL7 streams with this CVE patch (see https://bugzilla.redhat.com/show_bug.cgi?id=1877575#c20) commit d33030e2ee3508d65db5644551435310df86010e Author: Jeffrey Mitchell <jeffrey.mitchell@starlab.io> Date: Tue Sep 15 16:42:52 2020 -0500 nfs: Fix security label length not being reset nfs_readdir_page_filler() iterates over entries in a directory, reusing the same security label buffer, but does not reset the buffer's length. This causes decode_attr_security_label() to return -ERANGE if an entry's security label is longer than the previous one's. This error, in nfs4_decode_dirent(), only gets passed up as -EAGAIN, which causes another failed attempt to copy into the buffer. The second error is ignored and the remaining entries do not show up in ls, specifically the getdents64() syscall. Reproduce by creating multiple files in NFS and giving one of the later files a longer security label. ls will not see that file nor any that are added afterwards, though they will exist on the backend. In nfs_readdir_page_filler(), reset security label buffer length before every reuse Signed-off-by: Jeffrey Mitchell <jeffrey.mitchell@starlab.io> Fixes: b4487b935452 ("nfs: Fix getxattr kernel panic and memory overflow") Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index e732580fe47b..cb52db9a0cfb 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -579,6 +579,9 @@ int nfs_readdir_page_filler(nfs_readdir_descriptor_t *desc, struct nfs_entry *en xdr_set_scratch_buffer(&stream, page_address(scratch), PAGE_SIZE); do { + if (entry->label) + entry->label->len = NFS4_MAXLABELLEN; + status = xdr_decode(desc, entry, &stream); if (status != 0) { if (status == -EAGAIN)
create 2 files, with the second having a longer-length label: # echo file 1 > file_1 ; chcon -t public_context_1 file_1 # echo file 2 > file_2 ; chcon -t public_context__2 file_2 # ls -lZ -rw-r--r--. root root unconfined_u:object_r:public_context_1:s0 file_1 -rw-r--r--. root root unconfined_u:object_r:public_context__2:s0 file_2 and over nfs: # ls -l total 4 -rw-r--r--. 1 root root 7 Jan 18 11:22 file_1 only the first file is shown in the directory listing, however both can be accessed: # cat file_1 file 1 # cat file_2 file 2
Created attachment 1748508 [details] v1 of reproducer for this bug - could use a bit of cleanup but works to show the problem Reproducer for this bug. Note you only need: - mount with "vers=4.2" and an export with "security_label" option - 2 files on the export, the second one with a label longer than the first [root@rhel7u9-node1 1917504]# ./t0_bz1917504.sh Assuming NFS vers=4.2 and filesystem xfs and mount options rw Mon Jan 18 13:24:40 EST 2021: 0. On NFS server, export one subdirectory with specific filesystem and mount opts /dev/loop0 /export/bz1917504-export xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 Mon Jan 18 13:24:40 EST 2021: 1. On NFS server, create two files in the export, with different size labels Mon Jan 18 13:24:40 EST 2021: Running command: echo 1 > /export/bz1917504-export/file1 Mon Jan 18 13:24:40 EST 2021: Running command: echo 2 > /export/bz1917504-export/file2 Mon Jan 18 13:24:40 EST 2021: Running command: semanage fcontext -a -t public_content_t /export/bz1917504-export/file2 Mon Jan 18 13:24:45 EST 2021: Running command: restorecon -F -R -v /export/bz1917504-export restorecon reset /export/bz1917504-export context system_u:object_r:unlabeled_t:s0->system_u:object_r:usr_t:s0 restorecon reset /export/bz1917504-export/file1 context unconfined_u:object_r:unlabeled_t:s0->system_u:object_r:usr_t:s0 restorecon reset /export/bz1917504-export/file2 context unconfined_u:object_r:unlabeled_t:s0->system_u:object_r:public_content_t:s0 Mon Jan 18 13:24:45 EST 2021: 2. On NFS client, mount -o vers=4.2, 127.0.0.1:/export/bz1917504-export /mnt/bz1917504-mnt Mon Jan 18 13:24:45 EST 2021: mount -o vers=4.2 127.0.0.1:/export/bz1917504-export /mnt/bz1917504-mnt Mon Jan 18 13:24:47 EST 2021: 3. On NFS client, list the directory and see if both files show up Mon Jan 18 13:24:47 EST 2021: directory listing showed 1 != 2 FAILED TEST ./t0_bz1917504.sh on kernel 3.10.0-1160.11.1.el7.x86_64 with NFS vers=4.2 [root@rhel7u9-node1 1917504]# REBOOT to older kernel: [root@rhel7u9-node1 1917504]# ./t0_bz1917504.sh Assuming NFS vers=4.2 and filesystem xfs and mount options rw Trying to start nfs-server Mon Jan 18 13:30:01 EST 2021: 0. On NFS server, export one subdirectory with specific filesystem and mount opts /dev/loop0 /export/bz1917504-export xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 Mon Jan 18 13:30:01 EST 2021: 1. On NFS server, create two files in the export, with different size labels Mon Jan 18 13:30:01 EST 2021: Running command: echo 1 > /export/bz1917504-export/file1 Mon Jan 18 13:30:01 EST 2021: Running command: echo 2 > /export/bz1917504-export/file2 Mon Jan 18 13:30:01 EST 2021: Running command: semanage fcontext -a -t public_content_t /export/bz1917504-export/file2 Mon Jan 18 13:30:07 EST 2021: Running command: restorecon -F -R -v /export/bz1917504-export restorecon reset /export/bz1917504-export context system_u:object_r:unlabeled_t:s0->system_u:object_r:usr_t:s0 restorecon reset /export/bz1917504-export/file1 context unconfined_u:object_r:unlabeled_t:s0->system_u:object_r:usr_t:s0 restorecon reset /export/bz1917504-export/file2 context unconfined_u:object_r:unlabeled_t:s0->system_u:object_r:public_content_t:s0 Mon Jan 18 13:30:07 EST 2021: 2. On NFS client, mount -o vers=4.2, 127.0.0.1:/export/bz1917504-export /mnt/bz1917504-mnt Mon Jan 18 13:30:07 EST 2021: mount -o vers=4.2 127.0.0.1:/export/bz1917504-export /mnt/bz1917504-mnt Mon Jan 18 13:30:10 EST 2021: 3. On NFS client, list the directory and see if both files show up PASSED TEST ./t0_bz1917504.sh on kernel 3.10.0-1160.6.1.el7.x86_64 with NFS vers=4.2 [root@rhel7u9-node1 1917504]#
[root@rhel7u9-node1 1917504]# ./t0_bz1917504.sh Assuming NFS vers=4.2 and filesystem xfs and mount options rw Trying to start nfs-server Mon Jan 18 13:56:11 EST 2021: 0. On NFS server, export one subdirectory with specific filesystem and mount opts /dev/loop0 /export/bz1917504-export xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0 Mon Jan 18 13:56:12 EST 2021: 1. On NFS server, create two files in the export, with different size labels Mon Jan 18 13:56:12 EST 2021: Running command: echo 1 > /export/bz1917504-export/file1 Mon Jan 18 13:56:12 EST 2021: Running command: echo 2 > /export/bz1917504-export/file2 Mon Jan 18 13:56:12 EST 2021: Running command: semanage fcontext -a -t public_content_t /export/bz1917504-export/file2 Mon Jan 18 13:56:17 EST 2021: Running command: restorecon -F -R -v /export/bz1917504-export restorecon reset /export/bz1917504-export context system_u:object_r:unlabeled_t:s0->system_u:object_r:usr_t:s0 restorecon reset /export/bz1917504-export/file1 context unconfined_u:object_r:unlabeled_t:s0->system_u:object_r:usr_t:s0 restorecon reset /export/bz1917504-export/file2 context unconfined_u:object_r:unlabeled_t:s0->system_u:object_r:public_content_t:s0 Mon Jan 18 13:56:17 EST 2021: 2. On NFS client, mount -o vers=4.2, 127.0.0.1:/export/bz1917504-export /mnt/bz1917504-mnt Mon Jan 18 13:56:17 EST 2021: mount -o vers=4.2 127.0.0.1:/export/bz1917504-export /mnt/bz1917504-mnt Mon Jan 18 13:56:20 EST 2021: 3. On NFS client, list the directory and see if both files show up PASSED TEST ./t0_bz1917504.sh on kernel 3.10.0-1160.11.1.el7.bz1917504.1.x86_64 with NFS vers=4.2 [root@rhel7u9-node1 1917504]#
Created attachment 1749129 [details] v2 of reproducer for this bug Small tweaks to run on rhel8 as well as rhel7 - use "ls -1Z" - always show the output of the listing - in cleanup_exit remove the label with "semanage fcontext -d"
Patch(es) committed on kernel-3.10.0-1160.15.2.el7
*** Bug 1916129 has been marked as a duplicate of this bug. ***
*** Bug 1912550 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: kernel security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:0336