Bug 1931565 - NFS server should enable RDMA by default
Summary: NFS server should enable RDMA by default
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: nfs-utils
Version: rawhide
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Steve Dickson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-02-22 17:01 UTC by Chuck Lever
Modified: 2022-02-04 04:26 UTC (History)
3 users (show)

Fixed In Version: nfs-utils-2.5.3-2.rc1.fc34
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1939629 (view as bug list)
Environment:
Last Closed: 2021-03-31 00:15:28 UTC
Type: Bug


Attachments (Terms of Use)

Description Chuck Lever 2021-02-22 17:01:41 UTC
Description of problem:
Product is shipped with NFS/RDMA disabled by default. An extra step is needed when setting up an NFS server to support NFS/RDMA clients.

Version-Release number of selected component (if applicable):
All Fedora releases to date that support NFS/RDMA.

This request is to change the default setting in /etc/nfs.conf from rdma=n to rdma=y.

[nfsd]
# debug=0
threads=16
# host=
# port=0
# grace-time=90
# lease-time=90
# udp=n
# tcp=y
# vers2=n
# vers3=y
# vers4=y
# vers4.0=y
# vers4.1=y
# vers4.2=y
rdma=y
rdma-port=20049

Comment 1 Chuck Lever 2021-02-22 17:03:21 UTC
Bruce Fields further comments that before changing this setting, we should investigate possible inadvertent exposure of server memory when NFS/RDMA is used with iWARP on open networks.

Comment 2 J. Bruce Fields 2021-03-09 15:15:08 UTC
I was going on a very dim memory that one of the early memory registration mechanisms by design exposed more memory than necessary.  I assume that's not a problem any more (but it'd be good to have confirmation).

How much configuration is still left for the user to do after this is defaulted to on?  (Do you still need -ordman,port=20049 on the client side?)

Are there any drawbacks to a non-rdma user to having this on?  Does it use any resources beyond the one listening socket?  Any common setup where rdma might be available but broken somehow, and making it available would cause mounts to fail?  (I assume not, if it still requires explicit configuration on the client.)

It sounds like a good idea, I just want to make sure we haven't overlooked anything.

Comment 3 Chuck Lever 2021-03-09 15:26:47 UTC
(In reply to J. Bruce Fields from comment #2)
> I was going on a very dim memory that one of the early memory registration
> mechanisms by design exposed more memory than necessary.  I assume that's
> not a problem any more (but it'd be good to have confirmation).

The Linux NFS/RDMA client implementation used to have a registration mode that used the same R_key for all of its memory. That mode was removed long ago. The server does not expose its memory.

The memory registration mechanism ensures that the only memory that is exposed on the client is the region that is part of a pending RPC/RDMA transaction. That registration is invalidated once the Reply has been received.


> How much configuration is still left for the user to do after this is
> defaulted to on?  (Do you still need -ordman,port=20049 on the client side?)

Contemporary NFS/RDMA clients need only "proto=rdma" or just "rdma".


> Are there any drawbacks to a non-rdma user to having this on?  Does it use
> any resources beyond the one listening socket?

A listener is created, but other resources are not provisioned until the server accepts a connection.


> Any common setup where rdma
> might be available but broken somehow, and making it available would cause
> mounts to fail?  (I assume not, if it still requires explicit configuration
> on the client.)

I can't think of any problem scenarios. Solaris NFS servers enable RDMA by default, fwiw.


> It sounds like a good idea, I just want to make sure we haven't overlooked
> anything.

Enabling this setting in Rawhide/Fedora is one way to find out if there is an issue we haven't foreseen.

Comment 4 J. Bruce Fields 2021-03-09 15:29:15 UTC
Sounds good, let's do it.

Comment 5 Fedora Update System 2021-03-16 18:05:27 UTC
FEDORA-2021-e31a53a752 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-e31a53a752

Comment 6 Fedora Update System 2021-03-16 23:30:36 UTC
FEDORA-2021-e31a53a752 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-e31a53a752`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-e31a53a752

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2021-03-31 00:15:28 UTC
FEDORA-2021-e31a53a752 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 8 Fujitsu kernel team 2022-02-04 04:26:09 UTC
An objection.

Following additional kernel modules are loaded automatically even if the NFS server never use RDMA:

 ib_cm
 ib_core
 iw_cm
 rdma_cm
 rpcrdma

And Red Hat CVE Database shows following major vulnerabilities for RDMA:

 CVE-2021-4028
 CVE-2020-36385
 CVE-2016-4565

It just looks the security risk increased for non-RDMA users.
I wonder how many users receive benefits from this enablement by default.

Regards,
Ikarashi


Note You need to log in before you can comment on or make changes to this bug.