Description of problem: To protect from node and service failures, it is desirable to deploy NFS-Ganesha in active/active highly available configurations. The Ceph community has designed an architecture which puts one or more nfs-ganesha servers behind an ingress service. The ingress service comprises of haproxy and keepalived [1]. While the solution works well, it is less secure because users cannot enforce client IP restrictions in this architecture. HAProxy terminates client connections and NFS-Ganesha only sees the traffic as originating from the HAProxy node/s, which would invalidate any IP address specified in the CLIENT block of NFS-Ganesha export configs. It would be feasible to use the IP address of the HAProxy node in the CLIENT block, but this will allow everyone with access to the HAProxy node to have access to the filesystem exported. One solution could be supporting the PROXY protocol [2][3] where packets originating from HAProxy would have an additional header that contains the source client's IP address. NFS-Ganesha could parse this IP address and return information directly to it. Open source projects that are benefactors of this solution would include Ceph, OpenStack Manila and Rook among a host of others. [1] https://docs.ceph.com/en/latest/cephadm/services/nfs/ [2] https://www.haproxy.com/blog/haproxy/proxy-protocol/ [3] http://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
From the perspective of the upstream effort for this, I will conduct most of the design through the github issue and a Google Doc I have started. We can use this for more internal/downstream considerations.
patch is posted upstream, will be in nfs-ganesha-5
Hi, When tried argument " --ingress-mode" while creating cephfs nfs cluster seeing below error [root@ceph-amk-fs-tools-dcnfas-node7 ~]# ceph nfs cluster create cephfs --ingress --virtual-ip=10.0.209.126/30 --ingress-mode=haproxy-protocol Invalid command: Unexpected argument '--ingress-mode=haproxy-protocol' nfs cluster create <cluster_id> [<placement>] [--ingress] [--virtual_ip <value>] [--port <int>] : Create an NFS Cluster Error EINVAL: invalid command [root@ceph-amk-fs-tools-dcnfas-node7 ~]# ceph versions { "mon": { "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 3 }, "mgr": { "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 2 }, "osd": { "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 12 }, "mds": { "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 3 }, "overall": { "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 20 } } [root@ceph-amk-fs-tools-dcnfas-node7 ~]# Regards, Amarnath
Missed the 6.1 z1 window, retargeting it to 6.1 z2.