Bug 2097490

Summary:	[RFE] Support HAProxy's PROXY protocol
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Goutham Pacha Ravi <gouthamr>
Component:	NFS-Ganesha	Assignee:	Frank Filz <ffilz>
Status:	ASSIGNED ---	QA Contact:	Vidushi Mishra <vimishra>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	5.2	CC:	amk, cephqe-warriors, ffilz, gouthamr, kdreyer, kkeithle, mbenjamin, mkasturi, sostapov, tchandra, tserlin, vimartin
Target Milestone:	---	Keywords:	FutureFeature
Target Release:	6.1z2	Flags:	amk: needinfo? (ffilz)
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:	nfs-ganesha-5.1-1.el9cp	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2176300, 2024129

Description Goutham Pacha Ravi 2022-06-15 19:28:03 UTC

Description of problem:

To protect from node and service failures, it is desirable to deploy NFS-Ganesha in active/active highly available configurations. The Ceph community has designed an architecture which puts one or more nfs-ganesha servers behind an ingress service. The ingress service comprises of haproxy and keepalived [1].

While the solution works well, it is less secure because users cannot enforce client IP restrictions in this architecture. HAProxy terminates client connections and NFS-Ganesha only sees the traffic as originating from the HAProxy node/s, which would invalidate any IP address specified in the CLIENT block of NFS-Ganesha export configs.

It would be feasible to use the IP address of the HAProxy node in the CLIENT block, but this will allow everyone with access to the HAProxy node to have access to the filesystem exported.

One solution could be supporting the PROXY protocol [2][3] where packets originating from HAProxy would have an additional header that contains the source client's IP address. NFS-Ganesha could parse this IP address and return information directly to it.

Open source projects that are benefactors of this solution would include Ceph, OpenStack Manila and Rook among a host of others.

[1] https://docs.ceph.com/en/latest/cephadm/services/nfs/
[2] https://www.haproxy.com/blog/haproxy/proxy-protocol/
[3] http://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

Comment 1 Frank Filz 2022-06-16 18:09:36 UTC

From the perspective of the upstream effort for this, I will conduct most of the design through the github issue and a Google Doc I have started.

We can use this for more internal/downstream considerations.

Comment 5 Kaleb KEITHLEY 2023-01-24 22:22:16 UTC

patch is posted upstream, will be in nfs-ganesha-5

Comment 15 Amarnath 2023-06-06 10:22:35 UTC

Hi,

When tried argument " --ingress-mode" while creating cephfs nfs cluster seeing below error

[root@ceph-amk-fs-tools-dcnfas-node7 ~]# ceph nfs cluster create cephfs --ingress --virtual-ip=10.0.209.126/30 --ingress-mode=haproxy-protocol
Invalid command: Unexpected argument '--ingress-mode=haproxy-protocol'
nfs cluster create <cluster_id> [<placement>] [--ingress] [--virtual_ip <value>] [--port <int>] :  Create an NFS Cluster
Error EINVAL: invalid command
[root@ceph-amk-fs-tools-dcnfas-node7 ~]# ceph versions
{
    "mon": {
        "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 3
    },
    "mgr": {
        "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 2
    },
    "osd": {
        "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 12
    },
    "mds": {
        "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 3
    },
    "overall": {
        "ceph version 17.2.6-70.el9cp (fe62dcdbb2c6e05782a3e2b67d025b84ff5047cc) quincy (stable)": 20
    }
}
[root@ceph-amk-fs-tools-dcnfas-node7 ~]#

Regards,
Amarnath

Comment 20 Scott Ostapovicz 2023-07-12 13:34:10 UTC

Missed the 6.1 z1 window, retargeting it to 6.1 z2.