Bug 2089166

Description of problem:

cephadm allows ingress service (haporxy+keepalived) to be deployed in front of NFS-ganesha service (a cluster of NFS-ganesha daemons) to provide HA stable virtual IP to NFS clients, https://docs.ceph.com/en/quincy/mgr/nfs/#ingress . 
One of the downsides of this HA NFS service setup is that the backend NFS servers cannot see the source client IPs, and can only see the proxy server's IP. 
So use-cases such as OpenStack manila, where the NFS-ganesha enforces client IP based authorization for export access cannot be met.

A HA model for NFS service was discussed earlier in https://pad.ceph.com/p/cephadm-nfs-ha (Option 2 and 2a) and https://www.spinics.net/lists/dev-ceph/msg03442.html (Option 5 and 6), where cephadm internally manages the virtual IP of ganesha servers. 
cephadm would set up one virtual IP per each ganesha daemon it deploys.
It would add or remove the virtual IP when creating or removing a ganesha server.

From https://www.spinics.net/lists/dev-ceph/msg03442.html,

"
single ganesha + single virtual IP
- 1 ganesha daemon
- 1 virtual IP that follows the ganesha daemon
- on failure, cephadm would deploy ganesha elsewhere + move virtual IP
- not implemented

multiple ganesha + multiple virtual IPs
- N ganesha daemons
- N virtual IPs
- requires ganesha changes to (1) make ganesha aware of peers and (2)
instruct clients to move around
- on failure, cephadm would deploy failed ganesha elsewhere + move
that virtual IP
- not implemented (in cephadm or ganesha)
"

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: