DescriptionGoutham Pacha Ravi
2019-08-22 21:56:57 UTC
Description of problem:
When CephFS-via-NFS is chosen as a driver with OpenStack/Manila; we have a choice of deploying Ceph alongside the OpenStack overcloud - in this configuration, we create pacemaker bundles for managing manila's share manager container and the ceph-nfs (NFS-Ganesha) container. Neither of these services (manila-share and nfs-ganesha) have been tested thoroughly when configured in a highly available active-active manner. This is why pacemaker is being used to run them in a highly available, but active-passive manner.
The pacemaker bundles have a collocation constraint applied to them [1]. This was done because, the CephFS-via-NFS driver in Manila communicates with nfs-ganesha over DBUS commands.
However, the driver does support using DBUS-over-SSH; it requires configuring a username, password/private-key to communicate to the ganesha host and issue DBUS commands. If we provide these details, there's no need for these containers to share a DBUS socket (currently, the controller host's DBUS socket is being mounted on both these containers [2]).
Impact of dropping the collocation constraint and using DBUS-over-SSH:
----------------------------------------------------------------------
- Provides a clean separation of concerns between the ceph-nfs and manila-share container.
- is more secure; since we're not exposing the DBUS socket file to the manila-share container
- allows for each process to have its own failure domain, and be moved independently
- when one of the processes fails, the other one is not impacted - this is critical for multi-backend with manila (See [3])
- allow each process to evolve independently to being served active-active (Rook and Kubernetes are being considered to orchestrate nfs-ganesha in an active-active configuration [4]; openstack-manila components have always supported active-active deployment; however, they haven't been qualified thoroughly against issues relating to concurrency of operations, but this is in the works)
Version-Release number of selected component (if applicable): Proposed for OSP 17, but changes may be appropriate for back port to long life releases.
[1] https://opendev.org/openstack/puppet-tripleo/src/commit/9aaaa42414a8a2026d715a12ffda1066ef95a543/manifests/profile/pacemaker/manila/share_bundle.pp#L228
[2] https://opendev.org/openstack/puppet-tripleo/src/commit/9aaaa42414a8a2026d715a12ffda1066ef95a543/manifests/profile/pacemaker/manila/share_bundle.pp#L180-L199
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1701074
[4] https://ceph.io/community/deploying-a-cephnfs-server-cluster-with-rook/