.Colocation of containerized Ceph daemons
With this release, you can colocate specific containerized Ceph daemons with OSD daemons on the same node. This approach significantly improves total cost of ownership (TCO) at small scale, reduces the minimum configuration from six nodes to three, makes upgrading more convenient, and provides better resource isolation. Also, each daemon has system resources reserved to avoid the "noisy neighbor" effect.
For details, see the link:https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html-single/container_guide/#colocation-of-containerized-ceph-daemons[Colocation of Containerized Ceph Daemons] chapter in the Container Guide for Red Hat Ceph Storage 3.
We want to allow containerized deployments to co-locate daemons on the same host, with the containers reserving CPU and RAM resources to each.
This delivers to customers a significant improvement in TCO at the small scale, particularly as it eliminates the need to have standalone hosts for MON and RGW not justified by traffic at the small scale. It also reduces the minimum configuration from 6 nodes to three.
We have two constraints:
- deployment of containers via ceph-ansible should put in place dedicated CPU and RAM resources for each container.
- we only support one daemon in {RGW, IGW, MDS, RBD-mirror, NFS-Ganesha, (MON+MGR)} co-located per OSD host.
To repeat, cardinality is 1 per OSD host from the lit above. This gives an upper bound to QE effort with a 6 case test matrix.
Comment 2Federico Lucifredi
2017-09-06 20:01:18 UTC
How to verify?
1. Make sure the "docker run" command as the right flags
2. Run the tests mentioned above inside the container
If 2. fails this is most likely an issue with the Docker engine.
Hi,
Verified that docker run command is accompanied with right set of flags.
Moving to VERIFIED state based on e-mail conversation with Sebastien and Harish.
Thanks.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHEA-2017:3388
We want to allow containerized deployments to co-locate daemons on the same host, with the containers reserving CPU and RAM resources to each. This delivers to customers a significant improvement in TCO at the small scale, particularly as it eliminates the need to have standalone hosts for MON and RGW not justified by traffic at the small scale. It also reduces the minimum configuration from 6 nodes to three. We have two constraints: - deployment of containers via ceph-ansible should put in place dedicated CPU and RAM resources for each container. - we only support one daemon in {RGW, IGW, MDS, RBD-mirror, NFS-Ganesha, (MON+MGR)} co-located per OSD host. To repeat, cardinality is 1 per OSD host from the lit above. This gives an upper bound to QE effort with a 6 case test matrix.