Bug 1483096 - [RFE] colocated containerized daemons should reserve system resources
Summary: [RFE] colocated containerized daemons should reserve system resources
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Ceph-Ansible
Version: 3.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: rc
: 3.0
Assignee: leseb
QA Contact: Vasishta
Bara Ancincova
Depends On:
Blocks: 1494421
TreeView+ depends on / blocked
Reported: 2017-08-18 18:12 UTC by Andrew Schoen
Modified: 2017-12-05 23:18 UTC (History)
16 users (show)

Fixed In Version: RHEL: ceph-ansible-3.0.0-0.1.rc7.el7cp Ubuntu: ceph-ansible_3.0.0~rc7-2redhat1
Doc Type: Enhancement
Doc Text:
.Colocation of containerized Ceph daemons With this release, you can colocate specific containerized Ceph daemons with OSD daemons on the same node. This approach significantly improves total cost of ownership (TCO) at small scale, reduces the minimum configuration from six nodes to three, makes upgrading more convenient, and provides better resource isolation. Also, each daemon has system resources reserved to avoid the "noisy neighbor" effect. For details, see the link:https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html-single/container_guide/#colocation-of-containerized-ceph-daemons[Colocation of Containerized Ceph Daemons] chapter in the Container Guide for Red Hat Ceph Storage 3.
Clone Of:
Last Closed: 2017-12-05 23:18:20 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Github ceph ceph-ansible pull 1859 None None None 2017-09-06 09:53:32 UTC
Red Hat Product Errata RHEA-2017:3388 normal SHIPPED_LIVE new container image: rhceph-3-rhel7 2017-12-06 02:43:29 UTC

Description Andrew Schoen 2017-08-18 18:12:30 UTC
We want to allow containerized deployments to co-locate daemons on the same host, with the containers reserving CPU and RAM resources to each.

This delivers to customers a significant improvement in TCO at the small scale, particularly as it eliminates the need to have standalone hosts for MON and RGW not justified by traffic at the small scale. It also reduces the minimum configuration from 6 nodes to three.

We have two constraints:

- deployment of containers via ceph-ansible should put in place dedicated CPU and RAM resources for each container.

- we only support one daemon in {RGW, IGW, MDS, RBD-mirror, NFS-Ganesha, (MON+MGR)} co-located per OSD host.

To repeat, cardinality is 1 per OSD host from the lit above. This gives an upper bound to QE effort with a 6 case test matrix.

Comment 2 Federico Lucifredi 2017-09-06 20:01:18 UTC
this was incorrectly assigned to 3.1, fixed.

Comment 9 leseb 2017-10-20 09:26:25 UTC
Looks good, I just added a small change. Thanks, Bara.

Comment 11 leseb 2017-10-27 08:28:02 UTC
How to verify?

1. Make sure the "docker run" command as the right flags
2. Run the tests mentioned above inside the container

If 2. fails this is most likely an issue with the Docker engine.

Comment 12 Vasishta 2017-10-30 09:02:36 UTC

Verified that docker run command is accompanied with right set of flags.
Moving to VERIFIED state based on e-mail conversation with Sebastien and Harish.


Comment 15 errata-xmlrpc 2017-12-05 23:18:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.