Description of problem: [RFE][HCI] set osd max memory based on osd container memory This is an extension or a fix for this bug: [RFE] set osd max memory based on host memory https://bugzilla.redhat.com/show_bug.cgi?id=1595003 https://github.com/ceph/ceph-ansible/pull/3113 Version-Release number of selected component (if applicable): RHCS 3.1 BZ1595003 handles this for non-containerized(non-HCI) environment pretty well but in the container(HCI) environment we should not consider complete OSD host/node memory because container OSD will only get 5G from option ceph_osd_docker_memory_limit. https://bugzilla.redhat.com/show_bug.cgi?id=1591876 https://github.com/ceph/ceph-ansible/pull/2775 - I had discussed with Neha for this and We think in HCI check it should be: - {% set _osd_memory_target = (ansible_memtotal_mb * hci_safety_factor / _num_osds) %} + {% set _osd_memory_target = (ceph_osd_docker_memory_limit * hci_safety_factor) %} - This ceph_osd_docker_memory_limit defaults to 5G and we are giving 4G to BlueStore cache by default because there is no code for having BlueStore cache less than 4G. - So either we need to change the default osd_memory_target for the container(HCI) to less than 4G or we need to bump the default for ceph_osd_docker_memory_limit to 8G or something? Maybe we need to talk to the performance team for these defaults. We also need to think about the hci_safety_factor default value. The reason for all this is the container OSD will not have access to memory more than ceph_osd_docker_memory_limit configuration.
You have to set the container CGRoup memory limit based on size of OSD cache rather than what you said above here? So add a fixed amount to the OSD cache size and make that the CGroup limit, right?
During Bluestore discussion we came up with this idea to prevent the OSD (or other daemon) from getting into a situation where the CGroup memory limit for a Ceph daemon container was less than the amount of memory that it needed. During daemon startup, if it can determine the container ID (available from "docker inspect container-name" for example), it can get the memory limit at /sys/fs/cgroup/memory/system.slice/docker-$containerid.scope/memory.limit_in_bytes and exit with an error message if this is not sufficient (example: Bluestore OSDs that do caching in userspace and need lots of memory). This doesn't fix orchestration software or prevent the daemon from consuming too much memory, it just ensures that we don't get into a situation where the daemon is likely to die just because the CGroup limit was accidentally set too low, and informs the sysadmin of this right away.
Present in https://github.com/ceph/ceph-ansible/releases/tag/v3.2.0rc1
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0020