Description of problem:
Following errors happening on customer side and OSDs keep down/up on all osd-nodes.
filestore(/var/lib/ceph/osd/ceph-10) error (24) Too many open files not handled on operation 0x55dfec678a00 (805947768.0.0, or op 0, counting from 0)
checking nofile for ceph-osd container:
$ cat sos_commands/docker/docker_inspect_f2863fa486e0 | grep limit -A5
"Ulimits": [
{
"Name": "nofile",
"Hard": 4096,
"Soft": 1024
}
Version-Release number of selected component (if applicable):
$ rpm -qa | grep tripleo
ansible-tripleo-ipsec-8.1.1-0.20190513184007.7eb892c.el7ost.noarch
openstack-tripleo-heat-templates-8.3.1-87.el7ost.noarch
puppet-tripleo-8.4.1-27.el7ost.noarch
openstack-tripleo-image-elements-8.0.2-2.el7ost.noarch
openstack-tripleo-common-containers-8.6.8-16.el7ost.noarch
python-tripleoclient-9.2.7-11.el7ost.noarch
openstack-tripleo-puppet-elements-8.0.2-3.el7ost.noarch
openstack-tripleo-validations-8.4.5-2.el7ost.noarch
openstack-tripleo-ui-8.3.2-3.el7ost.noarch
openstack-tripleo-common-8.6.8-16.el7ost.noarch
How reproducible:
Deploy cephstorage nodes with director
Steps to Reproduce:
1.
2.
3.
Actual results:
The nofile is 4096 for hard limit
Expected results:
The nofile should be bigger, like 1048576
Additional info:
Working fine with
ceph-ansible-3.2.40-1
ceph-3.3-rhel-7-containers-candidate-31439-20200312223408
$ sudo docker exec -it ceph-osd-3 bash
# ulimit -n
1048576
# ulimit -Hn
1048576
# ulimit -Sn
1048576
Moving to VERIFIED state, Please let us know if there are any concerns.
Regards,
Vasishta Shastry
QE, Ceph
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2020:1320
Description of problem: Following errors happening on customer side and OSDs keep down/up on all osd-nodes. filestore(/var/lib/ceph/osd/ceph-10) error (24) Too many open files not handled on operation 0x55dfec678a00 (805947768.0.0, or op 0, counting from 0) checking nofile for ceph-osd container: $ cat sos_commands/docker/docker_inspect_f2863fa486e0 | grep limit -A5 "Ulimits": [ { "Name": "nofile", "Hard": 4096, "Soft": 1024 } Version-Release number of selected component (if applicable): $ rpm -qa | grep tripleo ansible-tripleo-ipsec-8.1.1-0.20190513184007.7eb892c.el7ost.noarch openstack-tripleo-heat-templates-8.3.1-87.el7ost.noarch puppet-tripleo-8.4.1-27.el7ost.noarch openstack-tripleo-image-elements-8.0.2-2.el7ost.noarch openstack-tripleo-common-containers-8.6.8-16.el7ost.noarch python-tripleoclient-9.2.7-11.el7ost.noarch openstack-tripleo-puppet-elements-8.0.2-3.el7ost.noarch openstack-tripleo-validations-8.4.5-2.el7ost.noarch openstack-tripleo-ui-8.3.2-3.el7ost.noarch openstack-tripleo-common-8.6.8-16.el7ost.noarch How reproducible: Deploy cephstorage nodes with director Steps to Reproduce: 1. 2. 3. Actual results: The nofile is 4096 for hard limit Expected results: The nofile should be bigger, like 1048576 Additional info: