Bug 1240587
| Summary: | Unable to control the file_descriptors limit for rabbitmq-server via the director. | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Lee Yarwood <lyarwood> | |
| Component: | openstack-puppet-modules | Assignee: | Ivan Chavero <ichavero> | |
| Status: | CLOSED ERRATA | QA Contact: | Marius Cornea <mcornea> | |
| Severity: | urgent | Docs Contact: | ||
| Priority: | high | |||
| Version: | 7.0 (Kilo) | CC: | calfonso, dmacpher, fdinitto, gfidente, jslagle, lyarwood, mburns, mcornea, mmagr, mwagner, nbarcet, oblaut, rbiba, rhel-osp-director-maint, ushkalim, yeylon | |
| Target Milestone: | z2 | Keywords: | Triaged, ZStream | |
| Target Release: | 7.0 (Kilo) | |||
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | openstack-puppet-modules-2015.1.8-13.el7ost | Doc Type: | Bug Fix | |
| Doc Text: |
Previously, the director contained no direct mechanism to control the file_descriptors limit for rabbitmq-server because the file limits snippet was not installed on Red Hat Enterprise Linux 7, which uses systemd. With this update, the file limits snippet is installed as /etc/security/limits.d/rabbitmq-server.conf unconditionally, which allows users to control the limit.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1241628 1267883 (view as bug list) | Environment: | ||
| Last Closed: | 2015-10-08 12:23:19 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1240561 | |||
| Bug Blocks: | 1241628, 1267883 | |||
if it can help, you can also use this parameter[1] to customize the limit. [1] https://github.com/puppetlabs/puppetlabs-rabbitmq/blob/95498e4174915dbce81c81bd42d6ec3b87df14b3/manifests/init.pp#L63 *** Bug 1255091 has been marked as a duplicate of this bug. *** this is a blocker for a1 IMO, as any deployment of decent scale (20 nodes) is going to start bumping into the 1024 limit reposting from https://bugzilla.redhat.com/show_bug.cgi?id=1255091#c3 ... don't want it to get lost: upstream pr - https://github.com/puppetlabs/puppetlabs-rabbitmq/pull/381 I merged it. we still need the tripleo-heat-templates update to set this parameter *** Bug 1257398 has been marked as a duplicate of this bug. *** *** Bug 1257400 has been marked as a duplicate of this bug. *** Failed QA -Tested on: openstack-puppet-modules-2015.1.8-17.el7ost.noarch steps: - Changed the values from 16384 to 8192 on instack: /etc/security/limits.d/rabbitmq-server.conf - Deployed the overcloud (tried with and without tuskar) - Overcloud controller rabbitmq FD limit was still 16384 Tried editing: - /usr/share/openstack-puppet/modules/rabbitmq/manifests/params.pp - Deployed the overcloud (tried with and without tuskar) - Overcloud controller rabbitmq FD limit was still 16384 According to comment #15 tht does not have support. If you would try to run puppet manifest manually without RHOSd, the parameter would be changed. stack@instack:~>>> cat templates/rabbit.yaml
parameters:
RabbitFDLimit: 7192
stack@instack:~>>> openstack overcloud deploy --templates ~/templates/my-overcloud -e ~/templates/my-overcloud/environments/network-isolation.yaml -e ~/templates/network-environment.yaml --control-scale 3 --compute-scale 1 --ceph-storage-scale 0 --ntp-server clock.redhat.com --libvirt-type qemu -e ~/templates/rabbit.yaml
stack@instack:~>>> for ip in $(nova list | grep controller | awk '{print $12;}' | cut -d "=" -f2); do ssh heat-admin@$ip 'sudo rabbitmqctl report | grep -A3 file_descriptors';done
{file_descriptors,[{total_limit,7092},
{total_used,3},
{sockets_limit,6380},
{sockets_used,1}]},
--
{file_descriptors,[{total_limit,7092},
{total_used,4},
{sockets_limit,6380},
{sockets_used,2}]},
--
{file_descriptors,[{total_limit,7092},
{total_used,197},
{sockets_limit,6380},
{sockets_used,195}]},
{file_descriptors,[{total_limit,7092},
{total_used,3},
{sockets_limit,6380},
{sockets_used,1}]},
--
{file_descriptors,[{total_limit,7092},
{total_used,197},
{sockets_limit,6380},
{sockets_used,195}]},
--
{file_descriptors,[{total_limit,7092},
{total_used,4},
{sockets_limit,6380},
{sockets_used,2}]},
{file_descriptors,[{total_limit,7092},
{total_used,197},
{sockets_limit,6380},
{sockets_used,195}]},
--
{file_descriptors,[{total_limit,7092},
{total_used,4},
{sockets_limit,6380},
{sockets_used,2}]},
--
{file_descriptors,[{total_limit,7092},
{total_used,3},
{sockets_limit,6380},
{sockets_used,1}]},
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:1872 |
Description of problem: Unable to control the file_descriptors limit for rabbitmq-server via the director. Version-Release number of selected component (if applicable): Current. How reproducible: Always. Steps to Reproduce: 1. Deploy an overcloud using OSP-d. Actual results: rabbitmq-sever is launched via the rabbitmq resource agent using the following file_descriptors limit : # pcs resource show rabbitmq Resource: rabbitmq (class=ocf provider=heartbeat type=rabbitmq-cluster) Attributes: set_policy="ha-all ^(?!amq\.).* {"ha-mode":"all"}" Operations: start interval=0s timeout=100 (rabbitmq-start-timeout-100) stop interval=0s timeout=90 (rabbitmq-stop-timeout-90) monitor interval=10 timeout=40 (rabbitmq-monitor-interval-10) # rabbitmqctl report | grep -A3 file_descriptors {file_descriptors,[{total_limit,924}, {total_used,3}, {sockets_limit,829}, {sockets_used,1}]}, -- {file_descriptors,[{total_limit,924}, {total_used,3}, {sockets_limit,829}, {sockets_used,1}]}, -- {file_descriptors,[{total_limit,924}, {total_used,152}, {sockets_limit,829}, {sockets_used,150}]}, Expected results: rabbitmq-server is launched via the rabbitmq resource agent but is able to control the file_descriptors limit via a parameter. As requested in BZ#1240561. Additional info: