Bug 1282491
| Summary: | [RabbitMQ] update default file descriptor setting | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Joe Talerico <jtaleric> | |
| Component: | openstack-tripleo-heat-templates | Assignee: | Chris Jones <chjones> | |
| Status: | CLOSED ERRATA | QA Contact: | Tomas Jamrisko <tjamrisk> | |
| Severity: | urgent | Docs Contact: | ||
| Priority: | urgent | |||
| Version: | 7.0 (Kilo) | CC: | adhingra, apevec, dh3, fahmed, fdinitto, jcoufal, jeckersb, jraju, jschluet, lhh, mburns, mcornea, mkrcmari, mlopes, oblaut, pcaruana, plemenko, rcernin, rhel-osp-director-maint, royoung, sputhenp, srevivo, vcojot | |
| Target Milestone: | Upstream M3 | Keywords: | Triaged | |
| Target Release: | 10.0 (Newton) | |||
| Hardware: | All | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | openstack-tripleo-heat-templates-5.0.0-0.20160820164503.6c537d2.1.el7ost | Doc Type: | Bug Fix | |
| Doc Text: |
Prior to this update, the RabbitMQ maximum open file descriptors was set to 4096. Consequently, customers with larger deployments could hit this limit and face stability issues. With this update, the maximum open file descriptor limit for RabbitMQ has been increased to 65536. As a result, larger deployments should now be significantly less likely to run into this issue.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1353943 (view as bug list) | Environment: | ||
| Last Closed: | 2016-12-14 15:17:43 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1353943, 1356490 | |||
eck, can you ack the 16k default? (In reply to Joe Talerico from comment #0) > Description of problem: > Current OSP Deployments deploy w/ ~4k file descriptors. I could of sworn > previously we deployed with 16k. > > From Rabbit: > {file_descriptors,[{total_limit,3996}, > {total_used,945}, > {sockets_limit,3594}, > {sockets_used,943}]}, > > Chatting with eck we should deploy with 16k, instead of 4k. I'm perfectly fine with even more. Let's say 32k or 64. However let's start from 16k first. (In reply to Joe Talerico from comment #2) > eck, can you ack the 16k default? ACK all the things Ok since we agree about this, let's reassign it to osp-director. This is a place where all the OpenStack configs lives. We hit the 4k limit for two important customers. Rakutan Marketing and Telefonica. Let us make sure that we increase it to 64k Please note hitting this limit and spending time to troubleshoot it again and again affects customer satisfaction. So nominating for Async and increasing priority to Urgent. Jarda, want to comment on prioritization? This bug did not make the OSP 8.0 release. It is being deferred to OSP 10. This bug was recently brought to my attention and the problem being reported is a well-known issue with MySQL and MariaDB. We should correct our default deployment to 32k, ideally 64k as soon as is reasonable as most workloads will hit or exceed the 4k limit running less than peak activity. This bug was recently brought to my attention. We should correct our default deployment to 32k, ideally 64k as soon as is reasonable as most workloads will hit or exceed the 4k limit running less than peak activity. Will defer to Jarda on what is reasonable in this case. RHOS 8/9/10 currently have a limit of 16k files. Patch proposed upstream to increase the limit to 64k files: https://review.openstack.org/#/c/332127/ openstack-tripleo-heat-templates-5.0.0-0.20160823140311.72404b.1.el7ost.noarch
Checked on clean osp10 deployment
{file_descriptors,[{total_limit,65436},
Need to be checked also on upgraded deployment osp9--->osp10
Verified after upgrade from 9 to 10. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html |
Description of problem: Current OSP Deployments deploy w/ ~4k file descriptors. I could of sworn previously we deployed with 16k. From Rabbit: {file_descriptors,[{total_limit,3996}, {total_used,945}, {sockets_limit,3594}, {sockets_used,943}]}, Chatting with eck we should deploy with 16k, instead of 4k.