Bug 1312098 - [RFE] Satellite 6 should ship with larger default configurations to support large client installs
[RFE] Satellite 6 should ship with larger default configurations to support l...
Status: ON_QA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Installer (Show other bugs)
6.2.0
Unspecified Unspecified
high Severity high (vote)
: Unspecified
: 6.X
Assigned To: Chris Roberts
Katello QA List
https://projects.theforeman.org/issue...
: FutureFeature, Performance, PrioBumpGSS, Triaged, UserExperience
: 1244112 1472587 (view as bug list)
Depends On: 1312680
Blocks: 1122832 CEE_Sat6_Top_BZs/GSS_Sat6_Top_Bugs 260381
  Show dependency treegraph
 
Reported: 2016-02-25 12:54 EST by Mike McCune
Modified: 2018-06-16 01:59 EDT (History)
19 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Foreman Issue Tracker 22513 None None None 2018-04-11 11:51 EDT
Red Hat Knowledge Base (Solution) 2981131 None None None 2018-01-18 14:27 EST

  None (edit)
Description Mike McCune 2016-02-25 12:54:14 EST
Right now we have a pre-set limit of 225 Content Hosts for QPID which is far too low, see the install guide section:


https://access.redhat.com/documentation/en-US/Red_Hat_Satellite/6.1/html/Installation_Guide/sect-Red_Hat_Satellite-Installation_Guide-Prerequisites.html#sect-Red_Hat_Satellite-Installation_Guide-Prerequisites-Large_deployments

"""
Increasing the Maximum Number of Client Connections
With more than 225 content hosts, qpidd reaches the maximum number of client connections. To increase it, first establish the new value of the limit that is calculated as: 
"""

we should have our installer configure the various system level values to increase the defaults so customers don't hit arbitrary limits with reasonable # of clients.

Ideally we can support up to 30k clients without requiring manual reconfiguration.
Comment 3 Brad Buckingham 2017-06-01 09:57:08 EDT
Hello Pradeep,

Based upon your testing, do you have any specific recommendations on the configurations/properties to tune and their values to support addressing this request?

Thanks!
Comment 5 Stephen Benjamin 2017-06-02 10:03:43 EDT
The 6.1 docs linked in comment 0 are a bit out of date.  6.2 has different requirements, documented at  https://access.redhat.com/documentation/en-us/red_hat_satellite/6.2/html/installation_guide/large_deployment_considerations.

Each item is detailed below. Some are entirely possible in the installer.  The postgresql configuration is complicated, see item #3.  I would not expect that to make 6.3.


1. "Increasing the Maximum Number of File Descriptors for Apache"

Not available. Maybe expose in puppet-foreman? But installer won't wipe out these changes if manually made for now as changes are made via systemd drop-ins.


2. "Increasing the Maximum Number of File Descriptors for qpid"

   PR (pending) https://github.com/Katello/puppet-qpid/pull/64

   Custom hiera usage:

   ::qpid::open_file_limit: 65536

3. "Increasing the Shared Buffer and Work Memory" (postgresql)

   Complicated, because of how puppetlabs-postgresql works.  I wouldn't expect this to make the installer for 6.3

   Other BZ's: https://bugzilla.redhat.com/show_bug.cgi?id=1440879, https://bugzilla.redhat.com/show_bug.cgi?id=1449707,    https://bugzilla.redhat.com/show_bug.cgi?id=1421636

   Upstream discussion: https://groups.google.com/d/msg/foreman-dev/mKELbDvo-TQ/kYRIHYZQDAAJ


3. "Increasing Concurrent Content Host Registrations"

   Partial. See https://bugzilla.redhat.com/show_bug.cgi?id=1449697


4. "Increasing the maximum number of open files for qdrouterd"

   PR (pending) https://github.com/Katello/puppet-qpid/pull/50

   Custom hiera usage:

   ::qpid::router::open_file_limit: 3160
Comment 7 Pradeep Kumar Surisetty 2017-06-06 23:48:45 EDT
1) Tuned profiles

    If its RHEL6, install tuned and setup throuput-performance profile
    If its RHEL7, its taken care
2)  Apache Configuration : Passenger config
    It depends on workloads. 

3) Increase apache open files



4) Increase qpid open files

5) Increase content_action_accept_timeout
 in Administer -> Settings -> Katello
 to more than default (20seconds) so clients have more time to answer

6) Increase qdrouterd open files

7) increase postgresql tunings
   shared_buffer
   max_connections
8) httpd_foreman_ssl_katello_keep_alive: 'On'
9) httpd_foreman_ssl_katello_keep_alive_timeout: '5'
10) httpd_foreman_ssl_katello_max_keep_alive_requests: '0'
11) httpd_foreman_ssl_passenger_max_requests: '1000'
12) httpd_foreman_ssl_passenger_max_preloader_idle_time: '0'
13) httpd_passenger_passenger_max_pool_size

    This needs good capacity planning. increasing it more can cause more CPU usage. For puppet workloads also, it will affect

14) postgres_log_min_duration_statement: '500'
15) Increase smart_proxy open files
Comment 8 Chris Roberts 2017-07-26 12:28:50 EDT
Based on Comment #5 moving this out of Sat 6.3 as its not possible.

2 PRS will be taken out of this 1 merged and another in the works see comment #5
Comment 11 Brad Buckingham 2017-08-08 07:00:02 EDT
*** Bug 1472587 has been marked as a duplicate of this bug. ***
Comment 12 Brad Buckingham 2017-08-08 07:01:20 EDT
Bug 1472587 has been marked as a duplicate of this bugzilla. 
 When addressing this one, please ensure to also cover the configuration parameters specified in: https://bugzilla.redhat.com/show_bug.cgi?id=1472587#c0
Comment 15 Bryan Kearney 2018-01-18 14:26:03 EST
*** Bug 1244112 has been marked as a duplicate of this bug. ***
Comment 17 Stephen Benjamin 2018-04-26 10:37:48 EDT
*** Bug 1440879 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.