Bug 1273168

Summary:	installer removes any qpid tuning for large deployments
Product:	Red Hat Satellite	Reporter:	Stuart Auchterlonie <sauchter>
Component:	Installation	Assignee:	Stephen Benjamin <stbenjam>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Katello QA List <katello-qa-list>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	6.1.3	CC:	bbuckingham, bkearney, cwelton, millard.matt, mmccune, peter.vreman, rmahique, snemeth, stbenjam
Target Milestone:	Unspecified	Keywords:	Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-10-14 14:04:12 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1122832, 1317008

Description Stuart Auchterlonie 2015-10-19 20:12:06 UTC

Description of problem:

Our own documentation tells customer to tune the qpid component
of Satellite 6 to support larger deployments. See [1]

Running either katello-installer or capsule-installer as part
of a normal upgrade will remove any tuning which has been done
for larger deployments, and will revert /etc/qpid/qpidd.conf
to a simple config for a small deployment

Version-Release number of selected component (if applicable):

6.1.3

How reproducible:

100%

Steps to Reproduce:
1. Edit /etc/qpid/qpidd.conf, set `max-connections=2500`
2. Run katello-installer or capsule-installer --upgrade
3. 

Actual results:

View /etc/qpid/qpidd.conf, note that max-connections setting is gone.

Expected results:

qpid tuning which is done according to our documentation to support
a larger deployment should be maintained.

Additional info:

[1] - https://access.redhat.com/documentation/en-US/Red_Hat_Satellite/6.1/html/Installation_Guide/sect-Red_Hat_Satellite-Installation_Guide-Prerequisites.html#sect-Red_Hat_Satellite-Installation_Guide-Prerequisites-Large_deployments

Comment 3 Stuart Auchterlonie 2015-11-26 11:02:16 UTC

This issue is wider than just qpid though. Anything that needs tuning for larger
deployments will have it's config overwritten on upgrade.

Comment 4 Stuart Auchterlonie 2015-12-02 12:09:30 UTC

Is it possibly we can implement this by adding something like the following
to the installer

--max-clients=<number>

and then do the calculations internally, and setup qpid (and any other subsystem)
that requires tuning for large number of clients automatically based on this
information????

Comment 5 Brad Buckingham 2015-12-09 17:14:27 UTC

Stuart, I think that would be a nice approach; however, we'll need to investigate.  It looks like the current issue is due to the installer managing the qpidd.conf via puppet; however, it doesn't currently manage the other configuration files suggested by the docs.  We'd need to make sure that Satellite can manage those without clobbering any user data.

Comment 6 Stuart Auchterlonie 2015-12-09 17:27:31 UTC

Brad,

My theory was, that we can at least use the figure provided by (--max-clients)
to work with puppet to configure qpid to resolve this issue.

Any other items which need tuning, we can then raise new bugzilla's for but
still leverage the configuration the customer has provided to the installer.


Regards
Stuart

Comment 7 Peter Vreman 2016-01-04 17:05:07 UTC

I have the same problem to fix https://bugzilla.redhat.com/show_bug.cgi?id=1172556, where the upgrade removes my tuning of the REST timeout value in /etc/foreman/plugins/katello.yaml where i have to set rest_client_timeout: 3600 (default is 120).

Comment 8 Peter Vreman 2016-01-04 17:07:27 UTC

And there are more tuning values from the tuning guide that are removed on upgrade.

For example the passenger tunings of PassengerMaxPoolSize, PassengerMinInstances, etc in the following files:

/etc/httpd/conf.d/passenger.conf
/etc/httpd/conf.d/05-foreman-ssl.conf
/etc/httpd/conf.d/25-puppet.conf

Comment 12 Stefan Nemeth 2016-02-23 14:59:37 UTC

/etc/sysconfig/elasticsearch  also reset to default by update

Comment 13 Matt Millard 2016-03-23 14:43:25 UTC

I was also bit with this during a katello-installer --upgrade.  I had just resolved a qpidd issue with max-connections and then did an upgrade the next day.  This wiped out my customizations. This should be noted somewhere in the release notes for upgrades with a list of possible config files that may need to be modified again after the installer is ran. Ideally this should just be fixed and not wipe customization made by the customer.

Comment 15 Bryan Kearney 2016-07-08 20:24:23 UTC

Per 6.3 planning, moving out non acked bugs to the backlog

Comment 17 Stephen Benjamin 2016-10-14 13:58:47 UTC

We can add max connections to the installer as an option for now, but the tuning guide has some other notable issues you might want to file bugs for.  Like this:

"On Red Hat Enterprise Linux 7, add the following line to /usr/lib/systemd/system/qpidd.service at the end of the [Service] section: "

You shouldn't edit units directly, use systemd drop-ins or similar.

Comment 18 Stephen Benjamin 2016-10-14 13:59:24 UTC

Created redmine issue http://projects.theforeman.org/issues/16943 from this bug

Comment 19 Stephen Benjamin 2016-10-14 14:04:12 UTC

Ah this is no longer an issue in 6.2.  

Clients connect to the dispatch router now, instead of qpid, and the 6.2 performance guide recommends increasing file limits with  /etc/systemd/system/qdrouterd.service.d/limits.conf --  the installer won't overwrite these changes.