Description of problem:

Satellite fails with "HTTP Error 502: Proxy Error" while ruining concurrent registrations 

Version-Release number of selected component (if applicable):

katello_version = katello-3.16.0-1.el7sat.noarch
satellite_version = satellite-6.8.0-1.el7sat.noarch

Steps to Reproduce:

- Try to run concurrent registration on Satellite 6.8

Actual results: 

~~~

An error occurred: HTTP Error 502: Proxy Error", "url: https://satellite.example.com:443/api/v2/hostgroups/?search=title%3D%22HostGroup%22", "code: 502", "error: HTTP Error 502: Proxy Error"]}

~~~

Expected results: It should register at least 300 hosts concurrently (300 number in comparison to Satellite 6.7) 


Additional info:

1. According to this bug https://projects.theforeman.org/issues/29370 and https://bugzilla.redhat.com/show_bug.cgi?id=1869812 

We have tried with multiple value : 5,16,30... but registration is still failing 

- in /etc/foreman/database.yml, change the 'pool' setting from '5' to '30':
-  Create a new file: /usr/share/foreman/config/initializers/foreman-tasks-db-pool-size.rb
- add this line : ForemanTasks.dynflow.config.db_pool_size = 30
-  run foreman-maintain service restart


2. In order to verify the concurrent registration on large scale, we have created 62 docker hosts and each docker hosts have 500 containers. We are performing concurrent registration in a batches = "3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20" which means - take containers according to the batch value from each docker host and register to the Satellite server and so on..

For Example - If batch = 3 which means take 3 containers from each docker host and that would be 62 * 3 = 186. Out of 186 only 51 got registered on the Satellite server and rest all are failed with specified error.

3. We are using this script to perform the experiment (concurrent registration) : https://github.com/redhat-performance/satperf/blob/master/experiment/regs.sh