Bug 1332784

Summary: Only 800 hosts can be registered to satellite 6.2 by default
Product: Red Hat Satellite Reporter: Pradeep Kumar Surisetty <psuriset>
Component: PerformanceAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact:
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: bbuckingham, psuriset
Target Milestone: Unspecified   
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-11 15:58:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
apache scale none

Description Pradeep Kumar Surisetty 2016-05-04 05:36:46 UTC
Created attachment 1153694 [details]
apache scale

Description of problem:

Every new host registration takes huge number of apache open connections on satellite. 

I started registering hosts in bunch of 50's concurrently to satellite.  After 700-800, satellite server rejects all new registrations due to lot of open apache connections. 

As shown in attachment, for every batch of 50, number of open apache connections increases. Once it reach 110 open apache connects, satellite stops accepting  any new registrations. 

 lsof | cut -d ' ' -f 1 | sort | uniq -c | sort -n | tail

<snip>
110534 httpd






Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Scale satellite 
2.
3.

Actual results:

There are too many apache open connections per registration. For every 50 host registrations about 4k-5k apache open connections are left.

Expected results:

Not many apache open connections
Additional info:

Comment 1 Pradeep Kumar Surisetty 2016-05-04 05:37:27 UTC
IN attchmanent: 

x axis: time
y axis: Number of open apache connections

Comment 2 Pradeep Kumar Surisetty 2016-05-04 05:39:44 UTC
Work around:  Increase number of apache connections limit

cat /etc/systemd/system/httpd.service.d/limits.conf
[Service]
LimitNOFILE=20000

systemctl daemon-reload
systemctl restart httpd

This is just a workaround. Would be interesting to find out why do we need these many apache connections

Comment 3 Brad Buckingham 2016-05-11 15:41:37 UTC
Hi Pradeep, is this a dupe of bug 1328984 ?

Comment 4 Pradeep Kumar Surisetty 2016-05-11 15:58:05 UTC
(In reply to Brad Buckingham from comment #3)
> Hi Pradeep, is this a dupe of bug 1328984 ?

yep

*** This bug has been marked as a duplicate of bug 1328984 ***