Bug 1298202

Summary:	various issues and JVM dead when registering lots of profiles in parallel
Product:	Red Hat Satellite 5	Reporter:	Jan Hutař <jhutar>
Component:	Server	Assignee:	Tomáš Kašpárek <tkasparek>
Status:	CLOSED DUPLICATE	QA Contact:	Red Hat Satellite QA List <satqe-list>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	570	CC:	dyordano, ggainey, tlestach
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-01-29 18:47:45 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jan Hutař 2016-01-13 13:26:47 UTC

Description of problem:
There are various issues in logs and killed JVM when registering lots of profiles in parallel.

Satellite's HW was:
  Intel Platform: DigitalOffice CPU: SandyBridge D2
  Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz
  3717 MB memory, 500 GB disk space


Version-Release number of selected component (if applicable):
Satellite 5.7.0 @ embedded PostgreSQL
satellite-schema-5.7.0.20-1.el6sat.noarch
spacewalk-java-2.3.8-120.el6sat.noarch
spacewalk-backend-2.3.3-33.el6sat.noarch


How reproducible:
always


Steps to Reproduce:
1. Register 500 systems in parallel


Actual results:
JVM dies (i.e. are down) and there is one APPARENT DEADLOCK in catalina.out.

Only 292 systems out of 500 gets registered, 105 of these were missing package profile.


Expected results:
Satellite survives. If the system gets registered, its package profile is loaded.

Comment 2 Jan Hutař 2016-01-14 06:30:06 UTC

When you have taskomatic and rhn-search stopped while registering, all 500 systems get registered.

After I have started rhn-search, it took to it about 10 minutes to process new stuff on the Satellite.

After I have started ErrataCacheTask, it took to it only about 1 minute to process 500 new items in ErrataCacheTask queue.

Comment 3 Jan Hutař 2016-01-14 06:31:27 UTC

(note that each system had 45/47 applicable erratas/packages)

Comment 4 Grant Gainey 2016-01-29 18:47:45 UTC

In spite of having different databases, the problem here is the same as in 1298061 - more incoming connections than the allowed-max-db-connections. I'm closing this as a dup of that BZ.

*** This bug has been marked as a duplicate of bug 1298061 ***