Bug 1435809

Summary: OOM kill Candlepin by updating a custom product on a very large pool
Product: Red Hat Satellite Reporter: Barnaby Court <bcourt>
Component: CandlepinAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: jcallaha
Severity: high Docs Contact:
Priority: high    
Version: 6.2.6CC: bbuckingham, bkearney, candlepin-bugs, cduryee, jcallaha, jhutar, katello-qa-list, mmccune, oshtaier, redakkan, skallesh, zhunting
Target Milestone: UnspecifiedKeywords: Performance, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: candlepin-0.9.54.21-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1435807
: 1446715 (view as bug list) Environment:
Last Closed: 2017-06-20 17:23:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1435807    
Bug Blocks:    

Description Barnaby Court 2017-03-24 21:00:20 UTC
+++ This bug was initially created as a clone of Bug #1435807 +++

Description of problem:

Out of memory when updating a content set that is associated to a pool with very large numbers of entitlements:

This was found using candlepin-0.9.54.17-1.el7.noarch

2017-03-24 18:30:06,614 [thread=http-bio-8443-exec-3] [req=7a4c428e-9f46-45fe-b7ad-b796c9da3cb8, org=] WARN  org.hibernate.engine.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: 53200
2017-03-24 18:30:06,614 [thread=http-bio-8443-exec-3] [req=7a4c428e-9f46-45fe-b7ad-b796c9da3cb8, org=] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Ran out of memory retrieving query results.
2017-03-24 18:30:06,632 [thread=http-bio-8443-exec-3] [req=7a4c428e-9f46-45fe-b7ad-b796c9da3cb8, org=] ERROR org.candlepin.common.exceptions.mappers.CandlepinExceptionMapper - Runtime Error Ran out of memory ret
rieving query results.
org.postgresql.util.PSQLException: Ran out of memory retrieving query results.
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1817) ~[postgresql-9.0-801.jdbc4.jar:na]
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) ~[postgresql-9.0-801.jdbc4.jar:na]
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500) ~[postgresql-9.0-801.jdbc4.jar:na]
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388) ~[postgresql-9.0-801.jdbc4.jar:na]
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:273) ~[postgresql-9.0-801.jdbc4.jar:na]
        at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:76) ~[c3p0-0.9.1.2.jar:0.9.1.2]
        at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:56) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.loader.Loader.getResultSet(Loader.java:2040) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1837) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1816) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.loader.Loader.doQuery(Loader.java:900) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:342) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:312) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.loader.Loader.loadCollection(Loader.java:2238) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:65) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:674) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.event.internal.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:85) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.internal.SessionImpl.initializeCollection(SessionImpl.java:1849) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.collection.internal.AbstractPersistentCollection$4.doWork(AbstractPersistentCollection.java:549) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:234) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.collection.internal.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:545) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.collection.internal.AbstractPersistentCollection.read(AbstractPersistentCollection.java:124) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.hibernate.collection.internal.PersistentSet.iterator(PersistentSet.java:180) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
        at org.candlepin.controller.CandlepinPoolManager.regenerateCertificatesOf(CandlepinPoolManager.java:1025) ~[CandlepinPoolManager.class:na]
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:58) ~[guice-persist-3.0.jar:na]
        at org.candlepin.controller.CandlepinPoolManager.regenerateCertificatesOf(CandlepinPoolManager.java:1139) ~[CandlepinPoolManager.class:na]
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:66) ~[guice-persist-3.0.jar:na]
        at org.candlepin.resource.ContentResource.updateContent(ContentResource.java:193) ~[ContentResource.class:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_131]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_131]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_131]
        at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_131]

Comment 3 Chris Duryee 2017-03-30 14:24:20 UTC
time to alter URL on product without patch: 171 sec

with patch: 6 sec

Comment 4 Barnaby Court 2017-04-03 15:25:45 UTC
Fixed in Candlepin 0.9.54.21-1

Comment 5 Chris Duryee 2017-04-13 12:42:05 UTC
*** Bug 1441949 has been marked as a duplicate of this bug. ***

Comment 6 jcallaha 2017-06-01 14:04:44 UTC
Verified in Satellite 6.2.10 Snap 2.

I tested this on 5 total products, all of which were aligned to 25k content hosts. The time it took for changes to individual repositories varied from 2 to 8 seconds. The average completion time was right around 6 seconds.

Comment 8 errata-xmlrpc 2017-06-20 17:23:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1553