Bug 991544 - Runtime Error No row with the given identifier exists: [org.candlepin.model.ProvidedProduct#8a90f81a403bb40601403db8ca435015] at org.hibernate.UnresolvableObjectException.throwIfNull:65
Runtime Error No row with the given identifier exists: [org.candlepin.model.P...
Status: CLOSED WORKSFORME
Product: Candlepin
Classification: Community
Component: candlepin (Show other bugs)
0.9
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: Jesus M. Rodriguez
Katello QA List
:
Depends On:
Blocks: rhsm-rhel70
  Show dependency treegraph
 
Reported: 2013-08-02 13:23 EDT by John Sefler
Modified: 2015-05-14 10:52 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-22 12:06:25 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
catalina.out containing six occurances of "Runtime Error No row with the given identifier exists" (17.97 MB, text/plain)
2013-08-02 13:31 EDT, John Sefler
no flags Details

  None (edit)
Description John Sefler 2013-08-02 13:23:32 EDT
Description of problem:
Unfortunately I do not yet have a manually repeatable test scenario for this bug, but our nightly test automation runs have been hitting this Runtime Error regularly against Candlepin master since July 11-ish, 2013.

On the command line from the subscription-manager client....
201308020231:22.850 - FINE: ssh root@jsefler-5server.usersys.redhat.com ls -1 -v /etc/pki/entitlement/*.pem | grep -v key.pem
201308020231:22.919 - FINE: Stdout:
/etc/pki/entitlement/4098574580346795424.pem
/etc/pki/entitlement/5626592194288091665.pem
201308020231:22.920 - FINE: Stderr: 
201308020231:22.921 - FINE: ExitCode: 0
201308020231:22.921 - FINE: ssh root@jsefler-5server.usersys.redhat.com subscription-manager unregister
201308020231:24.126 - FINE: Stdout: Runtime Error No row with the given identifier exists: [org.candlepin.model.ProvidedProduct#8a90f81a403bb40601403db8ca435015] at org.hibernate.UnresolvableObjectException.throwIfNull:65
201308020231:24.126 - FINE: Stderr: 
201308020231:24.126 - FINE: ExitCode: 255

Corresponding catalina.out from candlepin...
Aug 02 02:31:17 [http-8443-10] INFO  org.candlepin.model.PoolCurator - Omitting pool due to failed rule check: 8a90f81a403bb40601403bb57df90786
Aug 02 02:31:17 [http-8443-10] INFO  org.candlepin.model.PoolCurator -  Errors: [rulefailed.consumer.type.mismatch]
Aug 02 02:31:17 [http-8443-10] INFO  org.candlepin.model.PoolCurator - Omitting pool due to failed rule check: 8a90f81a403bb40601403bb579160597
Aug 02 02:31:17 [http-8443-10] INFO  org.candlepin.model.PoolCurator -  Warnings: [rulewarning.architecture.mismatch]
Aug 02 02:31:24 [http-8443-10] ERROR org.candlepin.exceptions.mappers.CandlepinExceptionMapper - Runtime Error No row with the given identifier exists: [org.candlepin.model.ProvidedProduct#8a90f81a403bb40601403db8ca435015] at org.hibernate.UnresolvableObjectException.throwIfNull:65
org.hibernate.UnresolvableObjectException: No row with the given identifier exists: [org.candlepin.model.ProvidedProduct#8a90f81a403bb40601403db8ca435015]
        at org.hibernate.UnresolvableObjectException.throwIfNull(UnresolvableObjectException.java:65)
        at org.hibernate.event.def.DefaultRefreshEventListener.onRefresh(DefaultRefreshEventListener.java:148)
        at org.hibernate.impl.SessionImpl.fireRefresh(SessionImpl.java:939)
        at org.hibernate.impl.SessionImpl.refresh(SessionImpl.java:922)
        at org.hibernate.engine.CascadingAction$3.cascade(CascadingAction.java:192)
        at org.hibernate.engine.Cascade.cascadeToOne(Cascade.java:292)
        at org.hibernate.engine.Cascade.cascadeAssociation(Cascade.java:240)
        at org.hibernate.engine.Cascade.cascadeProperty(Cascade.java:193)
        at org.hibernate.engine.Cascade.cascadeCollectionElements(Cascade.java:320)
        at org.hibernate.engine.Cascade.cascadeCollection(Cascade.java:266)
        at org.hibernate.engine.Cascade.cascadeAssociation(Cascade.java:243)
        at org.hibernate.engine.Cascade.cascadeProperty(Cascade.java:193)
        at org.hibernate.engine.Cascade.cascade(Cascade.java:154)
        at org.hibernate.event.def.DefaultRefreshEventListener.onRefresh(DefaultRefreshEventListener.java:121)
        at org.hibernate.event.def.DefaultRefreshEventListener.onRefresh(DefaultRefreshEventListener.java:62)
        at org.hibernate.impl.SessionImpl.fireRefresh(SessionImpl.java:930)
        at org.hibernate.impl.SessionImpl.refresh(SessionImpl.java:918)
        at org.candlepin.model.PoolCurator.lockAndLoad(PoolCurator.java:455)
        at org.candlepin.controller.CandlepinPoolManager.removeEntitlement(CandlepinPoolManager.java:767)
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:58)
        at org.candlepin.controller.CandlepinPoolManager.revokeAllEntitlements(CandlepinPoolManager.java:836)
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:58)
        at org.candlepin.resource.ConsumerResource.deleteConsumer(ConsumerResource.java:1119)
        at org.candlepin.auth.interceptor.SecurityInterceptor.invoke(SecurityInterceptor.java:127)
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:66)
        at sun.reflect.GeneratedMethodAccessor420.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:155)
        at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:257)
        at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:222)
        at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:211)
        at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:525)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:502)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119)
        at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
        at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
        at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
        at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
        at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
        at org.candlepin.servlet.filter.ContentTypeHackFilter.doFilter(ContentTypeHackFilter.java:58)
        at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
        at org.candlepin.servlet.filter.logging.LoggingFilter.doFilter(LoggingFilter.java:60)
        at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
        at org.candlepin.servlet.filter.CandlepinPersistFilter.doFilter(CandlepinPersistFilter.java:48)
        at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
        at org.candlepin.servlet.filter.CandlepinScopeFilter.doFilter(CandlepinScopeFilter.java:57)
        at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
        at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
        at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
        at java.lang.Thread.run(Thread.java:636)


Version-Release number of selected component (if applicable):
[root@jsefler-f14-5candlepin candlepin]# git branch
* master
[root@jsefler-f14-5candlepin candlepin]# git show-ref | head -1
85940a01293f29bd927e6e1c81fa2f584286d8ac refs/heads/master
Comment 1 John Sefler 2013-08-02 13:31:05 EDT
Created attachment 782083 [details]
catalina.out containing six occurances of "Runtime Error No row with the given identifier exists"

Attaching catalina.out which contains these six occurances of this Runtime Error during a nightly test automation run...

[root@jsefler-f14-5candlepin tomcat6]# grep "No row" catalina.out 
Aug 02 02:31:24 [http-8443-10] ERROR org.candlepin.exceptions.mappers.CandlepinExceptionMapper - Runtime Error No row with the given identifier exists: [org.candlepin.model.ProvidedProduct#8a90f81a403bb40601403db8ca435015] at org.hibernate.UnresolvableObjectException.throwIfNull:65
org.hibernate.UnresolvableObjectException: No row with the given identifier exists: [org.candlepin.model.ProvidedProduct#8a90f81a403bb40601403db8ca435015]
Aug 02 02:32:01 [http-8443-10] ERROR org.candlepin.exceptions.mappers.CandlepinExceptionMapper - Runtime Error No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026] at org.hibernate.UnresolvableObjectException.throwIfNull:65
org.hibernate.UnresolvableObjectException: No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026]
Aug 02 02:32:04 [http-8443-10] ERROR org.candlepin.exceptions.mappers.CandlepinExceptionMapper - Runtime Error No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026] at org.hibernate.UnresolvableObjectException.throwIfNull:65
org.hibernate.UnresolvableObjectException: No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026]
Aug 02 02:32:07 [http-8443-10] ERROR org.candlepin.exceptions.mappers.CandlepinExceptionMapper - Runtime Error No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026] at org.hibernate.UnresolvableObjectException.throwIfNull:65
org.hibernate.UnresolvableObjectException: No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026]
Aug 02 02:32:10 [http-8443-10] ERROR org.candlepin.exceptions.mappers.CandlepinExceptionMapper - Runtime Error No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026] at org.hibernate.UnresolvableObjectException.throwIfNull:65
org.hibernate.UnresolvableObjectException: No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026]
Aug 02 10:50:06 [http-8443-5] ERROR org.candlepin.exceptions.mappers.CandlepinExceptionMapper - Runtime Error No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026] at org.hibernate.UnresolvableObjectException.throwIfNull:65
org.hibernate.UnresolvableObjectException: No row with the given identifier exists: [org.candlepin.model.DerivedProvidedProduct#8a90f81a403bb40601403db941c55026]
Comment 2 Devan Goodwin 2013-08-13 12:49:07 EDT
The error always appears to be in CandlepinPoolManager.removeEntitlement. The call stack can be one of two paths:

DELETE /consumers/uuid/entitlements (revoke all entitlements) -> ConsumerResource.unbindAll -> CandlepinPoolManager.removeEntitlement

DELETE/consumers/uuid (delete consumer) -> ConsumerResource.deleteConsumer -> CandlepinPoolManager.revokeAllEntitlements -> CandlepinPoolManager.removeEntitlement

John: I'm assuming you must repeatedly call these two functions. Could you provide some info on how they're used? 

Do you always for example do a "delete all entitlements", then "delete the consumer" in some kind of teardown? (only one of those is actually necessary)

Do you do much asynchronously where jobs are fired off simultaneously or do you always block and wait for a call to complete before moving on?

Similar question for owner cleanup/deletion? Is there any cleanup of owners or anything else that might delete pools, that could be running around the same time as the consumer cleanup?
Comment 3 John Sefler 2013-11-22 12:06:25 EST
I have not seen this error in awhile.
Let's close this issue for now.

Note You need to log in before you can comment on or make changes to this bug.