Bug 2122945

Summary: Satellite manifest upload/import error: Unexpected exception occurred while executing transactional block
Product: Red Hat Satellite Reporter: Moustafa Harbi <mharbi>
Component: CandlepinAssignee: Barnaby Court <bcourt>
Status: CLOSED ERRATA QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 6.11.1CC: ahumbe, bcourt, crog, dhjoshi, momran, nmoumoul, osousa, pcreech, tasander, wpinheir, zhunting
Target Milestone: 6.12.0Keywords: PrioBumpGSS, Regression, Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: candlepin-4.2.6-1, candlepin-4.1.15-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2123382 2123384 (view as bug list) Environment:
Last Closed: 2022-11-16 13:35:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2123382, 2123384    
Bug Blocks:    

Description Moustafa Harbi 2022-08-31 11:36:00 UTC 
Description of problem:
Unable to upload satellite manifest.

Version-Release number of selected component (if applicable):
satellite-6.11.1.1-1.el8sat.noarch
candlepin-4.1.13-1.el8sat.noarch


How reproducible:
Navigate to content-> Subscriptions -> Manage Manifest -> Delete old manifest then attach a new one. 

Steps to Reproduce:
1. Delete old manifest
2. Try to attach a new manifest


Actual results:
Getting the following error in candlepin logs:

~~~
2022-08-30 15:30:20,833 [thread=Thread-1084 (ActiveMQ-client-global-threads)] [job=8ae7830582e10ea40182eebb22430186, job_key=ImportJob, org=CustomerX, csid=0dec6cf8-240c-4ceb-bdc2-47ef615ffb23] ERROR org.candlepin.controller.refresher.visitors.ProductNodeVisitor - Entity version collision detected: Product [uuid: null, id: 183, name: JBoss Enterprise Application Platform] != Product [uuid: 8ae78305826a963e01826cd4d89c030f, id: 183, name: JBoss Enterprise Application Platform]
2022-08-30 15:30:20,876 [thread=https-jsse-nio-127.0.0.1-23443-exec-18] [req=8886ab99-bbee-46f8-9fdd-e27f9ef04e6a, org=, csid=] INFO  org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/5d5db0ad-8832-4474-92ae-71812d1cab8b/accessible_content
2022-08-30 15:30:20,877 [thread=https-jsse-nio-127.0.0.1-23443-exec-20] [req=a5e9d2db-1867-42b4-b762-a6bf0a02ff52, org=, csid=] INFO  org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/3447365f-e44a-41d1-bf8c-15f630e60eff/content_overrides
2022-08-30 15:30:20,886 [thread=https-jsse-nio-127.0.0.1-23443-exec-20] [req=a5e9d2db-1867-42b4-b762-a6bf0a02ff52, org=, csid=] INFO  org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=9
2022-08-30 15:30:20,889 [thread=https-jsse-nio-127.0.0.1-23443-exec-18] [req=8886ab99-bbee-46f8-9fdd-e27f9ef04e6a, org=CustomerX, csid=] INFO  org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=13
2022-08-30 15:30:20,974 [thread=Thread-1084 (ActiveMQ-client-global-threads)] [job=8ae7830582e10ea40182eebb22430186, job_key=ImportJob, org=CustomerX, csid=0dec6cf8-240c-4ceb-bdc2-47ef615ffb23] ERROR org.candlepin.sync.Importer - Recording import failure
org.candlepin.util.TransactionExecutionException: Unexpected exception occured while executing transactional block
        at org.candlepin.util.Transactional.executeNested(Transactional.java:379)
        at org.candlepin.util.Transactional.execute(Transactional.java:501)
        at org.candlepin.controller.refresher.RefreshWorker.execute(RefreshWorker.java:488)
        at org.candlepin.controller.CandlepinPoolManager.refreshPoolsWithRegeneration(CandlepinPoolManager.java:270)
        at org.candlepin.controller.Refresher.run(Refresher.java:154)
        at org.candlepin.sync.Importer.importObjects(Importer.java:602)
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:56)
        at org.candlepin.sync.Importer.doExport(Importer.java:429)
        at org.candlepin.sync.Importer.loadStoredExport(Importer.java:210)
        at org.candlepin.controller.ManifestManager.importStoredManifest(ManifestManager.java:220)
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:64)
        at org.candlepin.async.tasks.ImportJob.execute(ImportJob.java:229)
        at org.candlepin.async.JobManager.executeJob(JobManager.java:1271)
        at org.candlepin.async.JobMessageReceiver$MessageListener.handleMessage(JobMessageReceiver.java:350)
        at org.candlepin.messaging.impl.artemis.ArtemisConsumer$ArtemisMessageForwarder.onMessage(ArtemisConsumer.java:59)
        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1013)
        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1133)
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
        at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: java.lang.IllegalStateException: Entity version collision detected: Product [uuid: null, id: 183, name: JBoss Enterprise Application Platform] != Product [uuid: 8ae78305826a963e01826cd4d89c030f, id: 183, name: JBoss Enterprise Application Platform]
        at org.candlepin.controller.refresher.visitors.ProductNodeVisitor.resolveEntityVersion(ProductNodeVisitor.java:249)
        at org.candlepin.controller.refresher.visitors.ProductNodeVisitor.applyChanges(ProductNodeVisitor.java:199)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.applyChangesImpl(NodeProcessor.java:218)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.lambda$applyChangesImpl$5(NodeProcessor.java:214)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.applyChangesImpl(NodeProcessor.java:214)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.lambda$processNodes$2(NodeProcessor.java:161)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.processNodes(NodeProcessor.java:161)
        at org.candlepin.controller.refresher.RefreshWorker.lambda$execute$0(RefreshWorker.java:473)
        at org.candlepin.util.Transactional.executeNested(Transactional.java:376)
        ... 22 common frames omitted
2022-08-30 15:30:20,977 [thread=Thread-1084 (ActiveMQ-client-global-threads)] [job=8ae7830582e10ea40182eebb22430186, job_key=ImportJob, org=CustomerX, csid=0dec6cf8-240c-4ceb-bdc2-47ef615ffb23] INFO  org.candlepin.controller.ManifestManager - Deleting stored manifest file: 8ae7830582e10ea40182eebb211f0185
2022-08-30 15:30:20,978 [thread=https-jsse-nio-127.0.0.1-23443-exec-9] [req=dd0695ff-3513-402a-8bc8-923d2129ddd3, org=, csid=] INFO  org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/5d5db0ad-8832-4474-92ae-71812d1cab8b/content_overrides
2022-08-30 15:30:20,983 [thread=Thread-1084 (ActiveMQ-client-global-threads)] [job=8ae7830582e10ea40182eebb22430186, job_key=ImportJob, org=CustomerX, csid=0dec6cf8-240c-4ceb-bdc2-47ef615ffb23] ERROR org.candlepin.async.JobManager - Job "Import Manifest" failed in 3294ms
org.candlepin.async.JobExecutionException: Unexpected exception occured while executing transactional block
        at org.candlepin.async.tasks.ImportJob.execute(ImportJob.java:249)
        at org.candlepin.async.JobManager.executeJob(JobManager.java:1271)
        at org.candlepin.async.JobMessageReceiver$MessageListener.handleMessage(JobMessageReceiver.java:350)
        at org.candlepin.messaging.impl.artemis.ArtemisConsumer$ArtemisMessageForwarder.onMessage(ArtemisConsumer.java:59)
        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1013)
        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1133)
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
        at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: org.candlepin.util.TransactionExecutionException: Unexpected exception occured while executing transactional block
        at org.candlepin.util.Transactional.executeNested(Transactional.java:379)
        at org.candlepin.util.Transactional.execute(Transactional.java:501)
        at org.candlepin.controller.refresher.RefreshWorker.execute(RefreshWorker.java:488)
        at org.candlepin.controller.CandlepinPoolManager.refreshPoolsWithRegeneration(CandlepinPoolManager.java:270)
        at org.candlepin.controller.Refresher.run(Refresher.java:154)
        at org.candlepin.sync.Importer.importObjects(Importer.java:602)
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:56)
        at org.candlepin.sync.Importer.doExport(Importer.java:429)
        at org.candlepin.sync.Importer.loadStoredExport(Importer.java:210)
        at org.candlepin.controller.ManifestManager.importStoredManifest(ManifestManager.java:220)
        at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:64)
        at org.candlepin.async.tasks.ImportJob.execute(ImportJob.java:229)
        ... 11 common frames omitted
Caused by: java.lang.IllegalStateException: Entity version collision detected: Product [uuid: null, id: 183, name: JBoss Enterprise Application Platform] != Product [uuid: 8ae78305826a963e01826cd4d89c030f, id: 183, name: JBoss Enterprise Application Platform]
        at org.candlepin.controller.refresher.visitors.ProductNodeVisitor.resolveEntityVersion(ProductNodeVisitor.java:249)
        at org.candlepin.controller.refresher.visitors.ProductNodeVisitor.applyChanges(ProductNodeVisitor.java:199)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.applyChangesImpl(NodeProcessor.java:218)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.lambda$applyChangesImpl$5(NodeProcessor.java:214)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.applyChangesImpl(NodeProcessor.java:214)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.lambda$processNodes$2(NodeProcessor.java:161)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at org.candlepin.controller.refresher.visitors.NodeProcessor.processNodes(NodeProcessor.java:161)
        at org.candlepin.controller.refresher.RefreshWorker.lambda$execute$0(RefreshWorker.java:473)
        at org.candlepin.util.Transactional.executeNested(Transactional.java:376)
        ... 22 common frames omitted
~~~


Expected results:
Manifest upload should succeed but an exception is triggered with the following message: "Unexpected exception occured while executing transactional block"

Additional Info:

Executing the steps mentioned in this KCS [1] helped to solve the issue.

[1] https://access.redhat.com/solutions/6958601
Comment 5 Lai 2022-09-23 17:48:27 UTC 
Steps to retest:

1. Upload a manifest
2. Delete the manifest from step 1
3. Import a new manifest

Expected results:
Manifest upload should be successful

Actual:
Manifest upload is successful

I checked the candlepin log and there was no issues.

Verified on 6.12.0 snap 12 with candlepin-4.1.15-1.el8sat.noarch
Comment 10 errata-xmlrpc 2022-11-16 13:35:34 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.12 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:8506