Bug 2084130 - CertificateCleanupJob fails with foreign key constraint violation on table cp_upstream_consumer
Summary: CertificateCleanupJob fails with foreign key constraint violation on table cp...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Candlepin
Version: 6.10.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: 6.12.0
Assignee: satellite6-bugs
QA Contact:
URL:
Whiteboard:
Depends On: 2084516 2084517 2084518
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-11 13:46 UTC by Nikos Moumoulidis
Modified: 2023-04-08 00:47 UTC (History)
5 users (show)

Fixed In Version: candlepin-4.0.18-1, candlepin-4.1.14-1, candlepin-4.2.2-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2084516 2084517 2084518 (view as bug list)
Environment:
Last Closed: 2022-11-16 13:33:51 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker ENT-4986 0 None None None 2022-05-12 10:36:02 UTC
Red Hat Issue Tracker SAT-13088 0 None None None 2022-09-22 19:14:17 UTC
Red Hat Product Errata RHSA-2022:8506 0 None None None 2022-11-16 13:34:04 UTC

Description Nikos Moumoulidis 2022-05-11 13:46:10 UTC
Description of problem:
When a manifest that is old enough (1+ year) is imported into satellite, the CertificateCleanupJob fails with a foreign key constraint error.
This has no immediate impact, other than the fact that expired certificates and serials stay in the relevant database tables over time (which CertificateCleanupJob is supposed to solve).

Version-Release number of selected component (if applicable):
satellite 6.10.0 onwards
candlepin-4.0.8-1 onwards

How reproducible:
100%

Steps to Reproduce:
1. Have a Subscription Allocation (manifest) in the customer portal that was created at least 1 year ago (so that it's Identity Certificate is expired).
2. Export/Import the manifest into Satellite
3. Wait until the CertificateCleanupJob runs in Candlepin (should be running at 12:00 every day, but you could alter it's schedule, e.g. by setting
candlepin.async.jobs.CertificateCleanupJob.schedule=0 * * * * ?
in candlepin.conf and restarting tomcat to make it run every minute).

Actual results:

The following stack trace is seen in candlepin.log:
2022-04-18 12:00:00,080 [thread=Thread-25536 (ActiveMQ-client-global-threads)] [job=4cc353697fe0e74101803d9de68456da, job_key=CertificateCleanupJob, org=, csid=] WARN  org.hibernate.engine.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: 23503
2022-04-18 12:00:00,081 [thread=Thread-25536 (ActiveMQ-client-global-threads)] [job=4cc353697fe0e74101803d9de68456da, job_key=CertificateCleanupJob, org=, csid=] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - ERROR: update or delete on table "cp_id_cert" violates foreign key constraint "fk54b0f288a0b39916" on table "cp_upstream_consumer"
  Detail: Key (id)=(4cc35369617122e3016171cfa631004e) is still referenced from table "cp_upstream_consumer".
2022-04-18 12:00:00,095 [thread=Thread-25536 (ActiveMQ-client-global-threads)] [job=4cc353697fe0e74101803d9de68456da, job_key=CertificateCleanupJob, org=, csid=] ERROR org.candlepin.async.JobManager - Job "CertificateCleanupJob" failed in 55ms
javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement
	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154)
	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181)
	at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1614)
	at org.candlepin.model.IdentityCertificateCurator.deleteByIds(IdentityCertificateCurator.java:74)
	at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:56)
	at org.candlepin.async.tasks.CertificateCleanupJob.cleanupExpiredIdentityCerts(CertificateCleanupJob.java:91)
	at org.candlepin.async.tasks.CertificateCleanupJob.execute(CertificateCleanupJob.java:70)
	at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:64)
	at org.candlepin.async.JobManager.executeJob(JobManager.java:1333)
	at org.candlepin.async.JobMessageReceiver$MessageListener.handleMessage(JobMessageReceiver.java:394)
	at org.candlepin.messaging.impl.artemis.ArtemisConsumer$ArtemisMessageForwarder.onMessage(ArtemisConsumer.java:59)
	at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1030)
	at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:49)
	at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1153)
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement
	at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:109)
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:200)
	at org.hibernate.hql.internal.ast.exec.BasicExecutor.doExecute(BasicExecutor.java:100)
	at org.hibernate.hql.internal.ast.exec.BasicExecutor.execute(BasicExecutor.java:59)
	at org.hibernate.hql.internal.ast.exec.DeleteExecutor.execute(DeleteExecutor.java:113)
	at org.hibernate.hql.internal.ast.QueryTranslatorImpl.executeUpdate(QueryTranslatorImpl.java:454)
	at org.hibernate.engine.query.spi.HQLQueryPlan.performExecuteUpdate(HQLQueryPlan.java:377)
	at org.hibernate.internal.SessionImpl.executeUpdate(SessionImpl.java:1420)
	at org.hibernate.query.internal.AbstractProducedQuery.doExecuteUpdate(AbstractProducedQuery.java:1623)
	at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1605)
	... 17 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "cp_id_cert" violates foreign key constraint "fk54b0f288a0b39916" on table "cp_upstream_consumer"
  Detail: Key (id)=(4cc35369617122e3016171cfa631004e) is still referenced from table "cp_upstream_consumer".
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155)
	at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:132)
	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:462)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197)
	... 25 common frames omitted


Expected results:
The CertificateCleanupJob finishes successfully, without errors in the log.

Additional info:

Comment 3 Lai 2022-09-22 19:12:23 UTC
Followed the directions in the description.

Expected result:
The CertificateCleanupJob finishes successfully, without errors in the log.

Actual:
The CertificateCleanupJob finishes successfully, without errors in the log.

I checked the log and there was no errors and no warning associated with the issue outline in this bz.

Verified on 6.12.0 snap 11 with candlepin-4.1.14-1.el8sat.noarch

Comment 8 errata-xmlrpc 2022-11-16 13:33:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.12 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:8506


Note You need to log in before you can comment on or make changes to this bug.