Description of problem: When a manifest that is old enough (1+ year) is imported into satellite, the CertificateCleanupJob fails with a foreign key constraint error. This has no immediate impact, other than the fact that expired certificates and serials stay in the relevant database tables over time (which CertificateCleanupJob is supposed to solve). Version-Release number of selected component (if applicable): satellite 6.10.0 onwards candlepin-4.0.8-1 onwards How reproducible: 100% Steps to Reproduce: 1. Have a Subscription Allocation (manifest) in the customer portal that was created at least 1 year ago (so that it's Identity Certificate is expired). 2. Export/Import the manifest into Satellite 3. Wait until the CertificateCleanupJob runs in Candlepin (should be running at 12:00 every day, but you could alter it's schedule, e.g. by setting candlepin.async.jobs.CertificateCleanupJob.schedule=0 * * * * ? in candlepin.conf and restarting tomcat to make it run every minute). Actual results: The following stack trace is seen in candlepin.log: 2022-04-18 12:00:00,080 [thread=Thread-25536 (ActiveMQ-client-global-threads)] [job=4cc353697fe0e74101803d9de68456da, job_key=CertificateCleanupJob, org=, csid=] WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: 23503 2022-04-18 12:00:00,081 [thread=Thread-25536 (ActiveMQ-client-global-threads)] [job=4cc353697fe0e74101803d9de68456da, job_key=CertificateCleanupJob, org=, csid=] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - ERROR: update or delete on table "cp_id_cert" violates foreign key constraint "fk54b0f288a0b39916" on table "cp_upstream_consumer" Detail: Key (id)=(4cc35369617122e3016171cfa631004e) is still referenced from table "cp_upstream_consumer". 2022-04-18 12:00:00,095 [thread=Thread-25536 (ActiveMQ-client-global-threads)] [job=4cc353697fe0e74101803d9de68456da, job_key=CertificateCleanupJob, org=, csid=] ERROR org.candlepin.async.JobManager - Job "CertificateCleanupJob" failed in 55ms javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154) at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181) at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1614) at org.candlepin.model.IdentityCertificateCurator.deleteByIds(IdentityCertificateCurator.java:74) at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:56) at org.candlepin.async.tasks.CertificateCleanupJob.cleanupExpiredIdentityCerts(CertificateCleanupJob.java:91) at org.candlepin.async.tasks.CertificateCleanupJob.execute(CertificateCleanupJob.java:70) at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:64) at org.candlepin.async.JobManager.executeJob(JobManager.java:1333) at org.candlepin.async.JobMessageReceiver$MessageListener.handleMessage(JobMessageReceiver.java:394) at org.candlepin.messaging.impl.artemis.ArtemisConsumer$ArtemisMessageForwarder.onMessage(ArtemisConsumer.java:59) at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1030) at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:49) at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1153) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:109) at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:200) at org.hibernate.hql.internal.ast.exec.BasicExecutor.doExecute(BasicExecutor.java:100) at org.hibernate.hql.internal.ast.exec.BasicExecutor.execute(BasicExecutor.java:59) at org.hibernate.hql.internal.ast.exec.DeleteExecutor.execute(DeleteExecutor.java:113) at org.hibernate.hql.internal.ast.QueryTranslatorImpl.executeUpdate(QueryTranslatorImpl.java:454) at org.hibernate.engine.query.spi.HQLQueryPlan.performExecuteUpdate(HQLQueryPlan.java:377) at org.hibernate.internal.SessionImpl.executeUpdate(SessionImpl.java:1420) at org.hibernate.query.internal.AbstractProducedQuery.doExecuteUpdate(AbstractProducedQuery.java:1623) at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1605) ... 17 common frames omitted Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "cp_id_cert" violates foreign key constraint "fk54b0f288a0b39916" on table "cp_upstream_consumer" Detail: Key (id)=(4cc35369617122e3016171cfa631004e) is still referenced from table "cp_upstream_consumer". at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155) at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:132) at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:462) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197) ... 25 common frames omitted Expected results: The CertificateCleanupJob finishes successfully, without errors in the log. Additional info:
Followed the directions in the description. Expected result: The CertificateCleanupJob finishes successfully, without errors in the log. Actual: The CertificateCleanupJob finishes successfully, without errors in the log. I checked the log and there was no errors and no warning associated with the issue outline in this bz. Verified on 6.12.0 snap 11 with candlepin-4.1.14-1.el8sat.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Satellite 6.12 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:8506