A user had reported that they couldn't perform a capsule sync on satellite. Upon further investigation, their ueber cert data was not in the correct state, even after applying the cleanup that was released with an update.
The reported error was along the lines of:
2017-02-01 12:14:14,368 [thread=http-nio-8443-exec-10] [req=5c42b668-c68c-45cc-b2eb-a7a2b321f64c, org=admin] ERROR org.candlepin.model.UeberCertificateGenerator - Problem generating ueber cert for owner: admin
org.hibernate.NonUniqueResultException: query did not return a unique result: 2
at org.hibernate.internal.AbstractQueryImpl.uniqueElement(AbstractQueryImpl.java:914) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:396) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
at org.candlepin.model.PoolCurator.findUeberPool(PoolCurator.java:337) ~[classes/:]
at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:58) ~[guice-persist-3.0.jar:na]
at org.candlepin.model.UeberCertificateGenerator.generate(UeberCertificateGenerator.java:110) ~[classes/:]
at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:66) ~[guice-persist-3.0.jar:na]
The cleanup queries supplied in previous candlepin updates missed some of the bad ueber pools because they didn't account for candlepin's loose product references in 0.9.54 (cp_pool.product_id is not a foreign_key). Because of this, an ueber pool can exist in the DB without the actual ueber product being there. The cleanup queries unfortunately relied on this relationship when deleting the bad data.
While the product/pool relationship should always be present, older bugs in ueber cert generation and/or cleanup patches, could potentially allow the pool to exist without the product.
Steps to Reproduce:
I was unable to determine the steps to reproduce from an installed candlepin, however, there is a way to manually put candlepin in the bad state to test the fix.
Once the patch is in github, I'll post the testing setup details here.
See the github tracker in this BZ to see manual steps to trigger this issue.
Created attachment 1248373 [details]
A script to manually clean up all ueber certs.