Bug 1418376

Summary: Bad ueber cert data can potentially occur after applying previous cleanup patches
Product: [Community] Candlepin Reporter: Michael Stead <mstead>
Component: candlepinAssignee: Michael Stead <mstead>
Status: CLOSED CURRENTRELEASE QA Contact: Katello QA List <katello-qa-list>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.9.54CC: bcourt, redakkan, skallesh, vrjain
Target Milestone: ---   
Target Release: 0.9.54   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: candlepin-0.9.54.19-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1418406 (view as bug list) Environment:
Last Closed: 2017-02-07 17:37:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1418406, 1533140    
Attachments:
Description Flags
A script to manually clean up all ueber certs. none

Description Michael Stead 2017-02-01 16:43:50 UTC 
A user had reported that they couldn't perform a capsule sync on satellite. Upon further investigation, their ueber cert data was not in the correct state, even after applying the cleanup that was released with an update.

The reported error was along the lines of:

2017-02-01 12:14:14,368 [thread=http-nio-8443-exec-10] [req=5c42b668-c68c-45cc-b2eb-a7a2b321f64c, org=admin] ERROR org.candlepin.model.UeberCertificateGenerator - Problem generating ueber cert for owner: admin
org.hibernate.NonUniqueResultException: query did not return a unique result: 2
	at org.hibernate.internal.AbstractQueryImpl.uniqueElement(AbstractQueryImpl.java:914) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
	at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:396) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
	at org.candlepin.model.PoolCurator.findUeberPool(PoolCurator.java:337) ~[classes/:]
	at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:58) ~[guice-persist-3.0.jar:na]
	at org.candlepin.model.UeberCertificateGenerator.generate(UeberCertificateGenerator.java:110) ~[classes/:]
	at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:66) ~[guice-persist-3.0.jar:na]


The cleanup queries supplied in previous candlepin updates missed some of the bad ueber pools because they didn't account for candlepin's loose product references in 0.9.54 (cp_pool.product_id is not a foreign_key). Because of this, an ueber pool can exist in the DB without the actual ueber product being there. The cleanup queries unfortunately relied on this relationship when deleting the bad data.

While the product/pool relationship should always be present, older bugs in ueber cert generation and/or cleanup patches, could potentially allow the pool to exist without the product.


Steps to Reproduce:
I was unable to determine the steps to reproduce from an installed candlepin, however, there is a way to manually put candlepin in the bad state to test the fix.

Once the patch is in github, I'll post the testing setup details here.
Comment 1 Michael Stead 2017-02-01 19:05:15 UTC 
See the github tracker in this BZ to see manual steps to trigger this issue.
Comment 2 Michael Stead 2017-02-07 12:00:35 UTC 
Created attachment 1248373 [details]
A script to manually clean up all ueber certs.