Bug 1468725

Summary:	[paid] Reconciler missing timestamp annotation to complete support subscriptions
Product:	OpenShift Online	Reporter:	Mark Turansky <mturansk>
Component:	Accounts and Billing	Assignee:	Mark Turansky <mturansk>
Status:	CLOSED CURRENTRELEASE	QA Contact:	zhaliu
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	3.x	CC:	aos-bugs, jokerman, mmccomas, yufchang
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-11-09 18:49:33 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Mark Turansky 2017-07-07 18:12:34 UTC

Description of problem:

When an error occurs while subscribing to support, the reconciler is expected to pick up the failed transaction and complete it. The reconciler looks for a timestamp (to determine how old the pending TX is) and fails when that timestamp is missing from the User's annotations.

Version-Release number of selected component (if applicable):


How reproducible:

Always


Steps to Reproduce:

Difficult to reproduce. It requires either a failed Aria API call or a failed OpenShift User update. Both are difficult to purposefully create.

Actual results:


Expected results:


Additional info:

Comment 1 Mark Turansky 2017-07-07 18:13:04 UTC

Fix in https://github.com/openshift/online/pull/1343

Comment 2 Abhishek Gupta 2017-07-19 20:01:47 UTC

The fix is now available in INT

Comment 3 zhaliu 2017-07-20 02:45:34 UTC

From the log of account pod in online-int, I find an error occurs again and again, that is "reconciler error for user "yasun-406": error reconciling user &TypeMeta{Kind:,APIVersion:,}: invalid user metadata: missing "openshift.io/support-start-time" annotation", it seems that the reconciler can't handle the the user with missing "openshift.io/support-start-time" annotation successfully.
I will add all the log to attachment.

Comment 5 Mark Turansky 2017-08-15 21:06:07 UTC

Can you verify this with a new account?

I suspect the account you used was created before the fix was deployed. As a result, it wouldn't have the required annotation and would still continue to fail in the reconciler.  Making your account work would require a "migration" of all user accounts that are missing the annotation. We're not currently planning on doing this 'migration' because we don't see this error in prod logs.

Comment 6 zhaliu 2017-08-16 03:11:49 UTC

I have looked up the account pod logs from online-int and online-stg, the account pod in online-stg has been running for 13d, except for several old user accounts, there are no new accounts meeting the similar problem. The pod account pod in online-int has been running for 3 hours, there are no similar problems too.
Because of no problem found now in online stg and online int and the difficulty in reproducing this problem ,I will change the status into "verified" temporally. If the problem occurs again I will reopen that.