Bug 1614084

Summary: 10 mins gaps of metrics
Product: OpenShift Container Platform Reporter: Wang Haoran <haowang>
Component: HawkularAssignee: Ruben Vargas Palma <rvargasp>
Status: CLOSED DUPLICATE QA Contact: Junqi Zhao <juzhao>
Severity: high Docs Contact:
Priority: high    
Version: 3.9.0CC: achvatal, aos-bugs, ddelcian, haowang, razevedo, rkant, stobin
Target Milestone: ---Keywords: OpsBlocker
Target Release: 3.9.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-10 19:56:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
update_metrics_logs none

Comment 21 Rafael Azevedo 2018-08-29 19:29:50 UTC
Created attachment 1479563 [details]
update_metrics_logs

Comment 25 John Sanda 2018-09-10 20:49:09 UTC
Is there any customer update?

Comment 26 Daniel Del Ciancio 2018-09-10 21:28:31 UTC
Hi John,
We're coordinating the cleanup maintenance between the SRE operations team and the customer.

Thanks,
Dan.

Comment 27 Daniel Del Ciancio 2018-09-10 21:45:01 UTC
- Customer wants to know how long this maintenance would take?
If it takes long to run the cleanup, they would prefer running it during APAC hours (4am UTC window). 

- Can someone confirm?

Comment 28 John Sanda 2018-09-10 22:18:39 UTC
The table truncation and restarting the pods should take no more than a couple minutes. It might take a few more minutes for the index tables to get rebuilt which happens by heapster resending the tags. This happens though while everything is back up and running.

Comment 31 Daniel Del Ciancio 2018-10-05 15:57:20 UTC
Manual cleanup of metrics tag index tables performed and customer no longer seeing gaps.

Is there any way to have this cleanup incorporated into hawkular so that this gets done automatically?

Comment 32 John Sanda 2018-10-05 16:04:00 UTC
(In reply to Daniel Del Ciancio from comment #31)
> Manual cleanup of metrics tag index tables performed and customer no longer
> seeing gaps.
> 
> Is there any way to have this cleanup incorporated into hawkular so that
> this gets done automatically?

This is being discussed in bug 1569096.

Comment 34 Ruben Vargas Palma 2019-04-10 19:56:06 UTC

*** This bug has been marked as a duplicate of bug 1569096 ***