This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1465086 - Hourly metrics_## tables grow filling up the VMDB filesystem when real-time purges fail
Hourly metrics_## tables grow filling up the VMDB filesystem when real-time p...
Status: VERIFIED
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Appliance (Show other bugs)
5.7.0
x86_64 Linux
medium Severity high
: GA
: 5.7.4
Assigned To: Jillian Tullo
Tasos Papaioannou
c&u
: ZStream
Depends On: 1445498
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-26 11:46 EDT by Satoe Imaishi
Modified: 2017-09-19 10:16 EDT (History)
15 users (show)

See Also:
Fixed In Version: 5.7.4.0
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1445498
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core


Attachments (Terms of Use)

  None (edit)
Comment 2 Jillian Tullo 2017-08-04 13:01:59 EDT
PR https://github.com/ManageIQ/manageiq/pull/15702
Comment 3 Tasos Papaioannou 2017-09-19 10:16:21 EDT
Verified on 5.8.1.0. Metric purging now uses 20 minute timeout, e.g.,

****
[----] I, [2017-09-19T03:50:12.715632 #3456:e3d12c]  INFO -- : MIQ(MiqQueue.put) Message id: [39327],  id: [], Zone: [], Role: [database_operations], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [VmdbMetric.purge_hourly], Timeout: [1200], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [2017-03-23 07:50:12 UTC]

[----] I, [2017-09-19T04:04:19.428016 #3447:e3d12c]  INFO -- : MIQ(MiqQueue.put) Message id: [39551],  id: [], Zone: [default], Role: [ems_metrics_processor], Server: [], Ident: [ems_metrics_processor], Target id: [], Instance id: [], Task id: [], Command: [Metric::Purging.purge_realtime], Timeout: [1200], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [2017-09-19 04:04:19 UTC]
****

After changing the database.metrics_history.queue_timeout and performance.history.queue_timeout settings to 10.minutes and restarting EVM server processes, the timeouts change to 10 minutes.

****
[----] I, [2017-09-19T09:23:21.079515 #19931:373138]  INFO -- : MIQ(MiqQueue.put) Message id: [47427],  id: [], Zone: [default], Role: [ems_metrics_processor], Server: [], Ident: [ems_metrics_processor], Target id: [], Instance id: [], Task id: [], Command: [Metric::Purging.purge_realtime], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [2017-09-19 09:23:21 UTC]

[----] I, [2017-09-19T09:50:09.367103 #19931:373138]  INFO -- : MIQ(MiqQueue.put) Message id: [47912],  id: [], Zone: [], Role: [database_operations], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [VmdbMetric.purge_hourly], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [2017-03-23 13:50:09 UTC]
****

Note You need to log in before you can comment on or make changes to this bug.