Description of problem: Metric_XX tables are growing exponentially and are not purging any of the data causing the database to grow in size rapidly. When running through purge_metrics ruby script encountering a undefined local variable or method error. Tested script on version 5.3.2.6 (Customer is reporting this on 5.2.1.8): ======================================================================================================================= [root@dhcp181-40 vmdb]# bin/rails r tools/purge_metrics.rb -- --mode=purge Purge Counts Realtime: 126,875 Hourly: 0 Daily: 0 Purging... /var/www/miq/vmdb/app/models/metric/purging.rb:111:in `block (2 levels) in purge': undefined local variable or method `count' for Metric::Purging:Module (NameError) | ETA: --:--:-- from /var/www/miq/vmdb/app/models/metric/purging.rb:81:in `loop' from /var/www/miq/vmdb/app/models/metric/purging.rb:81:in `block in purge' from /var/www/miq/lib/util/extensions/miq-benchmark.rb:12:in `realtime_store' from /var/www/miq/lib/util/extensions/miq-benchmark.rb:31:in `realtime_block' from /var/www/miq/vmdb/app/models/metric/purging.rb:72:in `purge' from tools/purge_metrics.rb:43:in `block in <top (required)>' from tools/purge_metrics.rb:41:in `each' from tools/purge_metrics.rb:41:in `<top (required)>' from /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.17/lib/rails/commands/runner.rb:52:in `eval' from /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.17/lib/rails/commands/runner.rb:52:in `<top (required)>' from /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.17/lib/rails/commands.rb:64:in `require' from /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.17/lib/rails/commands.rb:64:in `<top (required)>' from script/rails:6:in `require' from script/rails:6:in `<main>' =============================================================================================================================== Version-Release number of selected component (if applicable): 5.2.1.8 How reproducible: Very Steps to Reproduce: 1. SSH into appliance and navigate to /var/www/miq/vmdb/ directory 2. bin/rails r tools/purge_metrics.rb <--To get a count of what is there 3. Run bin/rails r tools/purge_metrics.rb -- --mode=purge Actual results: Script runs through the first batch of 1000 and then at the end of the loop script blows up before it makes it to the second batch (with above error) Expected results: Purge all records older then 4 hours Additional info: Will provide test appliance if needed
I have to verify this is a bug on 5.2, but it clearly a bug on upstream. I'm really surprised the specs did not find this as it is definitely tested.
customer for case 01377209 asks which data can be removed safely from the db to reduce the impact before he vacuums the database (external). he states that at the current groth rate, the appliance will no longer work in a couple weeks. as such requesting hotfix...
Yup, definitely seems to be the same bug on 5.2. I am simply amazed that no one else has seen this before.
New commit detected on manageiq/master: https://github.com/ManageIQ/manageiq/commit/f20800046e994760d5cc1bae7873e262ec398fd6 commit f20800046e994760d5cc1bae7873e262ec398fd6 Author: Jason Frey <jfrey> AuthorDate: Thu Mar 19 17:14:22 2015 -0400 Commit: Jason Frey <jfrey> CommitDate: Thu Mar 19 17:16:04 2015 -0400 Fix issue where count was undefined when purge called with a block. Note that this only really affects calling .purge from the tools/purge_metrics.rb script. Since normal scheduled purging does not pass the block it works correctly. https://bugzilla.redhat.com/show_bug.cgi?id=1201932 vmdb/app/models/metric/purging.rb | 14 +++--- vmdb/spec/models/metric/purging_spec.rb | 76 +++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+), 7 deletions(-) create mode 100644 vmdb/spec/models/metric/purging_spec.rb
Created attachment 1006871 [details] explain_metric_query Explain queries for the metrics table.
Jason, can this be cherry-pick back to 5.3.z?
Verified in 5.4.0.0.24
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-1100.html
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days