Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: ----------------------- Azure metrics collection is broken in CFME 5.11.0.21. It last worked in 5.11.0.19. Here's a snippet from evm.log: [----] E, [2019-08-22T18:51:52.469259 #13121:2b2740a605bc] ERROR -- : MIQ(MiqQueue#deliver) Message id: [758], Error: [undefined method `[]' for true:TrueClass] [----] E, [2019-08-22T18:51:52.469462 #13121:2b2740a605bc] ERROR -- : [NoMethodError]: undefined method `[]' for true:TrueClass Method:[block (2 levels) in <class:LogProxy>] [----] E, [2019-08-22T18:51:52.469528 #13121:2b2740a605bc] ERROR -- : /var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:201:in `block in just_perf_capture' /opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-ca1c762f8036/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/cfme-gems-pending-ca1c762f8036/lib/gems/pending/util/extensions/miq-benchmark.rb:35:in `realtime_block' /var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:195:in `just_perf_capture' /var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:141:in `perf_capture' /var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:117:in `perf_capture_realtime' /var/www/miq/vmdb/app/models/miq_queue.rb:479:in `block in dispatch_method' /usr/share/ruby/timeout.rb:93:in `block in timeout' /usr/share/ruby/timeout.rb:33:in `block in catch' /usr/share/ruby/timeout.rb:33:in `catch' /usr/share/ruby/timeout.rb:33:in `catch' /usr/share/ruby/timeout.rb:108:in `timeout' /var/www/miq/vmdb/app/models/miq_queue.rb:477:in `dispatch_method' /var/www/miq/vmdb/app/models/miq_queue.rb:454:in `block in deliver' /var/www/miq/vmdb/app/models/user.rb:290:in `with_user_group' /var/www/miq/vmdb/app/models/miq_queue.rb:454:in `deliver' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:104:in `deliver_queue_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:137:in `deliver_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:155:in `block in do_work' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `loop' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `do_work' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:329:in `block in do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:326:in `loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:326:in `do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:153:in `run' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:127:in `start' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:22:in `start_worker' /var/www/miq/vmdb/app/models/miq_worker.rb:398:in `block in start_runner_via_fork' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.4/lib/nakayoshi_fork.rb:23:in `fork' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.4/lib/nakayoshi_fork.rb:23:in `fork' /var/www/miq/vmdb/app/models/miq_worker.rb:396:in `start_runner_via_fork' /var/www/miq/vmdb/app/models/miq_worker.rb:386:in `start_runner' /var/www/miq/vmdb/app/models/miq_worker.rb:437:in `start' Version-Release number of selected component (if applicable): ------------------------------------------------------------- 5.11.0.21 How reproducible: ----------------- Always Steps to Reproduce: ------------------- 1.Manage Azure 2.Enable C&U collection for Azure 3.Check if data records are being written into the metrics table Actual results: -------------- Azure metrics collection is broken in CFME 5.11.0.21 Expected results: ----------------- It should be possible for CFME to fetch metrics data from Azure providers Additional info: ----------------
Nandini can you attach the logs?
Also, are you sure it worked in 5.11.0.19? There were 0 changes to the azure provider between 5.11.0.19 and 5.11.0.21. There was a fix for azure metrics collection that landed in 5.11.0.19 which might have broken something, but I believe azure metrics collection was totally broken before.
We need logs or access to the system
The interesting log message is just above: [----] I, [2019-08-22T16:24:15.076229 #7451:2b264d7ac5bc] INFO -- : MIQ(ManageIQ::Providers::Azure::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Azure::CloudManager: [azure] [----] W, [2019-08-22T16:24:16.595813 #9276:2b264d7ac5bc] WARN -- : MIQ(ManageIQ::Providers::Azure::CloudManager::MetricsCapture#perf_collect_metrics) Problem collecting metrics for cu-24x7/Automation. Region [eastus2] may not be supported. This triggers an exception handler which catches and logs the exception but doesn't re-raise it. The last line to execute is the _log.warn(), and that returns true. The calling code then sets counters_by_mor, counter_values_by_more = to the return of this method. We shouldn't be returning the result of _log.warn() but should instead just return nil.
https://github.com/ManageIQ/manageiq-providers-azure/pull/345
New commit detected on ManageIQ/manageiq-providers-azure/master: https://github.com/ManageIQ/manageiq-providers-azure/commit/17886fa2c484688646eb9c42d6a39f5e9a465540 commit 17886fa2c484688646eb9c42d6a39f5e9a465540 Author: Adam Grare <agrare> AuthorDate: Fri Aug 23 15:40:13 2019 -0400 Commit: Adam Grare <agrare> CommitDate: Fri Aug 23 15:40:13 2019 -0400 Always return counters and counter_values Always return two hashes from perf_capture_metrics because the calling method expects it. Catching an exception and logging the error without returning the proper values leads to: ``` [NoMethodError]: undefined method `[]' for true:TrueClass ``` Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1744845 app/models/manageiq/providers/azure/cloud_manager/metrics_capture.rb | 12 +- spec/models/manageiq/providers/azure/cloud_manager/metrics_capture_spec.rb | 16 +- 2 files changed, 22 insertions(+), 6 deletions(-)
New commit detected on ManageIQ/manageiq-providers-azure/ivanchuk: https://github.com/ManageIQ/manageiq-providers-azure/commit/5580800d8dda51769c023f794ae177e92ba4620f commit 5580800d8dda51769c023f794ae177e92ba4620f Author: Ladislav Smola <lsmola> AuthorDate: Fri Aug 23 17:36:45 2019 -0400 Commit: Ladislav Smola <lsmola> CommitDate: Fri Aug 23 17:36:45 2019 -0400 Merge pull request #345 from agrare/bz_1744845_azure_metrics_collection_failure Always return counters and counter_values (cherry picked from commit 6d6f4b97ba28e314f7afbb249e46fb1c62e960bd) https://bugzilla.redhat.com/show_bug.cgi?id=1744845 app/models/manageiq/providers/azure/cloud_manager/metrics_capture.rb | 12 +- spec/models/manageiq/providers/azure/cloud_manager/metrics_capture_spec.rb | 16 +- 2 files changed, 22 insertions(+), 6 deletions(-)
Fixed in 5.11.0.22
*** Bug 1762417 has been marked as a duplicate of this bug. ***