Bug 1443903

Summary: Metrics collection for osp nodes failed on RHOS11
Product: Red Hat CloudForms Management Engine Reporter: Ronnie Rasouli <rrasouli>
Component: ProvidersAssignee: Marek Aufart <maufart>
Status: CLOSED CURRENTRELEASE QA Contact: Ido Ovadia <iovadia>
Severity: high Docs Contact:
Priority: high    
Version: 5.8.0CC: cpelland, iovadia, jfrey, jhardy, obarenbo, simaishi, tzumainn
Target Milestone: GAKeywords: TestOnly
Target Release: 5.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: openstack
Fixed In Version: 5.9.0.1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1446739 (view as bug list) Environment:
Last Closed: 2018-03-06 14:37:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: Openstack Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1446739    

Description Ronnie Rasouli 2017-04-20 08:35:34 UTC
Description of problem:

In RHOS11 after upgrade, metrics collection for Compute\infrastructure\nodes are not being displayed.

Version-Release number of selected component (if applicable):
5.8.0.10-beta1

How reproducible:
100%

Steps to Reproduce:
1. enable metrics collection on CFME configuration
2. set on advanced configuration.ems_metrics_openstack_default_service to  'gnocchi'
3. navigate to: compute --> infrastructure --> nodes
4. click on monitoring 
Actual results:

Both, utilization and timelines are disabled

Expected results:

collect metrics from OSP nodes

Additional info:
[----] E, [2017-04-20T04:18:24.001862 #31459:95b140] ERROR -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::MetricsCapture#perf_collect_metrics) [realtime] for: [ManageIQ::Providers::Openstack::InfraManager::Host], [51000000000001], [a7df62aa-6990-4d34-a4ac-6a967da0a416 (CephStorage)] Unhandled exception during perf data collection: [Unexpected Metric filter "metadata.resource_id"], class: [RuntimeError]
[----] E, [2017-04-20T04:18:24.002184 #31459:95b140] ERROR -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::MetricsCapture#perf_collect_metrics) [realtime] for: [ManageIQ::Providers::Openstack::InfraManager::Host], [51000000000001], [a7df62aa-6990-4d34-a4ac-6a967da0a416 (CephStorage)]   Timings at time of error: {:heartbeat=>0.005605936050415039, :server_dequeue=>0.0038001537322998047, :server_monitor=>0.0844120979309082, :log_active_servers=>0.0070497989654541016, :capture_state=>7.9481024742126465, :connect=>1179.7523140907288, :capture_counters=>18.63476061820984, :total_time=>1321.0389595031738, :capture_counter_values=>21.70003080368042, :process_counter_values=>0.1471097469329834, :db_find_prev_perfs=>4.5457541942596436, :process_perfs=>11.094048976898193, :process_perfs_db=>70.83825826644897}
[----] E, [2017-04-20T04:18:24.002419 #31459:95b140] ERROR -- : [RuntimeError]: Unexpected Metric filter "metadata.resource_id"  Method:[rescue in perf_collect_metrics]
[----] E, [2017-04-20T04:18:24.002622 #31459:95b140] ERROR -- : /opt/rh/rh-ruby23/root/usr/share/ruby/delegate.rb:85:in `call'
/opt/rh/rh-ruby23/root/usr/share/ruby/delegate.rb:85:in `method_missing'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/openstack/openstack_handle/metric_delegate.rb:21:in `block in list_meters'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/openstack/openstack_handle/metric_delegate.rb:19:in `each'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/openstack/openstack_handle/metric_delegate.rb:19:in `list_meters'
/var/www/miq/vmdb/app/models/manageiq/providers/openstack/base_metrics_capture.rb:81:in `block in list_meters'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/var/www/miq/vmdb/app/models/manageiq/providers/openstack/base_metrics_capture.rb:80:in `list_meters'
/var/www/miq/vmdb/app/models/manageiq/providers/openstack/base_metrics_capture.rb:116:in `list_metadata_meters'
/var/www/miq/vmdb/app/models/manageiq/providers/openstack/base_metrics_capture.rb:97:in `find_meter_counters'
/var/www/miq/vmdb/app/models/manageiq/providers/openstack/base_metrics_capture.rb:51:in `perf_capture_data_openstack_base'
/var/www/miq/vmdb/app/models/manageiq/providers/openstack/infra_manager/metrics_capture.rb:125:in `perf_capture_data'
/var/www/miq/vmdb/app/models/manageiq/providers/openstack/base_metrics_capture.rb:14:in `perf_collect_metrics'
/var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:6:in `perf_collect_metrics'
/var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:180:in `block in perf_capture'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:177:in `perf_capture'
/var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:117:in `perf_capture_realtime'
/var/www/miq/vmdb/app/models/miq_queue.rb:347:in `block in deliver'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:91:in `block in timeout'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `block in catch'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:106:in `timeout'
/var/www/miq/vmdb/app/models/miq_queue.rb:343:in `deliver'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:107:in `deliver_queue_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:135:in `deliver_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:153:in `block in do_work'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:147:in `loop'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:147:in `do_work'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:336:in `block in do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:333:in `loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:333:in `do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:155:in `run'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:130:in `start'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:21:in `start_worker'
/var/www/miq/vmdb/app/models/miq_worker.rb:339:in `block in start_runner'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork'
/var/www/miq/vmdb/app/models/miq_worker.rb:337:in `start_runner'
/var/www/miq/vmdb/app/models/miq_worker.rb:348:in `start'
/var/www/miq/vmdb/app/models/miq_worker.rb:266:in `start_worker'
/var/www/miq/vmdb/app/models/miq_worker.rb:150:in `block in sync_workers'
/var/www/miq/vmdb/app/models/miq_worker.rb:150:in `times'
/var/www/miq/vmdb/app/models/miq_worker.rb:150:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:53:in `block in sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers'
/var/www/miq/vmdb/app/models/miq_server.rb:348:in `block in monitor'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/var/www/miq/vmdb/app/models/miq_server.rb:348:in `monitor'
/var/www/miq/vmdb/app/models/miq_server.rb:370:in `block (2 levels) in monitor_loop'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-3eb0dd31cff7/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/var/www/miq/vmdb/app/models/miq_server.rb:370:in `block in monitor_loop'
/var/www/miq/vmdb/app/models/miq_server.rb:369:in `loop'
/var/www/miq/vmdb/app/models/miq_server.rb:369:in `monitor_loop'
/var/www/miq/vmdb/app/models/miq_server.rb:252:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:65:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:91:in `start'
/var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>'

Comment 3 CFME Bot 2017-04-28 15:43:07 UTC
New commit detected on ManageIQ/manageiq-gems-pending/master:
https://github.com/ManageIQ/manageiq-gems-pending/commit/f445f9d431e76fd5de1bc78ca7c507d50094c71c

commit f445f9d431e76fd5de1bc78ca7c507d50094c71c
Author:     Marek Aufart <maufart>
AuthorDate: Thu Apr 27 16:16:45 2017 +0200
Commit:     Marek Aufart <maufart>
CommitDate: Fri Apr 28 17:27:13 2017 +0200

    Fix Openstack Metrics resource_id field
    
    Openstack Gnocchi metrics store in some cases resource_id under
    metadata, which changes its name to metadata.resource_id.
    Updating code to accept it.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1443903

 lib/gems/pending/openstack/openstack_handle/metric_delegate.rb | 1 +
 1 file changed, 1 insertion(+)

Comment 5 Ido Ovadia 2017-12-10 13:09:20 UTC
Verified
========
5.9.0.12