Hide Forgot
Description of problem: Unable to capture the perf roll up from kubernates with the described error Version-Release number of selected component (if applicable): cfme-5.7.0.17-1 How reproducible: Steps to Reproduce: 1. add kubernates provider to cloudforms 2. 3. Actual results: [----] I, [2017-01-25T16:00:29.122387 #3495:f57134] INFO -- : MIQ(ManageIQ::Providers::Kubernetes::ContainerManager::Container#perf_rollup_to_parents) Queueing [daily] rollup to ManageIQ::Providers::Kubernetes::ContainerManager::Container id: [352] in time profile: [UTC] for times: ["2017-01-25T00:00:00Z"]... [----] E, [2017-01-25T16:00:29.123473 #3495:f57134] ERROR -- : MIQ(MiqQueue#deliver) Message id: [1496012], Error: [Container#my_zone delegated to ext_management_system.my_zone, but ext_management_system is nil: #<ManageIQ::Providers::Kubernetes::ContainerManager::Container id: 352, ems_ref: "3df46e85-e308-11e6-8207-005056997454_jenkins_10.6....", restart_count: 0, state: "running", name: "jenkins", backing_ref: "docker://baae19fb6ed4db3ca0803333f7dc9730e87e40167...", last_perf_capture_on: "2017-01-25 14:27:40", container_definition_id: 352, type: "ManageIQ::Providers::Kubernetes::ContainerManager:...", container_image_id: 224, reason: nil, started_at: nil, finished_at: nil, exit_code: nil, signal: nil, message: nil, last_state: nil, last_reason: nil, last_started_at: nil, last_finished_at: nil, last_exit_code: nil, last_signal: nil, last_message: nil, deleted_on: "2017-01-25 14:39:03", ems_id: nil, old_ems_id: 1>] [----] E, [2017-01-25T16:00:29.123658 #3495:f57134] ERROR -- : [Module::DelegationError]: Container#my_zone delegated to ext_management_system.my_zone, but ext_management_system is nil: #<ManageIQ::Providers::Kubernetes::ContainerManager: :Container id: 352, ems_ref: "3df46e85-e308-11e6-8207-005056997454_jenkins_10.6....", restart_count: 0, state: "running", name: "jenkins", backing_ref: "docker://baae19fb6ed4db3ca0803333f7dc9730e87e40167...", last_perf_capture_on: "2017-01-25 14:27:40", container_definition_id: 352, type: "ManageIQ::Providers::Kubernetes::ContainerManager:...", container_image_id: 224, reason: nil, started_at: nil, finished_at: nil, exit_code: nil, signal: nil, message: nil, last_state: nil, last_reason: nil, last_started_at: nil, last_finished_at: nil, last_exit_code: nil, last_signal: nil, last_message: nil, deleted_on: "2017-01-25 14:39:03", ems_id: nil, old_ems_id: 1> Method:[rescue in deliver] [----] E, [2017-01-25T16:00:29.123865 #3495:f57134] ERROR -- : /var/www/miq/vmdb/app/models/container.rb:21:in `rescue in my_zone' /var/www/miq/vmdb/app/models/container.rb:21:in `my_zone' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:62:in `perf_rollup_queue' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:30:in `block (3 levels) in perf_rollup_to_parents' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:30:in `each' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:30:in `block (2 levels) in perf_rollup_to_parents' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:27:in `each' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:27:in `block in perf_rollup_to_parents' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:14:in `each' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:14:in `each_slice' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:14:in `perf_rollup_to_parents' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:104:in `block in perf_rollup' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:82:in `perf_rollup' /var/www/miq/vmdb/app/models/miq_queue.rb:347:in `block in deliver' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:91:in `block in timeout' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `block in catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:106:in `timeout' /var/www/miq/vmdb/app/models/miq_queue.rb:343:in `deliver' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:106:in `deliver_queue_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:134:in `deliver_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:152:in `block in do_work' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:146:in `loop' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:146:in `do_work' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:334:in `block in do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:331:in `loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:331:in `do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:153:in `run' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:128:in `start' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:21:in `start_worker' /var/www/miq/vmdb/app/models/miq_worker.rb:343:in `block in start' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /var/www/miq/vmdb/app/models/miq_worker.rb:341:in `start' /var/www/miq/vmdb/app/models/miq_worker.rb:270:in `start_worker' /var/www/miq/vmdb/app/models/miq_worker.rb:150:in `block in sync_workers' /var/www/miq/vmdb/app/models/miq_worker.rb:150:in `times' /var/www/miq/vmdb/app/models/miq_worker.rb:150:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:52:in `block in sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers' /var/www/miq/vmdb/app/models/miq_server.rb:346:in `block in monitor' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/miq_server.rb:346:in `monitor' /var/www/miq/vmdb/app/models/miq_server.rb:368:in `block (2 levels) in monitor_loop' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/miq_server.rb:368:in `block in monitor_loop' /var/www/miq/vmdb/app/models/miq_server.rb:367:in `loop' /var/www/miq/vmdb/app/models/miq_server.rb:367:in `monitor_loop' /var/www/miq/vmdb/app/models/miq_server.rb:250:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:65:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:92:in `start' /var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>' [----] I, [2017-01-25T16:00:29.124018 #3495:f57134] INFO -- : MIQ(MiqQueue#delivered) Message id: [1496012], State: [error], Delivered in [0.051398686] seconds [----] I, [2017-01-25T16:00:29.139024 #3495:f57134] INFO -- : MIQ(MiqEmsMetricsProcessorWorker::Runner#get_message_via_drb) Message id: [1497724], MiqWorker id: [13], Zone: [default], Role: [ems_metrics_processor], Server: [], Ident: [ems_metrics_processor], Target id: [], Instance id: [328], Task id: [], Command: [ManageIQ::Providers::Kubernetes::ContainerManager::ContainerGroup.perf_rollup], Timeout: [1800], Priority: [100], State: [dequeue], Deliver On: [2017-01-25 15:00:00 UTC], Data: [], Args: ["2017-01-25T14:00:00Z", "hourly"], Dequeued in: [1056.771220319] seconds Expected results: perf collection proceeds Additional info:
Probably related to (or a duplicate of) bug 1408968. Ari, can you check?
>Probably related to (or a duplicate of) bug 1408968. Federico, yes its the same issue with zones of archived entities.
https://github.com/ManageIQ/manageiq/pull/13686
New commit detected on ManageIQ/manageiq/euwe: https://github.com/ManageIQ/manageiq/commit/b06ec630197334af8d98d20b4910939a5a3b1d7b commit b06ec630197334af8d98d20b4910939a5a3b1d7b Author: Jason Frey <fryguy9> AuthorDate: Fri Feb 24 09:18:21 2017 -0500 Commit: Joe Rafaniello <jrafanie> CommitDate: Fri Feb 24 10:18:37 2017 -0500 Merge pull request #13686 from zeari/cpature_dead_containers https://bugzilla.redhat.com/show_bug.cgi?id=1420721 Collect metrics for archived containers (cherry picked from commit 5c98cf715d1f3a21e7526a4fe558986cdc6d9da6) app/models/container.rb | 4 +--- app/models/container_definition.rb | 1 + app/models/container_group.rb | 4 +--- app/models/container_project.rb | 6 +----- .../metrics_capture/capture_context.rb | 2 +- app/models/metric/ci_mixin/capture.rb | 4 +++- app/models/mixins/archived_mixin.rb | 22 +++++++++++++++++++ spec/models/metric/ci_mixin/capture_spec.rb | 25 ++++++++++++++++++++++ 8 files changed, 55 insertions(+), 13 deletions(-) create mode 100644 app/models/mixins/archived_mixin.rb
*** Bug 1429626 has been marked as a duplicate of this bug. ***
Verified. Metrics are collected and no errors (mentioned in this BZ) are seen in the logs.
*** Bug 1408968 has been marked as a duplicate of this bug. ***