Description of problem: Rollup messages for class MiqRegion are consistently failing Version-Release number of selected component (if applicable): 5.5.0.9-beta2 How reproducible: start C&U and wait for MiqRegion rollups scheduled to begin Steps to Reproduce: 1. 2. 3. Actual results: message terminates in error Expected results: rollups should complete without error. Additional info: full log sequence from failing rollup message processing: ===== [----] I, [2015-11-08T18:22:48.002377 #22633:99b998] INFO -- : MIQ(MiqEmsMetricsProcessorWorker::Runner#get_message_via_drb) Message id: [441000000363311], MiqWorker id: [441000000002068], Zone: [default], Role: [ems_metrics_processor], Server: [], Ident: [ems_metrics_processor], Target id: [], Instance id: [441000000000001], Task id: [], Command: [MiqRegion.perf_rollup], Timeout: [1800], Priority: [100], State: [dequeue], Deliver On: [2015-11-08 23:00:00 UTC], Data: [], Args: ["2015-11-08T22:00:00Z", "hourly"], Dequeued in: [3.183835599] seconds [----] I, [2015-11-08T18:22:48.045191 #22633:99b998] INFO -- : MIQ(MiqRegion#perf_rollup) [hourly] Rollup for MiqRegion name: [Region 441], id: [441000000000001] for time: [2015-11-08T22:00:00Z]... [----] E, [2015-11-08T18:22:48.602061 #22633:99b998] ERROR -- : MIQ(MiqQueue#deliver) Message id: [441000000363311], Error: [Association named 'hosts' was not found on MiqRegion; perhaps you misspelled it?] [----] E, [2015-11-08T18:22:48.602407 #22633:99b998] ERROR -- : [ActiveRecord::AssociationNotFoundError]: Association named 'hosts' was not found on MiqRegion; perhaps you misspelled it? Method:[rescue in deliver] [----] E, [2015-11-08T18:22:48.602775 #22633:99b998] ERROR -- : /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/activerecord/lib/active_record/associations.rb:161:in `association' /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/activerecord/lib/active_record/associations/builder/collection_association.rb:66:in `host_ids' /var/www/miq/vmdb/app/models/mixins/aggregation_mixin.rb:107:in `aggregate_hardware' /var/www/miq/vmdb/app/models/mixins/aggregation_mixin.rb:35:in `aggregate_cpu_speed' /var/www/miq/vmdb/app/models/vim_performance_state.rb:131:in `capture_total' /var/www/miq/vmdb/app/models/vim_performance_state.rb:60:in `capture' /var/www/miq/vmdb/app/models/metric/ci_mixin/capture.rb:209:in `perf_capture_state' /var/www/miq/vmdb/app/models/metric/ci_mixin/state_finders.rb:21:in `vim_performance_state_for_ts' /var/www/miq/vmdb/app/models/metric/ci_mixin/state_finders.rb:41:in `ext_management_systems_from_vim_performance_state_for_ts' /var/www/miq/vmdb/app/models/metric/rollup.rb:189:in `rollup_child_metrics' /var/www/miq/vmdb/app/models/metric/rollup.rb:131:in `block in rollup_hourly' /var/www/miq/vmdb/app/models/metric/rollup.rb:131:in `each' /var/www/miq/vmdb/app/models/metric/rollup.rb:131:in `rollup_hourly' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:91:in `block (2 levels) in perf_rollup' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:90:in `block in perf_rollup' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /var/www/miq/vmdb/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/metric/ci_mixin/rollup.rb:77:in `perf_rollup' /var/www/miq/vmdb/app/models/miq_queue.rb:345:in `block in deliver' /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:89:in `block in timeout' /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:34:in `block in catch' /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:34:in `catch' /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:34:in `catch' /opt/rh/rh-ruby22/root/usr/share/ruby/timeout.rb:104:in `timeout' /var/www/miq/vmdb/app/models/miq_queue.rb:341:in `deliver' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:106:in `deliver_queue_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:134:in `deliver_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:151:in `block in do_work' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:145:in `loop' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:145:in `do_work' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:323:in `block in do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:320:in `loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:320:in `do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:142:in `run' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:123:in `start' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:24:in `start_worker' /var/www/miq/vmdb/lib/workers/bin/worker.rb:2:in `<top (required)>' /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/railties/lib/rails/commands/runner.rb:60:in `load' /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/railties/lib/rails/commands/runner.rb:60:in `<top (required)>' /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/railties/lib/rails/commands/commands_tasks.rb:123:in `require' /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/railties/lib/rails/commands/commands_tasks.rb:123:in `require_command!' /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/railties/lib/rails/commands/commands_tasks.rb:90:in `runner' /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/railties/lib/rails/commands/commands_tasks.rb:39:in `run_command!' /opt/rh/cfme-gemset/bundler/gems/rails-a37d0e57571f/railties/lib/rails/commands.rb:17:in `<top (required)>' /var/www/miq/vmdb/bin/rails:4:in `require' /var/www/miq/vmdb/bin/rails:4:in `<main>' [----] I, [2015-11-08T18:22:48.603324 #22633:99b998] INFO -- : MIQ(MiqQueue#delivered) Message id: [441000000363311], State: [error], Delivered in [0.60054032] seconds =====
*** This bug has been marked as a duplicate of bug 1277300 ***