Description of problem: On an upstream build from July 14 2017, there is a problem to do a full refresh, for a RHV-4.1 provider. It fails with this error message on provider summary page: "Error - 4 Minutes Ago undefined local variable or method `connection' for #<ManageIQ::Providers::Redhat::Inventory::Collector::InfraManager..." Version-Release number of selected component (if applicable): master.20170725061757_604c5f9 (but it is a nightly from July 14 2017). How reproducible: Tried twice, and it failed the same. evm.log: [----] E, [2017-07-25T06:23:40.341709 #13328:b89138] ERROR -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::Refresh::Strategies::Api3#refresh) EMS: [rhv-4.1], id: [1] Refresh failed [----] E, [2017-07-25T06:23:40.351905 #13328:b89138] ERROR -- : [NameError]: undefined local variable or method `connection' for #<ManageIQ::Providers::Redhat::Inventory::Collector::InfraManager:0x00000002dea5b0> Did you mean? concern Method:[block in method_missing] [----] E, [2017-07-25T06:23:40.352575 #13328:b89138] ERROR -- : /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/collector.rb:89:in `collect_disks_as_hash' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/collector.rb:83:in `preloaded_disks' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/collector.rb:76:in `block in collect_attached_disks' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/infra_manager/api_integration.rb:125:in `with_provider_connection' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/collector.rb:75:in `collect_attached_disks' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:369:in `storages' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:332:in `block in vms' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:321:in `each' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:321:in `vms' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:11:in `parse' /var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:37:in `block in inventory_collections' /var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:34:in `each' /var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:34:in `inventory_collections' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/infra_manager/refresh/refresher.rb:65:in `block in parse_targeted_inventory' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/infra_manager/refresh/refresher.rb:63:in `parse_targeted_inventory' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:87:in `block in refresh_targets_for_ems' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:86:in `refresh_targets_for_ems' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block (2 levels) in refresh' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block in refresh' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `each' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `refresh' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `block in refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:96:in `each' /var/www/miq/vmdb/app/models/ems_refresh.rb:96:in `refresh' /var/www/miq/vmdb/app/models/miq_queue.rb:387:in `block in deliver' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `block in catch' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:106:in `timeout' /var/www/miq/vmdb/app/models/miq_queue.rb:386:in `deliver' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:108:in `deliver_queue_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:137:in `deliver_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:155:in `block in do_work' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `loop' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `do_work' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:344:in `block in do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:341:in `loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:341:in `do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:162:in `run' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:136:in `start' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:23:in `start_worker' /var/www/miq/vmdb/app/models/miq_worker.rb:357:in `block in start_runner_via_fork' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /var/www/miq/vmdb/app/models/miq_worker.rb:355:in `start_runner_via_fork' /var/www/miq/vmdb/app/models/miq_worker.rb:349:in `start_runner' /var/www/miq/vmdb/app/models/miq_worker.rb:383:in `start' /var/www/miq/vmdb/app/models/miq_worker.rb:274:in `start_worker' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:68:in `start_worker_for_ems' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:46:in `block in sync_workers' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:45:in `each' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:45:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:53:in `block in sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers' /var/www/miq/vmdb/app/models/miq_server.rb:346:in `block in monitor' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/miq_server.rb:346:in `monitor' /var/www/miq/vmdb/app/models/miq_server.rb:368:in `block (2 levels) in monitor_loop' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/miq_server.rb:368:in `block in monitor_loop' /var/www/miq/vmdb/app/models/miq_server.rb:367:in `loop' /var/www/miq/vmdb/app/models/miq_server.rb:367:in `monitor_loop' /var/www/miq/vmdb/app/models/miq_server.rb:250:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:27:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:48:in `start' /var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>' [----] E, [2017-07-25T06:23:40.352855 #13328:b89138] ERROR -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::Refresh::Strategies::Api3#refresh) EMS: [rhv-4.1], id: [1] Unable to perform refresh for the following targets: [----] E, [2017-07-25T06:23:40.353000 #13328:b89138] ERROR -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::Refresh::Strategies::Api3#refresh) --- ManageIQ::Providers::Redhat::InfraManager [rhv-4.1] id [1] [----] I, [2017-07-25T06:23:40.372044 #13328:b89138] INFO -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::Refresh::Strategies::Api3#refresh) Refreshing all targets...Complete [----] E, [2017-07-25T06:23:40.378019 #13328:b89138] ERROR -- : MIQ(MiqQueue#deliver) Message id: [106], Error: [undefined local variable or method `connection' for #<ManageIQ::Providers::Redhat::Inventory::Collector::InfraManager:0x00000002dea5b0> Did you mean? concern] [----] E, [2017-07-25T06:23:40.378372 #13328:b89138] ERROR -- : [EmsRefresh::Refreshers::EmsRefresherMixin::PartialRefreshError]: undefined local variable or method `connection' for #<ManageIQ::Providers::Redhat::Inventory::Collector::InfraManager:0x00000002dea5b0> Did you mean? concern Method:[block in method_missing] [----] E, [2017-07-25T06:23:40.378673 #13328:b89138] ERROR -- : /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:50:in `refresh' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `block in refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:96:in `each' /var/www/miq/vmdb/app/models/ems_refresh.rb:96:in `refresh' /var/www/miq/vmdb/app/models/miq_queue.rb:387:in `block in deliver' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `block in catch' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch' /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:106:in `timeout' /var/www/miq/vmdb/app/models/miq_queue.rb:386:in `deliver' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:108:in `deliver_queue_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:137:in `deliver_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:155:in `block in do_work'
Created attachment 1304207 [details] evm.log
Created attachment 1304208 [details] automation.log
Created attachment 1304209 [details] production.log
Created attachment 1304210 [details] rhem log
Created attachment 1304211 [details] engine.log
Fixed by: https://github.com/ManageIQ/manageiq-providers-ovirt/pull/68
Verified that performing "Refresh Relationships and Power States" succeeds. CFME version 5.9.0.2, RHV provider Version 4.1.3.4