Bug 1474831

Summary: Full refresh fail on "undefined local variable or method `connection' for #<ManageIQ..."
Product: Red Hat CloudForms Management Engine Reporter: Ilanit Stein <istein>
Component: ProvidersAssignee: Boriso <bodnopoz>
Status: CLOSED CURRENTRELEASE QA Contact: Angelina Vasileva <anikifor>
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: gblomqui, jfrey, jhardy, obarenbo, rhrazdil, simaishi
Target Milestone: GAKeywords: TestOnly
Target Release: 5.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: rhev
Fixed In Version: 5.9.0.1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-03-06 15:06:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Bug
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: RHEVM Target Upstream Version:
Embargoed:
Attachments:
Description Flags
evm.log
none
automation.log
none
production.log
none
rhem log
none
engine.log none

Description Ilanit Stein 2017-07-25 13:24:55 UTC
Description of problem:
On an upstream build from July 14 2017, 
there is a problem to do a full refresh,
for a RHV-4.1 provider.
It fails with this error message on provider summary page:

"Error - 4 Minutes Ago
undefined local variable or method `connection' for #<ManageIQ::Providers::Redhat::Inventory::Collector::InfraManager..."

Version-Release number of selected component (if applicable):
master.20170725061757_604c5f9 (but it is a nightly from July 14 2017). 

How reproducible:
Tried twice, and it failed the same.

evm.log:

[----] E, [2017-07-25T06:23:40.341709 #13328:b89138] ERROR -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::Refresh::Strategies::Api3#refresh) EMS: [rhv-4.1], id: [1] Refresh failed 
[----] E, [2017-07-25T06:23:40.351905 #13328:b89138] ERROR -- : [NameError]: undefined local variable or method `connection' for #<ManageIQ::Providers::Redhat::Inventory::Collector::InfraManager:0x00000002dea5b0> 
Did you mean?  concern  Method:[block in method_missing] 
[----] E, [2017-07-25T06:23:40.352575 #13328:b89138] ERROR -- : /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/collector.rb:89:in `collect_disks_as_hash' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/collector.rb:83:in `preloaded_disks' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/collector.rb:76:in `block in collect_attached_disks' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/infra_manager/api_integration.rb:125:in `with_provider_connection' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/collector.rb:75:in `collect_attached_disks' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:369:in `storages' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:332:in `block in vms' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:321:in `each' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:321:in `vms' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/inventory/parser/infra_manager.rb:11:in `parse' 
/var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:37:in `block in inventory_collections' 
/var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:34:in `each' 
/var/www/miq/vmdb/app/models/manager_refresh/inventory.rb:34:in `inventory_collections' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/infra_manager/refresh/refresher.rb:65:in `block in parse_targeted_inventory' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-ovirt-985ded1b1db8/app/models/manageiq/providers/redhat/infra_manager/refresh/refresher.rb:63:in `parse_targeted_inventory' 
/var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:87:in `block in refresh_targets_for_ems' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' 
/var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:86:in `refresh_targets_for_ems' 
/var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block (2 levels) in refresh' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' 
/var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block in refresh' 
/var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `each' 
/var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `refresh' 
/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh' 
/var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `block in refresh' 
/var/www/miq/vmdb/app/models/ems_refresh.rb:96:in `each' 
/var/www/miq/vmdb/app/models/ems_refresh.rb:96:in `refresh' 
/var/www/miq/vmdb/app/models/miq_queue.rb:387:in `block in deliver' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `block in catch' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:106:in `timeout' 
/var/www/miq/vmdb/app/models/miq_queue.rb:386:in `deliver' 
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:108:in `deliver_queue_message' 
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:137:in `deliver_message' 
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:155:in `block in do_work' 
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `loop' 
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `do_work' 
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:344:in `block in do_work_loop' 
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:341:in `loop' 
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:341:in `do_work_loop' 
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:162:in `run' 
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:136:in `start' 
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:23:in `start_worker' 
/var/www/miq/vmdb/app/models/miq_worker.rb:357:in `block in start_runner_via_fork' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' 
/var/www/miq/vmdb/app/models/miq_worker.rb:355:in `start_runner_via_fork' 
/var/www/miq/vmdb/app/models/miq_worker.rb:349:in `start_runner' 
/var/www/miq/vmdb/app/models/miq_worker.rb:383:in `start' 
/var/www/miq/vmdb/app/models/miq_worker.rb:274:in `start_worker' 
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:68:in `start_worker_for_ems' 
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:46:in `block in sync_workers' 
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:45:in `each' 
/var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:45:in `sync_workers' 
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:53:in `block in sync_workers' 
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each' 
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers' 
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers' 
/var/www/miq/vmdb/app/models/miq_server.rb:346:in `block in monitor' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' 
/var/www/miq/vmdb/app/models/miq_server.rb:346:in `monitor' 
/var/www/miq/vmdb/app/models/miq_server.rb:368:in `block (2 levels) in monitor_loop' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' 
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-ca79d531a9d6/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' 
/var/www/miq/vmdb/app/models/miq_server.rb:368:in `block in monitor_loop' 
/var/www/miq/vmdb/app/models/miq_server.rb:367:in `loop' 
/var/www/miq/vmdb/app/models/miq_server.rb:367:in `monitor_loop' 
/var/www/miq/vmdb/app/models/miq_server.rb:250:in `start' 
/var/www/miq/vmdb/lib/workers/evm_server.rb:27:in `start' 
/var/www/miq/vmdb/lib/workers/evm_server.rb:48:in `start' 
/var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>' 
[----] E, [2017-07-25T06:23:40.352855 #13328:b89138] ERROR -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::Refresh::Strategies::Api3#refresh) EMS: [rhv-4.1], id: [1] Unable to perform refresh for the following targets: 
[----] E, [2017-07-25T06:23:40.353000 #13328:b89138] ERROR -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::Refresh::Strategies::Api3#refresh)  --- ManageIQ::Providers::Redhat::InfraManager [rhv-4.1] id [1] 
[----] I, [2017-07-25T06:23:40.372044 #13328:b89138]  INFO -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::Refresh::Strategies::Api3#refresh) Refreshing all targets...Complete 
[----] E, [2017-07-25T06:23:40.378019 #13328:b89138] ERROR -- : MIQ(MiqQueue#deliver) Message id: [106], Error: [undefined local variable or method `connection' for #<ManageIQ::Providers::Redhat::Inventory::Collector::InfraManager:0x00000002dea5b0> 
Did you mean?  concern] 
[----] E, [2017-07-25T06:23:40.378372 #13328:b89138] ERROR -- : [EmsRefresh::Refreshers::EmsRefresherMixin::PartialRefreshError]: undefined local variable or method `connection' for #<ManageIQ::Providers::Redhat::Inventory::Collector::InfraManager:0x00000002dea5b0> 
Did you mean?  concern  Method:[block in method_missing] 
[----] E, [2017-07-25T06:23:40.378673 #13328:b89138] ERROR -- : /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:50:in `refresh' 
/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh' 
/var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `block in refresh' 
/var/www/miq/vmdb/app/models/ems_refresh.rb:96:in `each' 
/var/www/miq/vmdb/app/models/ems_refresh.rb:96:in `refresh' 
/var/www/miq/vmdb/app/models/miq_queue.rb:387:in `block in deliver' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `block in catch' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch' 
/opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:106:in `timeout' 
/var/www/miq/vmdb/app/models/miq_queue.rb:386:in `deliver' 
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:108:in `deliver_queue_message' 
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:137:in `deliver_message' 
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:155:in `block in do_work'

Comment 2 Ilanit Stein 2017-07-25 13:25:29 UTC
Created attachment 1304207 [details]
evm.log

Comment 3 Ilanit Stein 2017-07-25 13:26:03 UTC
Created attachment 1304208 [details]
automation.log

Comment 4 Ilanit Stein 2017-07-25 13:26:25 UTC
Created attachment 1304209 [details]
production.log

Comment 5 Ilanit Stein 2017-07-25 13:26:49 UTC
Created attachment 1304210 [details]
rhem log

Comment 6 Ilanit Stein 2017-07-25 13:27:12 UTC
Created attachment 1304211 [details]
engine.log

Comment 8 Radim Hrazdil 2017-11-02 13:19:36 UTC
Verified that performing "Refresh Relationships and Power States" succeeds.
CFME version 5.9.0.2, RHV provider Version 4.1.3.4