Description of problem: Intermittent refresh failure of Openstack::NetworkManager provider with [NoMethodError]: undefined method `[]=' ERROR -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::Refresher#refresh) EMS: [OpenStack OverCloud POD1 Network Manager], id: [1000000000006] Refresh failed ERROR -- : [NoMethodError]: undefined method `[]=' for nil:NilClass Method:[rescue in block in refresh] ERROR -- : /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:208:in `find_device_object' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:224:in `parse_network_port' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:110:in `block in get_network_ports' Version-Release number of selected component (if applicable): 5.8.1.5 How reproducible: Intermittent but seems to be related to the network port discovery. Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: [----] E, [2017-09-05T03:49:36.112260 #37347:4d1138] ERROR -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::Refresher#refresh) EMS: [OpenStack OverCloud POD1 Network Manager], id: [1000000000006] Refresh failed [----] E, [2017-09-05T03:49:36.114134 #37347:4d1138] ERROR -- : [NoMethodError]: undefined method `[]=' for nil:NilClass Method:[rescue in block in refresh] [----] E, [2017-09-05T03:49:36.114355 #37347:4d1138] ERROR -- : /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:208:in `find_device_object' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:224:in `parse_network_port' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:110:in `block in get_network_ports' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/refresh_parser_common/helper_methods.rb:15:in `process_collection_item' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/refresh_parser_common/helper_methods.rb:9:in `block (2 levels) in process_collection' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/refresh_parser_common/helper_methods.rb:24:in `safe_call' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/refresh_parser_common/helper_methods.rb:9:in `block in process_collection' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/refresh_parser_common/helper_methods.rb:9:in `each' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/refresh_parser_common/helper_methods.rb:9:in `process_collection' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:110:in `get_network_ports' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:40:in `ems_inv_to_hashes' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresh_parser.rb:8:in `ems_inv_to_hashes' /var/www/miq/vmdb/app/models/manageiq/providers/openstack/network_manager/refresher.rb:6:in `parse_legacy_inventory' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:146:in `block in parse_targeted_inventory' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:146:in `parse_targeted_inventory' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:87:in `block in refresh_targets_for_ems' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:86:in `refresh_targets_for_ems' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block (2 levels) in refresh' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block in refresh' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `each' /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `refresh' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:98:in `block in refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `each' /var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `refresh' /var/www/miq/vmdb/app/models/miq_queue.rb:347:in `block in deliver' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:91:in `block in timeout' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `block in catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:106:in `timeout' /var/www/miq/vmdb/app/models/miq_queue.rb:343:in `deliver' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:107:in `deliver_queue_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:135:in `deliver_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:153:in `block in do_work' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:147:in `loop' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:147:in `do_work' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:340:in `block in do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:337:in `loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:337:in `do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:160:in `run' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:134:in `start' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:21:in `start_worker' /var/www/miq/vmdb/app/models/miq_worker.rb:339:in `block in start_runner' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /var/www/miq/vmdb/app/models/miq_worker.rb:337:in `start_runner' /var/www/miq/vmdb/app/models/miq_worker.rb:348:in `start' /var/www/miq/vmdb/app/models/miq_worker.rb:266:in `start_worker' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:68:in `start_worker_for_ems' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:46:in `block in sync_workers' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:45:in `each' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:45:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:53:in `block in sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers' /var/www/miq/vmdb/app/models/miq_server.rb:349:in `block in monitor' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/miq_server.rb:349:in `monitor' /var/www/miq/vmdb/app/models/miq_server.rb:371:in `block (2 levels) in monitor_loop' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/miq_server.rb:371:in `block in monitor_loop' /var/www/miq/vmdb/app/models/miq_server.rb:370:in `loop' /var/www/miq/vmdb/app/models/miq_server.rb:370:in `monitor_loop' /var/www/miq/vmdb/app/models/miq_server.rb:253:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:65:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:91:in `start' /var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>' [----] E, [2017-09-05T03:49:36.114463 #37347:4d1138] ERROR -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::Refresher#refresh) EMS: [OpenStack OverCloud POD1 Network Manager], id: [1000000000006] Unable to perform refresh for the following targets: [----] E, [2017-09-05T03:49:36.114598 #37347:4d1138] ERROR -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::Refresher#refresh) --- ManageIQ::Providers::Openstack::NetworkManager [OpenStack OverCloud POD1 Network Manager] id [1000000000006] [----] I, [2017-09-05T03:49:36.150071 #37347:4d1138] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::Refresher#refresh) Refreshing all targets...Complete [----] E, [2017-09-05T03:49:36.150337 #37347:4d1138] ERROR -- : MIQ(MiqQueue#deliver) Message id: [1000001373770], Error: [undefined method `[]=' for nil:NilClass] [----] E, [2017-09-05T03:49:36.150483 #37347:4d1138] ERROR -- : [EmsRefresh::Refreshers::EmsRefresherMixin::PartialRefreshError]: undefined method `[]=' for nil:NilClass Method:[rescue in deliver] [----] E, [2017-09-05T03:49:36.150611 #37347:4d1138] ERROR -- : /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:50:in `refresh' /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:98:in `block in refresh' /var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `each' /var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `refresh' /var/www/miq/vmdb/app/models/miq_queue.rb:347:in `block in deliver' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:91:in `block in timeout' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `block in catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch' /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:106:in `timeout' /var/www/miq/vmdb/app/models/miq_queue.rb:343:in `deliver' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:107:in `deliver_queue_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:135:in `deliver_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:153:in `block in do_work' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:147:in `loop' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:147:in `do_work' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:340:in `block in do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:337:in `loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:337:in `do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:160:in `run' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:134:in `start' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:21:in `start_worker' /var/www/miq/vmdb/app/models/miq_worker.rb:339:in `block in start_runner' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /var/www/miq/vmdb/app/models/miq_worker.rb:337:in `start_runner' /var/www/miq/vmdb/app/models/miq_worker.rb:348:in `start' /var/www/miq/vmdb/app/models/miq_worker.rb:266:in `start_worker' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:68:in `start_worker_for_ems' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:46:in `block in sync_workers' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:45:in `each' /var/www/miq/vmdb/app/models/mixins/per_ems_worker_mixin.rb:45:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:53:in `block in sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers' /var/www/miq/vmdb/app/models/miq_server.rb:349:in `block in monitor' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/miq_server.rb:349:in `monitor' /var/www/miq/vmdb/app/models/miq_server.rb:371:in `block (2 levels) in monitor_loop' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store' /opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-8f7d63f441d5/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block' /var/www/miq/vmdb/app/models/miq_server.rb:371:in `block in monitor_loop' /var/www/miq/vmdb/app/models/miq_server.rb:370:in `loop' /var/www/miq/vmdb/app/models/miq_server.rb:370:in `monitor_loop' /var/www/miq/vmdb/app/models/miq_server.rb:253:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:65:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:91:in `start' /var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>'
This appears to already be addressed in the new refresh system, this PR should fix it in the old one: https://github.com/ManageIQ/manageiq-providers-openstack/pull/90
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:0380