Bug 1412847 - Swift Manager refresh fails with "undefined method `each' for nil:NilClass"
Summary: Swift Manager refresh fails with "undefined method `each' for nil:NilClass"
Keywords:
Status: CLOSED DUPLICATE of bug 1441144
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: GA
: cfme-future
Assignee: Jerry Keselman
QA Contact: Ola Pavlenko
URL:
Whiteboard: openstack:storage:swift
: 1439285 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-13 00:15 UTC by Richard Su
Modified: 2020-06-11 13:11 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-04-17 19:28:29 UTC
Category: ---
Cloudforms Team: Openstack
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3017181 None None None 2017-04-28 08:50:29 UTC

Description Richard Su 2017-01-13 00:15:53 UTC
Description of problem:
The Swift Manager refresh fails. Unable to collect swift containers and objects.

Version-Release number of selected component (if applicable):
master

How reproducible:
always

Steps to Reproduce:
1. Add a OpenStack cloud provider.
2. Navigate to Storage Managers > Swift Manager
3. Error is displayed in Last Refresh: "undefined method `each' for nil:NilClass"

Actual results:
Refresh fails

Expected results:
Refresh should succeed.

Additional info:

From log files

[----] I, [2017-01-12T16:02:45.884075 #1371:3fd98c]  INFO -- : MIQ(ManageIQ::Providers::StorageManager::SwiftManager::RefreshParser.ems_inv_to_hashes) Collecting data for EMS name: [overcloud Swift Manager] id: [2]...

==> evm.log <==
[----] E, [2017-01-12T16:02:45.886951 #1371:3fd98c] ERROR -- : <Fog> excon.error     #<Excon::Error::NotFound: Expected([200, 204]) <=> Actual(404 Not Found)
excon.error.response
  :body          => "<html>\n<head>\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=ISO-8859-1\"/>\n<title>Error 404 Not Found</title>\n</head>\n<body><h2>HTTP ERROR 404</h2>\n<p>Problem accessing /v1/AUTH_90533982c5eb4c17aa409e1214a98bd7/. Reason:\n<pre>    Not Found</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n\n</body>\n</html>\n"
  :cookies       => [
  ]
  :headers       => {
    "Cache-Control"          => "must-revalidate,no-cache,no-store"
    "Content-Length"         => "1406"
    "Content-Type"           => "text/html;charset=ISO-8859-1"
    "Server"                 => "Jetty(8.1.17.v20150415)"
    "X-Content-Type-Options" => "nosniff"
  }
  :host          => "192.0.2.10"
  :local_address => "10.0.1.5"
  :local_port    => 60306
  :path          => "/v1/AUTH_90533982c5eb4c17aa409e1214a98bd7/"
  :port          => 8080
  :reason_phrase => "Not Found"
  :remote_ip     => "192.0.2.10"
  :status        => 404
  :status_line   => "HTTP/1.1 404 Not Found\r\n"
>


==> fog.log <==
[----] E, [2017-01-12T16:02:45.887050 #1371:3fd98c] ERROR -- : excon.error     #<Excon::Error::NotFound: Expected([200, 204]) <=> Actual(404 Not Found)
excon.error.response
  :body          => "<html>\n<head>\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=ISO-8859-1\"/>\n<title>Error 404 Not Found</title>\n</head>\n<body><h2>HTTP ERROR 404</h2>\n<p>Problem accessing /v1/AUTH_90533982c5eb4c17aa409e1214a98bd7/. Reason:\n<pre>    Not Found</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n\n</body>\n</html>\n"
  :cookies       => [
  ]
  :headers       => {
    "Cache-Control"          => "must-revalidate,no-cache,no-store"
    "Content-Length"         => "1406"
    "Content-Type"           => "text/html;charset=ISO-8859-1"
    "Server"                 => "Jetty(8.1.17.v20150415)"
    "X-Content-Type-Options" => "nosniff"
  }
  :host          => "192.0.2.10"
  :local_address => "10.0.1.5"
  :local_port    => 60306
  :path          => "/v1/AUTH_90533982c5eb4c17aa409e1214a98bd7/"
  :port          => 8080
  :reason_phrase => "Not Found"
  :remote_ip     => "192.0.2.10"
  :status        => 404
  :status_line   => "HTTP/1.1 404 Not Found\r\n"
>

[----] W, [2017-01-12T16:02:45.887302 #1371:3fd98c]  WARN -- : MIQ(OpenstackHandle::Handle.accessor_for_accessible_tenants) HTTP 404 Error during OpenStack request. Skipping inventory item Storage #<Proc:0x0000000b90b928@/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/openstack/openstack_handle/multi_tenancy/loop.rb:7 (lambda)>
Expected([200, 204]) <=> Actual(404 Not Found)
excon.error.response
  :body          => "<html>\n<head>\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=ISO-8859-1\"/>\n<title>Error 404 Not Found</title>\n</head>\n<body><h2>HTTP ERROR 404</h2>\n<p>Problem accessing /v1/AUTH_90533982c5eb4c17aa409e1214a98bd7/. Reason:\n<pre>    Not Found</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n<br/>                                                \n\n</body>\n</html>\n"
  :cookies       => [
  ]
  :headers       => {
    "Cache-Control"          => "must-revalidate,no-cache,no-store"
    "Content-Length"         => "1406"
    "Content-Type"           => "text/html;charset=ISO-8859-1"
    "Server"                 => "Jetty(8.1.17.v20150415)"
    "X-Content-Type-Options" => "nosniff"
  }
  :host          => "192.0.2.10"
  :local_address => "10.0.1.5"
  :local_port    => 60306
  :path          => "/v1/AUTH_90533982c5eb4c17aa409e1214a98bd7/"
  :port          => 8080
  :reason_phrase => "Not Found"
  :remote_ip     => "192.0.2.10"
  :status        => 404
  :status_line   => "HTTP/1.1 404 Not Found\r\n"

[----] I, [2017-01-12T16:02:45.887358 #1371:3fd98c]  INFO -- : MIQ(ManageIQ::Providers::StorageManager::SwiftManager::RefreshParser.ems_inv_to_hashes) Collecting data for EMS name: [overcloud Swift Manager] id: [2]...Complete

==> development.log <==
[----] D, [2017-01-12T16:02:45.888989 #1371:3fd98c] DEBUG -- :   ManageIQ::Providers::StorageManager::SwiftManager Load (0.3ms)  SELECT  "ext_management_systems".* FROM "ext_management_systems" WHERE "ext_management_systems"."type" IN ('ManageIQ::Providers::StorageManager::SwiftManager') AND "ext_management_systems"."parent_ems_id" = $1 LIMIT $2  [["parent_ems_id", 3], ["LIMIT", 1]]
[----] D, [2017-01-12T16:02:45.889435 #1371:3fd98c] DEBUG -- :   ManageIQ::Providers::StorageManager::SwiftManager Inst Including Associations (0.2ms - 1rows)

==> evm.log <==
[----] E, [2017-01-12T16:02:45.890888 #1371:3fd98c] ERROR -- : MIQ(ManageIQ::Providers::StorageManager::SwiftManager::Refresher#refresh) EMS: [overcloud Swift Manager], id: [2] Refresh failed
[----] E, [2017-01-12T16:02:45.891033 #1371:3fd98c] ERROR -- : [NoMethodError]: undefined method `each' for nil:NilClass  Method:[rescue in block in refresh]
[----] E, [2017-01-12T16:02:45.891174 #1371:3fd98c] ERROR -- : /home/rwsu/github/temp/manageiq/app/models/manageiq/providers/storage_manager/swift_manager/refresh_parser/cross_linkers/openstack.rb:14:in `cross_link'
/home/rwsu/github/temp/manageiq/app/models/manageiq/providers/storage_manager/swift_manager/refresh_parser/cross_linkers.rb:13:in `cross_link'
/home/rwsu/github/temp/manageiq/app/models/manageiq/providers/storage_manager/swift_manager/refresh_parser.rb:30:in `ems_inv_to_hashes'
/home/rwsu/github/temp/manageiq/app/models/manageiq/providers/storage_manager/swift_manager/refresh_parser.rb:9:in `ems_inv_to_hashes'
/home/rwsu/github/temp/manageiq/app/models/manageiq/providers/storage_manager/swift_manager/refresher.rb:6:in `parse_legacy_inventory'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:122:in `block in parse_targeted_inventory'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:122:in `parse_targeted_inventory'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:87:in `block in refresh_targets_for_ems'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:86:in `refresh_targets_for_ems'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block (2 levels) in refresh'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block in refresh'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `each'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `refresh'
/home/rwsu/github/temp/manageiq/app/models/manageiq/providers/base_manager/refresher.rb:10:in `refresh'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh.rb:92:in `block in refresh'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh.rb:91:in `each'
/home/rwsu/github/temp/manageiq/app/models/ems_refresh.rb:91:in `refresh'
/home/rwsu/github/temp/manageiq/app/models/miq_queue.rb:347:in `block in deliver'
/home/rwsu/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/timeout.rb:89:in `block in timeout'
/home/rwsu/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/timeout.rb:34:in `block in catch'
/home/rwsu/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/timeout.rb:34:in `catch'
/home/rwsu/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/timeout.rb:34:in `catch'
/home/rwsu/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/timeout.rb:104:in `timeout'
/home/rwsu/github/temp/manageiq/app/models/miq_queue.rb:343:in `deliver'
/home/rwsu/github/temp/manageiq/app/models/miq_queue_worker_base/runner.rb:106:in `deliver_queue_message'
/home/rwsu/github/temp/manageiq/app/models/miq_queue_worker_base/runner.rb:134:in `deliver_message'
/home/rwsu/github/temp/manageiq/app/models/miq_queue_worker_base/runner.rb:152:in `block in do_work'
/home/rwsu/github/temp/manageiq/app/models/miq_queue_worker_base/runner.rb:146:in `loop'
/home/rwsu/github/temp/manageiq/app/models/miq_queue_worker_base/runner.rb:146:in `do_work'
/home/rwsu/github/temp/manageiq/app/models/miq_worker/runner.rb:334:in `block in do_work_loop'
/home/rwsu/github/temp/manageiq/app/models/miq_worker/runner.rb:331:in `loop'
/home/rwsu/github/temp/manageiq/app/models/miq_worker/runner.rb:331:in `do_work_loop'
/home/rwsu/github/temp/manageiq/app/models/miq_worker/runner.rb:153:in `run'
/home/rwsu/github/temp/manageiq/app/models/miq_worker/runner.rb:128:in `start'
/home/rwsu/github/temp/manageiq/app/models/miq_worker/runner.rb:21:in `start_worker'
/home/rwsu/github/temp/manageiq/app/models/miq_worker.rb:343:in `block in start'
/home/rwsu/.rvm/gems/ruby-2.2.2/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork'
/home/rwsu/.rvm/gems/ruby-2.2.2/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork'
/home/rwsu/github/temp/manageiq/app/models/miq_worker.rb:341:in `start'
/home/rwsu/github/temp/manageiq/app/models/miq_worker.rb:270:in `start_worker'
/home/rwsu/github/temp/manageiq/app/models/mixins/per_ems_worker_mixin.rb:68:in `start_worker_for_ems'
/home/rwsu/github/temp/manageiq/app/models/mixins/per_ems_worker_mixin.rb:46:in `block in sync_workers'
/home/rwsu/github/temp/manageiq/app/models/mixins/per_ems_worker_mixin.rb:45:in `each'
/home/rwsu/github/temp/manageiq/app/models/mixins/per_ems_worker_mixin.rb:45:in `sync_workers'
/home/rwsu/github/temp/manageiq/app/models/miq_server/worker_management/monitor.rb:52:in `block in sync_workers'
/home/rwsu/github/temp/manageiq/app/models/miq_server/worker_management/monitor.rb:50:in `each'
/home/rwsu/github/temp/manageiq/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers'
/home/rwsu/github/temp/manageiq/app/models/miq_server/worker_management/monitor.rb:22:in `monitor_workers'
/home/rwsu/github/temp/manageiq/app/models/miq_server.rb:346:in `block in monitor'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/home/rwsu/github/temp/manageiq/app/models/miq_server.rb:346:in `monitor'
/home/rwsu/github/temp/manageiq/app/models/miq_server.rb:368:in `block (2 levels) in monitor_loop'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/home/rwsu/.rvm/gems/ruby-2.2.2/bundler/gems/manageiq-gems-pending-732f36463f96/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/home/rwsu/github/temp/manageiq/app/models/miq_server.rb:368:in `block in monitor_loop'
/home/rwsu/github/temp/manageiq/app/models/miq_server.rb:367:in `loop'
/home/rwsu/github/temp/manageiq/app/models/miq_server.rb:367:in `monitor_loop'
/home/rwsu/github/temp/manageiq/app/models/miq_server.rb:250:in `start'
/home/rwsu/github/temp/manageiq/lib/workers/evm_server.rb:65:in `start'
/home/rwsu/github/temp/manageiq/lib/workers/evm_server.rb:92:in `start'
/home/rwsu/github/temp/manageiq/lib/workers/bin/evm_server.rb:4:in `<main>'
[----] E, [2017-01-12T16:02:45.891231 #1371:3fd98c] ERROR -- : MIQ(ManageIQ::Providers::StorageManager::SwiftManager::Refresher#refresh) EMS: [overcloud Swift Manager], id: [2] Unable to perform refresh for the following targets:
[----] E, [2017-01-12T16:02:45.891296 #1371:3fd98c] ERROR -- : MIQ(ManageIQ::Providers::StorageManager::SwiftManager::Refresher#refresh)  --- ManageIQ::Providers::StorageManager::SwiftManager [overcloud Swift Manager] id [2]

Comment 4 myoder 2017-01-26 15:56:22 UTC
Customer is on OSP 10 (14.0.2).

Comment 6 Jerry Keselman 2017-03-06 20:32:46 UTC
Richard, can you provider connectivity and credentials for the appliance in question? Thanks.

Comment 7 Richard Su 2017-03-14 16:56:26 UTC
Jerry, I don't have a public server to share that exhibits the problem. Are you having trouble reproducing?

Comment 8 Jerry Keselman 2017-03-14 17:04:15 UTC
Correct.

Comment 9 Tzu-Mainn Chen 2017-04-06 15:26:18 UTC
*** Bug 1439285 has been marked as a duplicate of this bug. ***

Comment 10 Tzu-Mainn Chen 2017-04-06 15:48:10 UTC
It looks like the error happens here:

https://github.com/ManageIQ/manageiq/blob/master/app/models/manageiq/providers/storage_manager/swift_manager/refresh_parser/cross_linkers/openstack.rb#L14

Given that, is it possible this error happens when swift exists, but no containers exist within swift (or no containers that the admin user can see)?

Comment 11 Saif Ali 2017-04-07 19:55:00 UTC
Tzu-Mainn, 
I verified this by creating container, and then uploaded file to that container and I'm no longer seen that error.

Comment 12 Jerry Keselman 2017-04-17 19:28:29 UTC
This is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1441144 and will be closed.

*** This bug has been marked as a duplicate of bug 1441144 ***


Note You need to log in before you can comment on or make changes to this bug.