Bug 1566198 - DRb 'close' error for closed connection
Summary: DRb 'close' error for closed connection
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Appliance
Version: 5.8.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.10.0
Assignee: Adam Grare
QA Contact: Dmitry Misharov
URL:
Whiteboard:
Depends On:
Blocks: 1566255 1566256
TreeView+ depends on / blocked
 
Reported: 2018-04-11 18:22 UTC by Tina Fitzgerald
Modified: 2019-02-11 14:01 UTC (History)
5 users (show)

Fixed In Version: 5.10.0.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1566255 1566256 (view as bug list)
Environment:
Last Closed: 2019-02-11 14:01:01 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:


Attachments (Terms of Use)

Description Tina Fitzgerald 2018-04-11 18:22:48 UTC
Description of problem:

The following error was discovered while investigating:
https://bugzilla.redhat.com/show_bug.cgi?id=1562401


----] I, [2018-04-03T08:05:57.972011 #105122:8ad130]  INFO -- : Q-task_id([miq_provision_20000000004950]) MIQ(MiqQueue#deliver) Message id: [20000247754617], Delivering...
[----] I, [2018-04-03T08:05:58.013981 #105122:8ad130]  INFO -- : Q-task_id([miq_provision_20000000004950]) MIQ(ManageIQ::Providers::Vmware::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Vmware::InfraManager: [SAT NDC vCenter]
[----] I, [2018-04-03T08:05:58.077885 #105122:8ad130]  INFO -- : Q-task_id([miq_provision_20000000004950]) MIQ(MiqFaultTolerantVim._connect) EMS: [SAT NDC vCenter] [Broker] Connecting with address: [tnwg01a-v1945.ad.cbp.dhs.gov], userid: [svc-edmed-cloudforms]...
[----] I, [2018-04-03T08:05:58.102451 #9566:8ad130]  INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/EmsEvent/EVM/CloneVM_Task_Complete#create  ManageIQ/System/Event/EmsEvent]
[----] E, [2018-04-03T08:05:58.110848 #105122:8ad130] ERROR -- : Q-task_id([miq_provision_20000000004950]) MIQ(MiqQueue#deliver) Message id: [20000247754617], Error: [undefined method `close' for nil:NilClass
Did you mean?  clone]
[----] E, [2018-04-03T08:05:58.111815 #105122:8ad130] ERROR -- : Q-task_id([miq_provision_20000000004950]) [NoMethodError]: undefined method `close' for nil:NilClass
Did you mean?  clone  Method:[rescue in deliver]
[----] E, [2018-04-03T08:05:58.112689 #105122:8ad130] ERROR -- : Q-task_id([miq_provision_20000000004950]) /opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1258:in `close'
/opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1237:in `block in open'
/opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:234:in `block in sync_synchronize'
/opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:231:in `handle_interrupt'
/opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:231:in `sync_synchronize'
/opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1235:in `open'
/opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1141:in `block in method_missing'
/opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1160:in `with_friend'
/opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1140:in `method_missing'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-a29aab0471c6/lib/gems/pending/VMwareWebService/MiqVimBroker.rb:419:in `getMiqVim'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-a29aab0471c6/lib/gems/pending/util/miq_fault_tolerant_vim.rb:187:in `_connect_with_broker'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-a29aab0471c6/lib/gems/pending/util/miq_fault_tolerant_vim.rb:173:in `block in _connect'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-a29aab0471c6/lib/gems/pending/util/miq_fault_tolerant_vim.rb:85:in `_execute_with_broker'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-a29aab0471c6/lib/gems/pending/util/miq_fault_tolerant_vim.rb:74:in `_execute'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-a29aab0471c6/lib/gems/pending/util/miq_fault_tolerant_vim.rb:171:in `_connect'
/opt/rh/cfme-gemset/bundler/gems/manageiq-gems-pending-a29aab0471c6/lib/gems/pending/util/miq_fault_tolerant_vim.rb:28:in `initialize'
/var/www/miq/vmdb/app/models/mixins/vim_connect_mixin.rb:23:in `new'
/var/www/miq/vmdb/app/models/mixins/vim_connect_mixin.rb:23:in `connect'
/var/www/miq/vmdb/app/models/mixins/vim_connect_mixin.rb:36:in `with_provider_connection'
/var/www/miq/vmdb/app/models/mixins/provider_object_mixin.rb:5:in `with_provider_connection'
/opt/rh/cfme-gemset/bundler/gems/manageiq-providers-vmware-d69f678e3cb3/app/models/manageiq/providers/vmware/infra_manager/provision/cloning.rb:3:in `do_clone_task_check'
/opt/rh/cfme-gemset/bundler/gems/manageiq-providers-vmware-d69f678e3cb3/app/models/manageiq/providers/vmware/infra_manager/provision/state_machine.rb:28:in `poll_clone_complete'
/var/www/miq/vmdb/app/models/miq_queue.rb:347:in `block in deliver'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:91:in `block in timeout'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `block in catch'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch'

Comment 2 Tina Fitzgerald 2018-04-11 18:24:07 UTC
Adam created a PR to resolve this issue:
https://github.com/ManageIQ/manageiq/pull/17267

Comment 3 Tina Fitzgerald 2018-04-11 18:24:45 UTC
New commit detected on ManageIQ/manageiq/master:

https://github.com/ManageIQ/manageiq/commit/76309b91e41725be72c5abadb19c0731b07733f7
commit 76309b91e41725be72c5abadb19c0731b07733f7
Author:     Adam Grare <agrare@redhat.com>
AuthorDate: Mon Apr  9 14:37:53 2018 -0400
Commit:     Adam Grare <agrare@redhat.com>
CommitDate: Mon Apr  9 14:37:53 2018 -0400

    Set DRb conn pool to [] after closing connections

    After forking and closing open DRb pooled connections set the pool to []
    to prevent other calls calling `conn.close` on already closed
    connection.  This prevents the following:

    ```
    [NoMethodError]: undefined method `close' for nil:NilClass
    Did you mean?  clone
    /opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1258:in `close'
    /opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1237:in `block in open'
    /opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:234:in `block in sync_synchronize'
    /opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:231:in `handle_interrupt'
    /opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:231:in `sync_synchronize'
    /opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1235:in `open'
    /opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1141:in `block in method_missing'
    /opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1160:in `with_friend'
    /opt/rh/rh-ruby23/root/usr/share/ruby/drb/drb.rb:1140:in `method_missing'
    lib/gems/pending/VMwareWebService/MiqVimBroker.rb:419:in `getMiqVim'
    ```

    https://bugzilla.redhat.com/show_bug.cgi?id=1562401

 app/models/miq_worker.rb | 1 +
 1 file changed, 1 insertion(+)

Comment 7 Dmitry Misharov 2018-06-26 14:00:00 UTC
Verified in 5.10.0.1.20180619163011_900fdc4.


Note You need to log in before you can comment on or make changes to this bug.