Bug 832175 - deadlock in rails related to connecting to rhevm provider after upgrade
Summary: deadlock in rails related to connecting to rhevm provider after upgrade
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: CloudForms Cloud Engine
Classification: Retired
Component: aeolus-conductor
Version: 1.0.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: rc
Assignee: Angus Thomas
QA Contact: Rehana
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-06-14 18:20 UTC by Dave Johnson
Modified: 2012-10-17 20:06 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-08-08 18:53:08 UTC
Embargoed:


Attachments (Terms of Use)

Description Dave Johnson 2012-06-14 18:20:44 UTC
Description of problem:
=========================
Ran into this issue today but when I tried to reproduce, I could not...  This bz is at least documenting it possibility.

basically upgraded a configured CFCE v1.0 system and when I logged in to add a rhevm account to the rhevm provider it wasn;t working, looking through the logs I noticed a deadlock stacetrace which appeared each time I tests the provider connection.


Version-Release number of selected component (if applicable):
aeolus-conductor-0.8.27-1.el6_3.noarch


How reproducible:  so far it is not reproducible (trying to provision a third system to try on)


Steps to Reproduce:

1. I installed CFCE v1.0 and configured for all providers
2. Ran yum update aeolus-all aeolus-configure deltacloud-core
3. Ran upgrade script
4. Logged into web UI
5. Tried to add rhevm provider account and noticed error message
The provider rhevm-default is currently not available.
6. Test the connection on the provider page
7. Looked in /var/log/aeolus-conductor/rails.log and noticed deadlock stack trace
  
Actual results:

  ProviderAccount Load (0.1ms)  SELECT "provider_accounts".* FROM "provider_accounts" WHERE ("provider_accounts".provider_id = 8)
  ProviderType Load (0.1ms)  SELECT "provider_types".* FROM "provider_types" WHERE "provider_types"."id" = 3 LIMIT 1
Error connecting to framework: 500 : Thread#join: deadlock 0x7f181417eef8 - mutual join(0x7f181417f088)
Backtrace: 
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
/usr/share/deltacloud-core/lib/deltacloud/drivers.rb:68:in `driver'
/usr/share/deltacloud-core/lib/deltacloud/server.rb:112:in `GET /api/?'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:1151:in `call'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:1151:in `compile!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:724:in `instance_eval'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:724:in `route_eval'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:708:in `route!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:758:in `process_route'
  SQL (0.4ms)  SELECT COUNT(*) FROM "provider_accounts" WHERE ("provider_accounts".provider_id = 8)
  SQL (0.1ms)  SELECT COUNT(*) FROM "realms" WHERE ("realms".provider_id = 8)


Expected results:
provider account created


Additional info:

Perhaps this was related to how I upgraded?  Instead of doing 'yum upgrade' I performed a 'yum upgrade aeolus-all deltacloud-core aeolus-configure'???

When I tried to reproduce on a second system, I did do a full 'yum update', not sure if that would make a difference.

Comment 1 James Laska 2012-10-17 20:06:00 UTC
I just hit this problem also on a CloudForms-1.1 beta5 instance.  I observed this in the logs while attempting to determine why conductor reported that my correctly configured RHEVM provider was unavailable.

>   SQL (0.2ms)  SELECT 1 FROM "providers" WHERE ("providers"."name" = 'rhevm-default') AND ("providers".id <> 9) LIMIT 1
>   ProviderType Load (0.2ms)  SELECT "provider_types".* FROM "provider_types" WHERE "provider_types"."id" = 3 LIMIT 1
> Error connecting to framework: 500 : Thread#join: deadlock 0x7f1f93c51ed0 - mutual join(0x7f1f93c52128)
> Backtrace:
> /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
> /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
> /usr/share/deltacloud-core/lib/deltacloud/drivers.rb:68:in `driver'
> /usr/share/deltacloud-core/lib/deltacloud/server.rb:112:in `GET /api/?'
> /usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:1151:in `call'
> /usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:1151:in `compile!'
> /usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:724:in `instance_eval'
> /usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:724:in `route_eval'
> /usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:708:in `route!'
> /usr/lib/ruby/gems/1.8/gems/sinatra-1.2.6/lib/sinatra/base.rb:758:in `process_route'
>   SQL (0.1ms)  ROLLBACK


Note You need to log in before you can comment on or make changes to this bug.