Bug 784140

Summary: exception in dbomatic log, unable to stop deployments, instances
Product: [Retired] CloudForms Cloud Engine Reporter: wes hayutin <whayutin>
Component: aeolus-conductorAssignee: Jason Guiditta <jguiditt>
Status: CLOSED DUPLICATE QA Contact: wes hayutin <whayutin>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 1.0.0CC: akarol, deltacloud-maint, morazi, ssachdev
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
URL: https://qeblade30.rhq.lab.eng.bos.redhat.com/conductor/pools?details_tab=instances&only_tab=true&view=filter
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-01-24 17:22:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 744194    

Description wes hayutin 2012-01-23 22:58:36 UTC 
Description of problem:

12-01-23 17:49:26 DEBUG Deltacloud instances refresh started
2012-01-23 17:49:27 DEBUG updating instance state for Deployment-ConfigSrv-x7qxn/fedoraEC2Instance: i-35846850
2012-01-23 17:49:28 DEBUG updating instance state for Deployment-ConfigSrv-hmt0h/RHELEC2Instance: i-51e90534
2012-01-23 17:49:28 DEBUG Deltacloud instances refresh completed
2012-01-23 17:50:28 DEBUG Deltacloud instances refresh started
2012-01-23 17:50:29 DEBUG updating instance state for Deployment-ConfigSrv-x7qxn/fedoraEC2Instance: i-35846850
2012-01-23 17:50:30 ERROR /usr/lib/ruby/gems/1.8/gems/activemodel-3.0.10/lib/active_model/attribute_methods.rb:392:in `method_missing': undefined method `tasks' for #<Instance:0x7f6a8b753940>
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/attribute_methods.rb:46:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/share/aeolus-conductor/app/models/instance.rb:493:in `destroy_on_provider'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:440:in `_run_destroy_callbacks'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:256:in `destroy'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations.rb:1614:in `has_many_dependent_destroy_for_instances'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `each'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `send'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations.rb:1606:in `has_many_dependent_destroy_for_instances'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:415:in `_run_destroy_callbacks'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:256:in `destroy'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `send'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/share/aeolus-conductor/app/models/instance_observer.rb:127:in `after_save'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/observer.rb:118:in `send'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/observer.rb:118:in `_notify_instance_observer_for_after_save'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:464:in `_run_save_callbacks'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:273:in `create_or_update'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/persistence.rb:60:in `save!'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/validations.rb:49:in `save!'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/attribute_methods/dirty.rb:30:in `save!'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:245:in `save!'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:245:in `save!'
2012-01-23 17:50:30 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:197:in `check_one_account'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `each'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `send'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:50:30 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:179:in `check_one_account'
2012-01-23 17:50:30 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:242:in `refresh_instances'
2012-01-23 17:50:30 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:237:in `each'
2012-01-23 17:50:30 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:237:in `refresh_instances'
2012-01-23 17:50:30 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:287
2012-01-23 17:50:30 DEBUG Deltacloud instances refresh completed
2012-01-23 17:50:59 WARN Connection timeout of 30 seconds exceeded, backend clouds could not be contacted.  Will try again in 60 seconds
2012-01-23 17:51:00 DEBUG Deltacloud instances refresh completed
2012-01-23 17:52:01 DEBUG Deltacloud instances refresh started
2012-01-23 17:52:01 DEBUG updating instance state for Deployment-ConfigSrv-x7qxn/fedoraEC2Instance: i-35846850
2012-01-23 17:52:02 ERROR /usr/lib/ruby/gems/1.8/gems/activemodel-3.0.10/lib/active_model/attribute_methods.rb:392:in `method_missing': undefined method `tasks' for #<Instance:0x7f6a8b739f90>
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/attribute_methods.rb:46:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/share/aeolus-conductor/app/models/instance.rb:493:in `destroy_on_provider'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:440:in `_run_destroy_callbacks'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:256:in `destroy'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations.rb:1614:in `has_many_dependent_destroy_for_instances'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `each'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `send'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations.rb:1606:in `has_many_dependent_destroy_for_instances'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:415:in `_run_destroy_callbacks'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:256:in `destroy'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `send'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/share/aeolus-conductor/app/models/instance_observer.rb:127:in `after_save'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/observer.rb:118:in `send'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/observer.rb:118:in `_notify_instance_observer_for_after_save'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:464:in `_run_save_callbacks'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:273:in `create_or_update'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/persistence.rb:60:in `save!'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/validations.rb:49:in `save!'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/attribute_methods/dirty.rb:30:in `save!'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:245:in `save!'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:245:in `save!'
2012-01-23 17:52:02 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:197:in `check_one_account'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `each'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `send'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:52:02 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:179:in `check_one_account'
2012-01-23 17:52:02 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:242:in `refresh_instances'
2012-01-23 17:52:02 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:237:in `each'
2012-01-23 17:52:02 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:237:in `refresh_instances'
2012-01-23 17:52:02 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:287
2012-01-23 17:52:02 DEBUG Deltacloud instances refresh completed
2012-01-23 17:52:32 WARN Connection timeout of 30 seconds exceeded, backend clouds could not be contacted.  Will try again in 60 seconds
2012-01-23 17:52:33 DEBUG Deltacloud instances refresh completed
2012-01-23 17:53:33 DEBUG Deltacloud instances refresh started
2012-01-23 17:53:33 DEBUG updating instance state for Deployment-ConfigSrv-x7qxn/fedoraEC2Instance: i-35846850
2012-01-23 17:53:34 ERROR /usr/lib/ruby/gems/1.8/gems/activemodel-3.0.10/lib/active_model/attribute_methods.rb:392:in `method_missing': undefined method `tasks' for #<Instance:0x7f6a8b724438>
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/attribute_methods.rb:46:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/share/aeolus-conductor/app/models/instance.rb:493:in `destroy_on_provider'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:440:in `_run_destroy_callbacks'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:256:in `destroy'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations.rb:1614:in `has_many_dependent_destroy_for_instances'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `each'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `send'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations.rb:1606:in `has_many_dependent_destroy_for_instances'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:415:in `_run_destroy_callbacks'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:256:in `destroy'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:235:in `destroy'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `send'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/share/aeolus-conductor/app/models/instance_observer.rb:127:in `after_save'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/observer.rb:118:in `send'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/observer.rb:118:in `_notify_instance_observer_for_after_save'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:464:in `_run_save_callbacks'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/callbacks.rb:273:in `create_or_update'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/persistence.rb:60:in `save!'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/validations.rb:49:in `save!'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/attribute_methods/dirty.rb:30:in `save!'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:245:in `save!'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:292:in `with_transaction_returning_status'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:139:in `transaction'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:207:in `transaction'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/transactions.rb:245:in `save!'
2012-01-23 17:53:34 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:197:in `check_one_account'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `each'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `send'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:220:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_collection.rb:431:in `method_missing'
2012-01-23 17:53:34 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:179:in `check_one_account'
2012-01-23 17:53:34 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:242:in `refresh_instances'
2012-01-23 17:53:34 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:237:in `each'
2012-01-23 17:53:34 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:237:in `refresh_instances'
2012-01-23 17:53:34 ERROR 	from /usr/share/aeolus-conductor/dbomatic/dbomatic:287
2012-01-23 17:53:34 DEBUG Deltacloud instances refresh completed
2012-01-23 17:54:04 WARN Connection timeout of 30 seconds exceeded, backend clouds could not be contacted.  Will try again in 60 seconds
2012-01-23 17:54:05 DEBUG Deltacloud instances refresh comp



[root@qeblade30 ~]# rpm -qa | grep aeolus
aeolus-conductor-0.8.0-9.el6.noarch
aeolus-all-0.8.0-9.el6.noarch
rubygem-aeolus-cli-0.3.0-5.el6.noarch
rubygem-aeolus-image-0.3.0-3.el6.noarch
aeolus-configure-2.5.0-6.el6.noarch
aeolus-conductor-doc-0.8.0-9.el6.noarch
aeolus-conductor-daemons-0.8.0-9.el6.noarch
Comment 1 Jason Guiditta 2012-01-24 16:58:31 UTC 
it seems this is likely corrected in patch:

commit 7a1a02058a5d4d135514abe01a3299014d7bb459
Author: Jan Provaznik <jprovazn>
Date:   Fri Jan 20 18:24:00 2012 +0100

    BZ 783079 - fixed instance's destroy_on_provider
    
    - added tasks association referenced in this method
    - added check if a provider_account is nil
    - fixed logical operator mistak - don't retry if number of retries is >500


However, this patch is not in the referenced release, it should be in the next one
Comment 2 Jason Guiditta 2012-01-24 17:22:34 UTC 

*** This bug has been marked as a duplicate of bug 783079 ***