Bug 785166 - aeolus components (factory, dc-core) crash during multiple build
Summary: aeolus components (factory, dc-core) crash during multiple build
Keywords:
Status: CLOSED DUPLICATE of bug 782573
Alias: None
Product: CloudForms Cloud Engine
Classification: Retired
Component: Distribution
Version: 1.0.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
Assignee: Hugh Brock
QA Contact: wes hayutin
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-01-27 13:59 UTC by wes hayutin
Modified: 2012-01-27 17:40 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-01-27 14:00:34 UTC
Embargoed:

Attachments (Terms of Use)

Description wes hayutin 2012-01-27 13:59:32 UTC 
Description of problem:

recreate:
 aeolus image build --target ec2,rhevm,vsphere --template RHEL62.tpl 




[root@qeblade31 ~]# aeolus image build --target ec2,rhevm,vsphere --template RHEL62.tpl 
Image                                    Build                                    Target Image                             Target      Status       
------------------------------------     ------------------------------------     ------------------------------------     -------     --------     
b372ee67-dda3-4385-afa9-d20fe5a0c342     a79fb278-0613-4354-b20a-d0ec0ca50217     39c020ae-43d1-4308-b5c7-542b69eee614     ec2         BUILDING     
b372ee67-dda3-4385-afa9-d20fe5a0c342     a79fb278-0613-4354-b20a-d0ec0ca50217     8ba30bfd-bcb9-47b4-9c16-bfe1cfb08d0b     rhevm       BUILDING     
b372ee67-dda3-4385-afa9-d20fe5a0c342     a79fb278-0613-4354-b20a-d0ec0ca50217     bed2541a-1ebc-4575-a6e2-d5abcc33e85d     vsphere     BUILDING     

factory log just stops...
[root@qeblade31 ~]# tail /var/log/imagefactory.log 
    </serial>
    <disk device="disk" type="file">
      <target dev="vda" bus="virtio"/>
      <source file="/var/lib/imagefactory/images/rhevm-image-e531d8e3-5e5e-459f-8dc2-fd96a2557211.dsk"/>
    </disk>
  </devices>
</domain>

2012-01-27 08:43:05,471 DEBUG oz.Guest.RHEL6Guest pid(24627) Message: Waiting for factory-build-79ae884e-7d8d-4957-92e2-54cb59e5a6fe to finish installing, 3600/3600
2012-01-27 08:43:05,496 DEBUG oz.Guest.RHEL6Guest pid(24627) Message: Waiting for factory-build-e531d8e3-5e5e-459f-8dc2-fd96a2557211 to finish installing, 3600/3600
[root@qeblade31 ~]# 


RAILS LOG:
R BY a.attnum

  Instance Load (0.4ms)  SELECT "instances".* FROM "instances" WHERE "instances"."pool_id" IN (1, 2) AND "instances"."deployment_id" = 8 ORDER BY "instances".name asc
  User Load (0.4ms)  SELECT "users".* FROM "users" WHERE ("users"."id" = 1)
  ProviderAccount Load (0.4ms)  SELECT "provider_accounts".* FROM "provider_accounts" WHERE "provider_accounts"."id" = 3 LIMIT 1
  Credential Load (0.3ms)  SELECT "credentials".* FROM "credentials" WHERE ("credentials".provider_account_id = 3)
  CredentialDefinition Load (0.3ms)  SELECT "credential_definitions".* FROM "credential_definitions" WHERE "credential_definitions"."id" = 7 LIMIT 1
  CredentialDefinition Load (0.3ms)  SELECT "credential_definitions".* FROM "credential_definitions" WHERE "credential_definitions"."id" = 8 LIMIT 1
  Provider Load (0.3ms)  SELECT "providers".* FROM "providers" WHERE "providers"."id" = 9 LIMIT 1
  ProviderType Load (0.3ms)  SELECT "provider_types".* FROM "provider_types" WHERE "provider_types"."id" = 5 LIMIT 1
Error connecting to framework: Network is unreachable - connect(2)
Backtrace: /usr/lib/ruby/1.8/net/http.rb:560:in `initialize'
/usr/lib/ruby/1.8/net/http.rb:560:in `open'
/usr/lib/ruby/1.8/net/http.rb:560:in `connect'
/usr/lib/ruby/1.8/timeout.rb:67:in `timeout'
/usr/lib/ruby/1.8/timeout.rb:101:in `timeout'
/usr/lib/ruby/1.8/net/http.rb:560:in `connect'
/usr/lib/ruby/1.8/net/http.rb:553:in `do_start'
/usr/lib/ruby/1.8/net/http.rb:542:in `start'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:166:in `transmit'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:60:in `execute'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:31:in `execute'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/resource.rb:54:in `get'
/usr/lib/ruby/gems/1.8/gems/deltacloud-client-0.4.0/lib/deltacloud.rb:370:in `send'
/usr/lib/ruby/gems/1.8/gems/deltacloud-client-0.4.0/lib/deltacloud.rb:370:in `request'
/usr/lib/ruby/gems/1.8/gems/deltacloud-client-0.4.0/lib/deltacloud.rb:261:in `discover_entry_points'
/usr/lib/ruby/gems/1.8/gems/deltacloud-client-0.4.0/lib/deltacloud.rb:77:in `initialize'
/usr/lib/ruby/gems/1.8/gems/deltacloud-client-0.4.0/lib/deltacloud.rb:40:in `new'
/usr/lib/ruby/gems/1.8/gems/deltacloud-client-0.4.0/lib/deltacloud.rb:40:in `new'
/usr/share/aeolus-conductor/app/models/provider_account.rb:129:in `connect'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `send'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `method_missing'
/usr/share/aeolus-conductor/app/models/instance_task.rb:70:in `valid_actions_for_instance_state'
/usr/share/aeolus-conductor/app/models/instance.rb:166:in `get_action_list'
/usr/share/aeolus-conductor/app/models/instance.rb:412:in `as_json'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:54:in `as_json'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:77:in `check_for_circular_references'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:53:in `as_json'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:209:in `as_json'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:209:in `map'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:209:in `as_json'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/relation.rb:80:in `as_json'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:46:in `encode'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:77:in `check_for_circular_references'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:45:in `encode'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/json/encoding.rb:30:in `encode'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/core_ext/object/to_json.rb:15:in `to_json'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/renderers.rb:74:in `_render_option_json'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/renderers.rb:32:in `_handle_render_options'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/renderers.rb:47:in `render_to_body'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/compatibility.rb:55:in `render_to_body'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/rendering.rb:102:in `render_to_string'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/rendering.rb:93:in `render'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rendering.rb:17:in `render'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:40:in `render'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/core_ext/benchmark.rb:5:in `ms'
/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/core_ext/benchmark.rb:5:in `ms'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:40:in `render'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:78:in `cleanup_view_runtime'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/railties/controller_runtime.rb:15:in `cleanup_view_runtime'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:39:in `render'
/usr/share/aeolus-conductor/app/controllers/instances_controller.rb:30:in `index'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/mime_responds.rb:192:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/mime_responds.rb:192:in `respond_to'
/usr/share/aeolus-conductor/app/controllers/instances_controller.rb:27:in `index'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/base.rb:150:in `process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rendering.rb:11:in `process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/callbacks.rb:18:in `process_action'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:451:in `_run__1752752833__process_action__199225275__callbacks'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in `send'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in `_run_process_action_callbacks'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:94:in `send'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:94:in `run_callbacks'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/callbacks.rb:17:in `process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rescue.rb:17:in `process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications.rb:52:in `instrument'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications.rb:52:in `instrument'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/base.rb:119:in `process'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/rendering.rb:41:in `process'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal.rb:138:in `dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal.rb:178:in `action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:62:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:62:in `dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:27:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:150:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:93:in `recognize'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:68:in `optimized_each'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:92:in `recognize'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:141:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:493:in `call'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:35:in `call'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:34:in `catch'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:34:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-restful_submit-1.2.1/lib/rack/rack-restful_submit.rb:29:in `call'
/usr/lib/ruby/gems/1.8/gems/sass-3.1.4/lib/sass/../sass/plugin/rack.rb:54:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/head.rb:14:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/methodoverride.rb:24:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/flash.rb:182:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/session/abstract_store.rb:149:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/cookies.rb:302:in `call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:32:in `call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in `cache'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:12:in `cache'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:31:in `call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/callbacks.rb:46:in `call'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:416:in `_run_call_callbacks'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/callbacks.rb:44:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/sendfile.rb:102:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/remote_ip.rb:48:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/show_exceptions.rb:47:in `call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/rack/logger.rb:13:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/runtime.rb:17:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/lock.rb:34:in `call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:168:in `call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:77:in `send'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:77:in `method_missing'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:52:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:46:in `each'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:46:in `call'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:84:in `pre_process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:in `catch'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:in `pre_process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:57:in `process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:42:in `receive_data'
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run_machine'
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/backends/base.rb:61:in `start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/server.rb:159:in `start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/controllers/controller.rb:86:in `start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:185:in `send'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:185:in `run_command'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:151:in `run!'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/bin/thin:6
/usr/bin/thin:19:in `load'
/usr/bin/thin:19
  InstanceKey Load (0.4ms)  SELECT "instance_keys".* FROM "instance_keys" WHERE ("instance_keys".instance_id = 8) LIMIT 1
Completed 200 OK in 135ms (Views: 15.0ms | ActiveRecord: 9.2ms)
[root@qeblade31 ~]#
Comment 1 wes hayutin 2012-01-27 14:00:11 UTC 
>> Changing process privilege to nobody:nobody
thin server (localhost:3002) [deltacloud-mock][24491]: >> Thin web server (v1.2.11 codename Bat-Shit Crazy)
thin server (localhost:3002) [deltacloud-mock][24491]: >> Debugging ON
thin server (localhost:3002) [deltacloud-mock][24491]: >> Maximum connections set to 1024
thin server (localhost:3002) [deltacloud-mock][24491]: >> Listening on localhost:3002, CTRL+C to stop
/usr/lib/ruby/1.8/net/protocol.rb:166: [BUG] Segmentation fault
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]


hrm.. recreated the ruby segfault .. not good

this is a dupe
Comment 2 wes hayutin 2012-01-27 14:00:34 UTC 

*** This bug has been marked as a duplicate of bug 782573 ***
Note You need to log in before you can comment on or make changes to this bug.