Bug 865811 - Pulp timeouts under load
Pulp timeouts under load
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Content Management (Show other bugs)
6.0.0
Unspecified Unspecified
medium Severity medium (vote)
: Unspecified
: --
Assigned To: Miroslav Suchý
Og Maciel
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-12 09:45 EDT by Thom Carlin
Modified: 2014-09-18 11:32 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Pulp froze due to a low concurrency threshold. Using too small number froze repository sync in System Engine. This fix adjusts the concurrency threshold to an optimal value based on the host CPU cores.
Story Points: ---
Clone Of:
: 867388 (view as bug list)
Environment:
Last Closed: 2012-12-04 14:57:21 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Thom Carlin 2012-10-12 09:45:04 EDT
Description of problem:

Pulp::Request timeouts

Version-Release number of selected component (if applicable):

pulp 1.1.12-1.el6cf

How reproducible:

Sporatic but frequent under system load

Steps to Reproduce:
1. Synchronize a few large channels (e.g. RHEL 6 Server RPM's, RHEL 5 Server RPM's) at the same time
2. Check Sync Status
3.
  
Actual results:

 Resources::Pulp::Repository: Request Timeout (GET
/pulp/api/repositories/?_intersect=groupid&groupid=env:2)
(RestClient::RequestTimeout) and 
Resources::Pulp::Consumer: Request Timeout  (GET
/pulp/api/consumers/applicable_errata_in_repos/?repoids=<long_list>

Expected results:

Actual status

Additional info:

/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:174:in
`transmit'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:60:in
`execute'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:31:in
`execute'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/resource.rb:54:in
`get'
/usr/share/katello/lib/http_resource.rb:97:in `get'
/usr/share/katello/lib/resources/pulp.rb:205:in `all'
/usr/share/katello/app/models/glue/pulp/repos.rb:139:in `prepopulate!'
/usr/share/katello/app/helpers/sync_management_helper.rb:42:in
`collect_repos'
/usr/share/katello/app/controllers/sync_management_controller.rb:75:in
`index'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/implicit_render.rb:4:in
`send_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/implicit_render.rb:4:in
`send_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/base.rb:150:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rendering.rb:11:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/callbacks.rb:18:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:467:in
`_run__248694912__process_action__1464761126__callbacks'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:221:in
`_conditional_callback_around_3376'
/usr/share/katello/lib/util/threadsession.rb:79:in `thread_locals'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:220:in
`_conditional_callback_around_3376'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:451:in
`_run__248694912__process_action__1464761126__callbacks'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in
`send'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in
`_run_process_action_callbacks'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:94:in
`send'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:94:in
`run_callbacks'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/callbacks.rb:17:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rescue.rb:17:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:30:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications.rb:52:in
`instrument'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications/instrumenter.rb:21:in
`instrument'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications.rb:52:in
`instrument'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:29:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/base.rb:119:in
`process'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/rendering.rb:41:in
`process'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal.rb:138:in
`dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rack_delegation.rb:14:in
`dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal.rb:178:in
`action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:62:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:62:in
`dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:27:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:150:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:93:in
`recognize'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:68:in
`optimized_each'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:92:in
`recognize'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:141:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:493:in
`call'
/usr/lib/ruby/gems/1.8/gems/apipie-rails-0.0.11/lib/apipie/static_dispatcher.rb:57:in
`call'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:35:in `call'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:34:in `catch'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:34:in `call'
/usr/lib/ruby/gems/1.8/gems/sass-3.1.4/lib/sass/../sass/plugin/rack.rb:54:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/best_standards_support.rb:17:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/head.rb:14:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/methodoverride.rb:24:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/params_parser.rb:21:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/flash.rb:182:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/session/abstract_store.rb:149:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/cookies.rb:302:in
`call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:32:in
`call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in
`cache'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:12:in
`cache'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:31:in
`call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/callbacks.rb:46:in
`call'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:416:in
`_run_call_callbacks'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/callbacks.rb:44:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/sendfile.rb:102:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/remote_ip.rb:48:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/show_exceptions.rb:47:in
`call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/rack/logger.rb:13:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/runtime.rb:17:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/lock.rb:34:in `call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:168:in
`call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:77:in
`send'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:77:in
`method_missing'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:52:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:46:in `each'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:46:in `call'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:84:in
`pre_process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:in `catch'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:in
`pre_process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:57:in
`process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:42:in
`receive_data'
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`run_machine'
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`run'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/backends/base.rb:61:in
`start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/server.rb:159:in `start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/controllers/controller.rb:86:in
`start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:185:in `send'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:185:in
`run_command'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:151:in `run!'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/bin/thin:6
/usr/share/katello/script/thin:24:in `load'
/usr/share/katello/script/thin:24

/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:174:in
`transmit'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:60:in
`execute'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:31:in
`execute'
/usr/lib/ruby/gems/1.8/gems/rest-client-1.6.1/lib/restclient/resource.rb:54:in
`get'
/usr/share/katello/lib/http_resource.rb:97:in `get'
/usr/share/katello/lib/resources/pulp.rb:480:in `errata_by_consumer'
/usr/share/katello/app/models/glue/pulp/errata.rb:29:in `errata_by_consumer'
/usr/share/katello/app/controllers/dashboard_controller.rb:40:in `errata'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/implicit_render.rb:4:in
`send_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/implicit_render.rb:4:in
`send_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/base.rb:150:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rendering.rb:11:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/callbacks.rb:18:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:462:in
`_run__1338139692__process_action__908252374__callbacks'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:221:in
`_conditional_callback_around_3373'
/usr/share/katello/lib/util/threadsession.rb:79:in `thread_locals'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:220:in
`_conditional_callback_around_3373'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:446:in
`_run__1338139692__process_action__908252374__callbacks'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in
`send'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in
`_run_process_action_callbacks'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:94:in
`send'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:94:in
`run_callbacks'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/callbacks.rb:17:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rescue.rb:17:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:30:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications.rb:52:in
`instrument'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications/instrumenter.rb:21:in
`instrument'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications.rb:52:in
`instrument'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:29:in
`process_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/base.rb:119:in
`process'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/rendering.rb:41:in
`process'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal.rb:138:in
`dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rack_delegation.rb:14:in
`dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal.rb:178:in
`action'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:62:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:62:in
`dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:27:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:150:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:93:in
`recognize'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:68:in
`optimized_each'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:92:in
`recognize'
/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:141:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:493:in
`call'
/usr/lib/ruby/gems/1.8/gems/apipie-rails-0.0.11/lib/apipie/static_dispatcher.rb:57:in
`call'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:35:in `call'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:34:in `catch'
/usr/lib/ruby/gems/1.8/gems/warden-1.0.5/lib/warden/manager.rb:34:in `call'
/usr/lib/ruby/gems/1.8/gems/sass-3.1.4/lib/sass/../sass/plugin/rack.rb:54:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/best_standards_support.rb:17:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/head.rb:14:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/methodoverride.rb:24:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/params_parser.rb:21:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/flash.rb:182:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/session/abstract_store.rb:149:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/cookies.rb:302:in
`call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:32:in
`call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in
`cache'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:12:in
`cache'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/query_cache.rb:31:in
`call'
/usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/callbacks.rb:46:in
`call'
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:416:in
`_run_call_callbacks'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/callbacks.rb:44:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/sendfile.rb:102:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/remote_ip.rb:48:in
`call'
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/show_exceptions.rb:47:in
`call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/rack/logger.rb:13:in
`call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/runtime.rb:17:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/lock.rb:34:in `call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:168:in
`call'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:77:in
`send'
/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:77:in
`method_missing'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:52:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:46:in `each'
/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/urlmap.rb:46:in `call'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:84:in
`pre_process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:in `catch'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:in
`pre_process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:57:in
`process'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:42:in
`receive_data'
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`run_machine'
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`run'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/backends/base.rb:61:in
`start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/server.rb:159:in `start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/controllers/controller.rb:86:in
`start'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:185:in `send'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:185:in
`run_command'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/runner.rb:151:in `run!'
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/bin/thin:6
/usr/share/katello/script/thin:24:in `load'
/usr/share/katello/script/thin:24
Comment 3 Jason Connor 2012-10-16 15:14:46 EDT
To keep Pulp from saturating its task queue, the following should be added to the /etc/pulp/pulp.conf file:

[tasking]
concurrency_threshold = N

Where N is: (2 * #CPUs) - 1

This recommended minimum # of CPUs is: 2



These configuration values will generally keep the number of concurrent syncs down to a level that keeps the server responsive to subsequent requests.
Comment 4 Miroslav Suchý 2012-10-17 07:12:39 EDT
https://github.com/Katello/katello/pull/868
Comment 5 James Laska 2012-10-17 07:31:59 EDT
(In reply to comment #4)
> https://github.com/Katello/katello/pull/868

Brilliant, thanks for the pull request link.  I see now how katello-configure will be responsible for dynamically setting the value in pulp.conf.   I was under the impression some RPM %postinstall logic would be used ... this is much better :)
Comment 6 Miroslav Suchý 2012-10-17 07:43:13 EDT
xsuchy 	865811 - set concurrency threshold for pulp 	0bd8ebe
Comment 8 Miroslav Suchý 2012-10-17 11:29:14 EDT
Note:
While Jason suggested value
 (2 * #CPUs) - 1
I used
  #CPUs <= 1 ? 1 : #CPU-1

where #CPU is core (not physical socket). I'm using this value mostly because we have to run other services (foreman, candlepin) and we could not allow pulp to consume all resources. At least as default value, which should be conservative.
Comment 10 Ivan Necas 2012-10-22 06:28:17 EDT
This calculation caused syncing to freeze (more tasks in queue, but no one being processed) when running 2 processors. I'm turning the calculation to what Jason suggests in https://github.com/Katello/katello/pull/885.
Comment 11 Ivan Necas 2012-10-22 08:07:22 EDT
After conversation on the pull-request, the calcuation was changed to #CPU + 1, as a compromise between the greedy and defensive solution.
Comment 14 Og Maciel 2012-10-24 13:48:57 EDT
Verified (synced 35GB of data):

* candlepin-0.7.8.1-1.el6cf.noarch
* candlepin-selinux-0.7.8.1-1.el6cf.noarch
* candlepin-tomcat6-0.7.8.1-1.el6cf.noarch
* katello-1.1.12-17.el6cf.noarch
* katello-all-1.1.12-17.el6cf.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.1.8-1.el6cf.noarch
* katello-cli-1.1.8-9.el6cf.noarch
* katello-cli-common-1.1.8-9.el6cf.noarch
* katello-common-1.1.12-17.el6cf.noarch
* katello-configure-1.1.9-8.el6cf.noarch
* katello-glue-candlepin-1.1.12-17.el6cf.noarch
* katello-glue-pulp-1.1.12-17.el6cf.noarch
* katello-qpid-broker-key-pair-1.0-1.noarch
* katello-qpid-client-key-pair-1.0-1.noarch
* katello-selinux-1.1.1-2.el6cf.noarch
* pulp-1.1.14-1.el6cf.noarch
* pulp-common-1.1.14-1.el6cf.noarch
* pulp-selinux-server-1.1.14-1.el6cf.noarch
Comment 16 errata-xmlrpc 2012-12-04 14:57:21 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-1543.html

Note You need to log in before you can comment on or make changes to this bug.