Bug 865811
Summary: | Pulp timeouts under load | |||
---|---|---|---|---|
Product: | Red Hat Satellite | Reporter: | Thom Carlin <tcarlin> | |
Component: | Content Management | Assignee: | Miroslav Suchý <msuchy> | |
Status: | CLOSED ERRATA | QA Contact: | Og Maciel <omaciel> | |
Severity: | medium | Docs Contact: | ||
Priority: | medium | |||
Version: | 6.0.0 | CC: | asettle, cwelton, dmacpher, inecas, jconnor, jlaska, mhuth, mmccune, msuchy, omaciel | |
Target Milestone: | Unspecified | Keywords: | Triaged | |
Target Release: | Unused | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | Doc Type: | Bug Fix | ||
Doc Text: |
Pulp froze due to a low concurrency threshold. Using too small number froze repository sync in System Engine. This fix adjusts the concurrency threshold to an optimal value based on the host CPU cores.
|
Story Points: | --- | |
Clone Of: | ||||
: | 867388 (view as bug list) | Environment: | ||
Last Closed: | 2012-12-04 19:57:21 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: |
Description
Thom Carlin
2012-10-12 13:45:04 UTC
To keep Pulp from saturating its task queue, the following should be added to the /etc/pulp/pulp.conf file: [tasking] concurrency_threshold = N Where N is: (2 * #CPUs) - 1 This recommended minimum # of CPUs is: 2 These configuration values will generally keep the number of concurrent syncs down to a level that keeps the server responsive to subsequent requests. (In reply to comment #4) > https://github.com/Katello/katello/pull/868 Brilliant, thanks for the pull request link. I see now how katello-configure will be responsible for dynamically setting the value in pulp.conf. I was under the impression some RPM %postinstall logic would be used ... this is much better :) xsuchy 865811 - set concurrency threshold for pulp 0bd8ebe Note: While Jason suggested value (2 * #CPUs) - 1 I used #CPUs <= 1 ? 1 : #CPU-1 where #CPU is core (not physical socket). I'm using this value mostly because we have to run other services (foreman, candlepin) and we could not allow pulp to consume all resources. At least as default value, which should be conservative. This calculation caused syncing to freeze (more tasks in queue, but no one being processed) when running 2 processors. I'm turning the calculation to what Jason suggests in https://github.com/Katello/katello/pull/885. After conversation on the pull-request, the calcuation was changed to #CPU + 1, as a compromise between the greedy and defensive solution. Verified (synced 35GB of data): * candlepin-0.7.8.1-1.el6cf.noarch * candlepin-selinux-0.7.8.1-1.el6cf.noarch * candlepin-tomcat6-0.7.8.1-1.el6cf.noarch * katello-1.1.12-17.el6cf.noarch * katello-all-1.1.12-17.el6cf.noarch * katello-candlepin-cert-key-pair-1.0-1.noarch * katello-certs-tools-1.1.8-1.el6cf.noarch * katello-cli-1.1.8-9.el6cf.noarch * katello-cli-common-1.1.8-9.el6cf.noarch * katello-common-1.1.12-17.el6cf.noarch * katello-configure-1.1.9-8.el6cf.noarch * katello-glue-candlepin-1.1.12-17.el6cf.noarch * katello-glue-pulp-1.1.12-17.el6cf.noarch * katello-qpid-broker-key-pair-1.0-1.noarch * katello-qpid-client-key-pair-1.0-1.noarch * katello-selinux-1.1.1-2.el6cf.noarch * pulp-1.1.14-1.el6cf.noarch * pulp-common-1.1.14-1.el6cf.noarch * pulp-selinux-server-1.1.14-1.el6cf.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2012-1543.html |