Red Hat Bugzilla – Bug 1464559
Deadlock when registering many consumers and consuming subscriptions
Last modified: 2017-09-22 15:15:57 EDT
Created attachment 1291242 [details]
Archive containing the test script and its required library
Description of problem:
While running a stress test to attempt to reproduce a deadlock issue in Candlepin, we managed to reliably trigger a deadlock within Katello.
The stress test creates many consumers, consumes many pools in random order, then unregisters the consumer, all in parallel. At the time of writing, the test runs 50 threads which each run through 50 iterations of creating a consumer, consuming 25 pools (in random order) and then unregistering. This script will be attached to this BZ, as well as the manifest used to set up the test data.
Note that the included manifest will fail signature validation within Candlepin. We worked around this by disabling the check in Candlepin before importing, but another option would be to bypass the check using the force flag(s). If neither of these options are acceptable, any manifest which creates subscriptions with sku products named "export_prod-0" through "export_prod-25" should be acceptable.
Also note that the KatelloModule must be disabled in Candlepin, and simple auth must be enabled to allow direct access to certain endpoints to fetch the proper IDs used by the script.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Import manifest containing test data to the default organization
2. Update the IDs in the script to properly match those of the default organization and default environment
3. Run the test script
Deadlock (see attached stack trace)
No deadlock in Katello
Created attachment 1291243 [details]
Manifest containing data used by the test script
Created attachment 1291244 [details]
Stack trace from foreman/production.log
*** This bug has been marked as a duplicate of bug 1425212 ***