Bug 1125229 - :subscriptions :upload - sequential API calls lead to random failures
Summary: :subscriptions :upload - sequential API calls lead to random failures
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: API
Version: Unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
medium vote
Target Milestone: Unspecified
Assignee: Tom McKay
QA Contact: Katello QA List
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-07-31 11:26 UTC by Lukas Pramuk
Modified: 2016-02-26 14:04 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-26 14:04:46 UTC


Attachments (Terms of Use)
ruby apipie-bindings reproducer (948 bytes, application/x-ruby)
2014-07-31 11:26 UTC, Lukas Pramuk
no flags Details
ruby apipie-bindings reproducer (968 bytes, text/plain)
2015-07-03 11:27 UTC, Lukas Pramuk
no flags Details

Description Lukas Pramuk 2014-07-31 11:26:12 UTC
Created attachment 922878 [details]
ruby apipie-bindings reproducer

Description of problem:
sequential API calls for uploading manifests lead to random failures.
Out of 10 calls only 2 or 3 are successful. The others are failures (likely with older composes) or even have manifests history empty like no calls were placed!

(Yes. The manifests are not the same. 10 unique manifests)

Version-Release number of selected component (if applicable):
Satellite-6.0.4-RHEL-6-20140723.0

How reproducible:
100%

Steps to Reproduce:

0. prepare multiple manifests named manifest-0..9.zip (manifests are unique)

1. use attached apipie-bindings reproducer:
# bundle exec 'reproducer.rb'
Creating organization #0
Creating organization #1
Creating organization #2
Creating organization #3
Creating organization #4
Creating organization #5
Creating organization #6
Creating organization #7
Creating organization #8
Creating organization #9
Uploading manifest #0
Uploading manifest #1
Uploading manifest #2
Uploading manifest #3
Uploading manifest #4
Uploading manifest #5
Uploading manifest #6
Uploading manifest #7
Uploading manifest #8
Uploading manifest #9

Manifest history
[{"statusMessage"=>"org-0 file imported successfully.",
  "status"=>"SUCCESS",
  "created"=>"2014-07-30T18:35:37.370+0000"}]

Manifest history
[{"statusMessage"=>"org-1 file imported successfully.",
  "status"=>"SUCCESS",
  "created"=>"2014-07-30T18:35:42.082+0000"}]

Manifest history
[]

Manifest history
[]

Manifest history
[]

Manifest history
[]

Manifest history
[]

Manifest history
[]

Manifest history
[]

Manifest history
[]


Actual results:
2/10 success

Expected results:
10/10 success

Additional info:
sleep between api calls has magic power to fix things

Comment 1 Lukas Pramuk 2014-07-31 11:38:10 UTC
Actions::Katello::Provider::ManifestImport

Input:

{"provider"=>{"id"=>332, "name"=>"Red Hat"},
 "organization"=>{"id"=>167, "name"=>"org-3", "label"=>"org-3"},
 "path"=>"/usr/share/foreman/tmp/import_9a6f6be2f343aad13bce.zip",
 "force"=>nil,
 "remote_user"=>"admin-305a5f03",
 "locale"=>"en"}

Output:

{}

Exception:

Katello::Errors::CandlepinError: Runtime Error ERROR: deadlock detected
  Detail: Process 9608 waits for ShareLock on transaction 154858; blocked by process 28108.
Process 28108 waits for ExclusiveLock on tuple (5,1) of relation 16582 of database 16386; blocked by process 9608.
  Hint: See server log for query details. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse:2,062

Backtrace:

/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:35:in `rescue in propagate_candlepin_errors'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:33:in `propagate_candlepin_errors'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:23:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:25:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:51:in `block (2 levels) in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/util/thread_session.rb:85:in `set_pulp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:41:in `as_pulp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:50:in `block in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/util/thread_session.rb:92:in `set_cp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:36:in `as_cp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:49:in `as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:25:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action/progress.rb:16:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/keep_locale.rb:24:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/keep_locale.rb:35:in `with_locale'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/keep_locale.rb:24:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:442:in `block (2 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:441:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:441:in `block in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:365:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:365:in `block in with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:365:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:365:in `with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:436:in `execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:230:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract.rb:152:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract.rb:152:in `with_meta_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/executors/parallel/worker.rb:20:in `block in on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in `block in assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `tap'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in `match_value'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in `block in match'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `match'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/executors/parallel/worker.rb:17:in `on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:82:in `on_envelope'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:72:in `receive'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `block (2 levels) in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `loop'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:13:in `block in initialize'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
caused by: (RestClient::InternalServerError) Katello::Resources::Candlepin::Owner: 500 Internal Server Error {"displayMessage":"Runtime Error ERROR: deadlock detected\n  Detail: Process 9608 waits for ShareLock on transaction 154858; blocked by process 28108.\nProcess 28108 waits for ExclusiveLock on tuple (5,1) of relation 16582 of database 16386; blocked by process 9608.\n  Hint: See server log for query details. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse:2,062","requestUuid":"9082e8e3-357e-48a0-acf3-8f1a8438fe02"} (POST /candlepin/owners/org-3/imports)
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result'
/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.28/lib/restclient_ext/request.rb:50:in `block in transmit'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:746:in `start'
/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.28/lib/restclient_ext/request.rb:44:in `transmit'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in `post'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/http_resource.rb:111:in `post'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/resources/candlepin.rb:388:in `import'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue/provider.rb:172:in `owner_import'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:177:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:127:in `block in process'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:114:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:114:in `process'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:36:in `on_save'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_9550'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:403:in `_run__3542827632456180223__save__3936344273939831370__callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/callbacks.rb:264:in `create_or_update'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/persistence.rb:104:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/validations.rb:56:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/attribute_methods/dirty.rb:33:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:246:in `block in save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:208:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:293:in `with_transaction_returning_status'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:246:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue/provider.rb:283:in `queue_import_manifest'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue/provider.rb:34:in `import_manifest'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/katello/provider/manifest_import.rb:42:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:443:in `block (3 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:23:in `block in run'
    ... skipped 61 lines

Comment 2 RHEL Product and Program Management 2014-07-31 11:44:00 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 4 Lukas Pramuk 2014-07-31 11:46:40 UTC
Actions::Katello::Provider::ManifestImport

Input:

{"provider"=>{"id"=>334, "name"=>"Red Hat"},
 "organization"=>{"id"=>168, "name"=>"org-4", "label"=>"org-4"},
 "path"=>"/usr/share/foreman/tmp/import_67576e9fdef65a8e88ae.zip",
 "force"=>nil,
 "remote_user"=>"admin-305a5f03",
 "locale"=>"en"}

Output:

{}

Exception:

Katello::Errors::CandlepinError: Runtime Error Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1 at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched:81

Backtrace:

/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:35:in `rescue in propagate_candlepin_errors'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:33:in `propagate_candlepin_errors'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:23:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:25:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:51:in `block (2 levels) in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/util/thread_session.rb:85:in `set_pulp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:41:in `as_pulp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:50:in `block in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/util/thread_session.rb:92:in `set_cp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:36:in `as_cp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:49:in `as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/remote_action.rb:25:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action/progress.rb:16:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/keep_locale.rb:24:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/keep_locale.rb:35:in `with_locale'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/keep_locale.rb:24:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:442:in `block (2 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:441:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:441:in `block in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:365:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:365:in `block in with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:365:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:365:in `with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:436:in `execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:230:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract.rb:152:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract.rb:152:in `with_meta_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/executors/parallel/worker.rb:20:in `block in on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in `block in assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `tap'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in `match_value'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in `block in match'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `match'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/executors/parallel/worker.rb:17:in `on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:82:in `on_envelope'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:72:in `receive'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `block (2 levels) in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `loop'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:99:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/micro_actor.rb:13:in `block in initialize'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
caused by: (RestClient::InternalServerError) Katello::Resources::Candlepin::Owner: 500 Internal Server Error {"displayMessage":"Runtime Error Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1 at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched:81","requestUuid":"251e7130-218d-46a7-9d82-d250a8220080"} (POST /candlepin/owners/org-4/imports)
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result'
/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.28/lib/restclient_ext/request.rb:50:in `block in transmit'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:746:in `start'
/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.28/lib/restclient_ext/request.rb:44:in `transmit'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in `post'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/http_resource.rb:111:in `post'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/resources/candlepin.rb:388:in `import'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue/provider.rb:172:in `owner_import'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:177:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:127:in `block in process'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:114:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:114:in `process'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue.rb:36:in `on_save'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_9550'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:403:in `_run__3542827632456180223__save__3936344273939831370__callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/callbacks.rb:264:in `create_or_update'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/persistence.rb:104:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/validations.rb:56:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/attribute_methods/dirty.rb:33:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:246:in `block in save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:208:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:293:in `with_transaction_returning_status'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:246:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue/provider.rb:283:in `queue_import_manifest'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/models/katello/glue/provider.rb:34:in `import_manifest'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/katello/provider/manifest_import.rb:42:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/action.rb:443:in `block (3 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.2/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:23:in `block in run'
    ... skipped 61 lines

Comment 5 Lukas Pramuk 2015-07-03 11:17:37 UTC
Revisited after 6.1 PB - the bug is still valid.
I've just needed to adapt 'reproducer.rb' to Sat6.1 API

Comment 6 Lukas Pramuk 2015-07-03 11:27:03 UTC
Created attachment 1045828 [details]
ruby apipie-bindings reproducer

Comment 7 Lukas Pramuk 2015-07-03 11:30:09 UTC
Action:

Actions::Katello::Provider::ManifestImport

Input:

{"provider"=>{"id"=>42, "name"=>"Red Hat"},
 "organization"=>{"id"=>22, "name"=>"oorg-6", "label"=>"oorg-6"},
 "path"=>"/usr/share/foreman/tmp/import_747c54c8508579c3e2d6.zip",
 "force"=>nil,
 "remote_user"=>"admin-9a8fdad5",
 "remote_cp_user"=>"admin",
 "locale"=>"en"}

Output:

{}

Exception:

Katello::Errors::CandlepinError: Runtime Error ERROR: deadlock detected
  Detail: Process 43457 waits for ShareLock on transaction 53875; blocked by process 42587.
Process 42587 waits for ExclusiveLock on tuple (0,76) of relation 16581 of database 16387; blocked by process 43457.
  Hint: See server log for query details. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse:2,094

Backtrace:

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/propagate_candlepin_errors.rb:33:in `rescue in propagate_candlepin_errors'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/propagate_candlepin_errors.rb:31:in `propagate_candlepin_errors'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/propagate_candlepin_errors.rb:21:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/remote_action.rb:27:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/remote_action.rb:57:in `block (2 levels) in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/katello/util/thread_session.rb:84:in `pulp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/remote_action.rb:43:in `as_pulp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/remote_action.rb:56:in `block in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/katello/util/thread_session.rb:91:in `cp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/remote_action.rb:38:in `as_cp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/remote_action.rb:55:in `as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/remote_action.rb:27:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action/progress.rb:16:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/keep_locale.rb:23:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/keep_locale.rb:34:in `with_locale'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/keep_locale.rb:23:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:480:in `block (2 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:479:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:479:in `block in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:396:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:396:in `block in with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:396:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:396:in `with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:474:in `execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:256:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/execution_plan/steps/abstract.rb:155:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/execution_plan/steps/abstract.rb:155:in `with_meta_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/executors/parallel/worker.rb:20:in `block in on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in `block in assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `tap'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in `match_value'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in `block in match'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `match'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/executors/parallel/worker.rb:17:in `on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/micro_actor.rb:82:in `on_envelope'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/micro_actor.rb:72:in `receive'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/micro_actor.rb:99:in `block (2 levels) in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/micro_actor.rb:99:in `loop'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/micro_actor.rb:99:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/micro_actor.rb:99:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/micro_actor.rb:99:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/micro_actor.rb:13:in `block in initialize'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
caused by: (RestClient::InternalServerError) Katello::Resources::Candlepin::Owner: 500 Internal Server Error {"displayMessage":"Runtime Error ERROR: deadlock detected\n  Detail: Process 43457 waits for ShareLock on transaction 53875; blocked by process 42587.\nProcess 42587 waits for ExclusiveLock on tuple (0,76) of relation 16581 of database 16387; blocked by process 43457.\n  Hint: See server log for query details. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse:2,094","requestUuid":"3c411c2a-4d49-4301-9f06-d59fcd51b2f2"} (POST /candlepin/owners/oorg-6/imports)
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result'
/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:50:in `block in transmit'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:746:in `start'
/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:44:in `transmit'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in `post'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/katello/http_resource.rb:110:in `post'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/katello/resources/candlepin.rb:398:in `import'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/models/katello/glue/provider.rb:153:in `owner_import'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/models/katello/glue.rb:176:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/models/katello/glue.rb:126:in `block in process'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/models/katello/glue.rb:113:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/models/katello/glue.rb:113:in `process'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/models/katello/glue.rb:35:in `on_save'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_11520'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:403:in `_run__2038188648126203631__save__1883018000536142230__callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/callbacks.rb:264:in `create_or_update'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/persistence.rb:104:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/validations.rb:56:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/attribute_methods/dirty.rb:33:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:246:in `block in save!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:208:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:293:in `with_transaction_returning_status'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:246:in `save!'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/models/katello/glue/provider.rb:260:in `queue_import_manifest'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/models/katello/glue/provider.rb:28:in `import_manifest'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/katello/provider/manifest_import.rb:40:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/action.rb:481:in `block (3 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:26:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.7/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.56/app/lib/actions/middleware/propagate_candlepin_errors.rb:21:in `block in run'
    ... skipped 61 lines


Note You need to log in before you can comment on or make changes to this bug.