Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1422051

Summary:	concurrent puppet reposyncs fail with PG::Error: ERROR: duplicate key value violates unique constraint "index_katello_puppet_modules_on_uuid"
Product:	Red Hat Satellite	Reporter:	Lukas Pramuk <lpramuk>
Component:	Content Management	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED WONTFIX	QA Contact:	Lukas Pramuk <lpramuk>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	6.2.8	CC:	bbuckingham, bkearney, hajek, jcallaha, lpramuk, mmccune, rplevka, sjagtap
Target Milestone:	Unspecified	Keywords:	Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-01-04 13:08:44 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lukas Pramuk 2017-02-14 11:38:17 UTC

Description of problem:
During indexing the puppet content that pulp has synced a race condition can occur: 

3:Katello: Actions::Pulp::Repository::Sync (success)
6:Actions::Katello::Repository::IndexContent fails when trying to insert puppet module uuid into katello_puppet_modules and failing due to unique constraint index_katello_puppet_modules_on_uuid

If the uuid is present why it just doesn't update it instead of insert?
Because it doesn't see it !!! in the meantime module uuids are inserted by another instance of reposync

=> race condition during Actions::Katello::Repository::IndexContent


Version-Release number of selected component (if applicable):
Sat6.2.8

How reproducible:
depends on timing

Steps to Reproduce:
1. run parallel puppet repository syncs of the new puppet content (that hasn't been yet synced)

Actual results:
Actions::Katello::Repository::IndexContent fails:
PG::Error: ERROR:  duplicate key value violates unique constraint "index_katello_puppet_modules_on_uuid"
DETAIL:  Key (uuid)=(5221bd7b-19e2-4b49-be3a-887a4bad0001) already exists.
: INSERT INTO "katello_puppet_modules" ("author", "created_at", "name", "sortable_version", "summary", "updated_at", "uuid", "version") VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING "id"

Expected results:
IndexContent handle race condition

Comment 1 Lukas Pramuk 2017-02-14 11:40:06 UTC

6: Actions::Katello::Repository::IndexContent (skipped) [ 5.25s / 2.59s ]

Started at: 2017-02-10 16:01:10 UTC

Ended at: 2017-02-10 16:01:15 UTC

Real time: 5.25s

Execution time (excluding suspended state): 2.59s

Input:

---
id: 4
contents_changed: true
current_user_id: 3

Output:

--- {}

Error:

ActiveRecord::RecordNotUnique

PG::Error: ERROR: duplicate key value violates unique constraint "index_katello_puppet_modules_on_uuid" DETAIL: Key (uuid)=(5221bd7b-19e2-4b49-be3a-887a4bad0001) already exists. : INSERT INTO "katello_puppet_modules" ("author", "created_at", "name", "sortable_version", "summary", "updated_at", "uuid", "version") VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING "id"

---
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:834:in
  `get_last_result'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:834:in
  `block in exec_cache'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract_adapter.rb:373:in
  `block in log'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications/instrumenter.rb:20:in
  `instrument'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract_adapter.rb:367:in
  `log'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:831:in
  `exec_cache'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql/database_statements.rb:138:in
  `exec_query'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql/database_statements.rb:177:in
  `exec_insert'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:95:in
  `insert'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/query_cache.rb:14:in
  `insert'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation.rb:64:in
  `insert'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/persistence.rb:502:in
  `_create_record'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/attribute_methods/dirty.rb:87:in
  `_create_record'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/callbacks.rb:306:in
  `block in _create_record'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:113:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:113:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in
  `block in halting_and_conditional'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in
  `block in halting_and_conditional'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in
  `block in halting_and_conditional'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in
  `block in halting_and_conditional'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in
  `run_callbacks'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/callbacks.rb:306:in
  `_create_record'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/timestamp.rb:57:in
  `_create_record'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/persistence.rb:482:in
  `create_or_update'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/callbacks.rb:302:in
  `block in create_or_update'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:113:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:113:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in
  `block in halting'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in
  `block in halting'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in
  `call'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in
  `run_callbacks'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/callbacks.rb:302:in
  `create_or_update'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/persistence.rb:125:in
  `save!'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/validations.rb:57:in
  `save!'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/attribute_methods/dirty.rb:29:in
  `save!'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in
  `block in save!'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:329:in
  `block in with_transaction_returning_status'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:199:in
  `transaction'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:208:in
  `transaction'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:326:in
  `with_transaction_returning_status'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in
  `save!'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/protected_attributes-1.1.3/lib/active_record/mass_assignment_security/persistence.rb:77:in
  `block in update!'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:329:in
  `block in with_transaction_returning_status'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in
  `block in transaction'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:209:in
  `within_new_transaction'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in
  `transaction'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:208:in
  `transaction'"
- "/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:326:in
  `with_transaction_returning_status'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/protected_attributes-1.1.3/lib/active_record/mass_assignment_security/persistence.rb:75:in
  `update!'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/models/katello/puppet_module.rb:78:in
  `update_from_json'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/models/katello/concerns/pulp_database_unit.rb:52:in
  `block (2 levels) in import_all'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/models/katello/concerns/pulp_database_unit.rb:47:in
  `each'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/models/katello/concerns/pulp_database_unit.rb:47:in
  `block in import_all'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/services/katello/pulp/pulp_content_unit.rb:51:in
  `block in fetch_by_uuids'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/services/katello/pulp/pulp_content_unit.rb:48:in
  `each_slice'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/services/katello/pulp/pulp_content_unit.rb:48:in
  `fetch_by_uuids'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/models/katello/concerns/pulp_database_unit.rb:62:in
  `import_all'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/models/katello/concerns/pulp_database_unit.rb:72:in
  `import_for_repository'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/models/katello/glue/pulp/repo.rb:721:in
  `index_content'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.102/app/lib/actions/katello/repository/index_content.rb:16:in
  `run'"

Comment 6 Mike McCune 2017-09-13 21:01:55 UTC

Is this still reproducable on 6.2.11 or later? if not, can we close this out?

Comment 9 Roman Plevka 2017-10-02 10:00:18 UTC

(In reply to Mike McCune from comment #6)
> Is this still reproducable on 6.2.11 or later? if not, can we close this out?

we just hit this on 6.3.0-18
I guess this indirectly answers your questions

Comment 12 Roman Plevka 2018-08-07 09:42:01 UTC

hit this on 6.4.0-16 during puppet module upload.

Comment 13 Bryan Kearney 2018-11-30 14:59:22 UTC

The Satellite Team is attempting to provide an accurate backlog of bugzilla requests which we feel will be resolved in the next few releases. We do not believe this bugzilla will meet that criteria, and have plans to close it out in 1 month. This is not a reflection on the validity of the request, but a reflection of the many priorities for the product. If you have any concerns about this, feel free to contact Rich Jerrido or Bryan Kearney or your account team. If we do not hear from you, we will close this bug out. Thank you.

Comment 14 Bryan Kearney 2019-01-04 13:08:44 UTC

Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this, please do not reopen. Instead, feel free to contact Red Hat Technical Support. Thank you.