Bug 2119888 - Another deadlock issue when syncing repos with high concurrency
Summary: Another deadlock issue when syncing repos with high concurrency
Keywords:
Status: CLOSED DUPLICATE of bug 2062526
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.12.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: 6.12.0
Assignee: satellite6-bugs
QA Contact: Lai
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-08-19 19:18 UTC by Grant Gainey
Modified: 2022-09-13 13:41 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-09-13 13:41:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github pulp pulpcore issues 3111 0 None closed bulk_update() can still deadlock in content_stages 2022-08-26 04:49:10 UTC
Red Hat Issue Tracker SAT-12357 0 None None None 2022-08-22 13:12:32 UTC

Description Grant Gainey 2022-08-19 19:18:31 UTC
This bug was initially created as a copy of Bug #2062526

I am copying this bug because: 
Need to insure the fix gets backported for Sat-6.12


Description of problem:
We fixed a deadlock issue of updating timestamp in bug #2043710. Now there is another deadlock error when bulk updating artifacts. Perhaps we need to fix all places that are using the bulk_update to completely avoid the possibility of deadlock.


pulpcore.tasking.pulpcore_worker:INFO: Task 01370352-5e23-454c-b44c-cf6c59f738fa failed (deadlock detected
DETAIL:  Process 13970 waits for ShareLock on transaction 3618061; blocked by process 13969.
Process 13969 waits for ShareLock on transaction 3618064; blocked by process 13970.
HINT:  See server log for query details.
CONTEXT:  while rechecking updated tuple (314,15) in relation "core_contentartifact"
)
pulpcore.tasking.pulpcore_worker:INFO:   File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 339, in _perform_task
result = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 491, in synchronize
version = dv.create()
File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 151, in create
loop.run_until_complete(pipeline)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
await asyncio.gather(*futures)
File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
await self.run()
File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py", line 160, in run
ContentArtifact.objects.bulk_update(to_update_ca_bulk, ["artifact"])
File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 525, in bulk_update
self.filter(pk__in=pks).update(**update_kwargs)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 741, in update
rows = query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1471, in execute_sql
cursor = super().execute_sql(result_type)
File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1142, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)

Comment 1 Grant Gainey 2022-09-13 13:41:05 UTC
Closing as a dup, 2062526 is "the" BZ for fixing this in satellite-6.12

*** This bug has been marked as a duplicate of bug 2062526 ***


Note You need to log in before you can comment on or make changes to this bug.