Description of problem:

Under certain circumstances, Upgrade to 6.13.1 can fail for Satellite as well as any external capsules, on the "pulpcore-manager migrate --noinput" step and it happens due to certain Deadlocks


Version-Release number of selected component (if applicable):

Satellite\Capsule 6.13.1


How reproducible:

For specific customers only


Steps to Reproduce ( Probable ):

1. Install a Satellite+Capsule 6.12 with lots of pulp and katello content.
2. Try upgrading the setup to 6.13.


Actual results:


2023-07-03 11:36:04 [NOTICE] [configure] 2250 configuration steps out of 3004 steps complete.
2023-07-03 11:36:05 [NOTICE] [configure] 2500 configuration steps out of 3004 steps complete.
2023-07-03 11:36:05 [NOTICE] [configure] 2750 configuration steps out of 3004 steps complete.
2023-07-03 11:36:27 [ERROR ] [configure] 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]
2023-07-03 11:36:27 [ERROR ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: change from 'notrun' to ['0'] failed: 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]
2023-07-03 11:36:28 [NOTICE] [configure] 3000 configuration steps out of 3004 steps complete.
2023-07-03 11:36:31 [NOTICE] [configure] System configuration has finished.

  There were errors detected during install.
  Please address the errors and re-run the installer to ensure the system is properly configured.
  Failing to do so is likely to result in broken functionality.

  The full log is at /var/log/foreman-installer/satellite.log
Package versions are being locked.
                                      [FAIL]
Failed executing satellite-installer, exit status 6
--------------------------------------------------------------------------------
Scenario [Migration scripts to Satellite 6.13] failed.



Expected results:


No such errors:


Additional info:


From upgrade log:


2023-07-03 10:59:18 [DEBUG ] [configure] Executing with uid=pulp: 'pulpcore-manager migrate --noinput'
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Operations to perform:
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   Apply all migrations: ansible, auth, certguard, container, contenttypes, core, file, rpm, sessions
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Running migrations:
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   Applying core.0090_char_to_text_field...Traceback (most recent call last):
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     return self.cursor.execute(sql, params)
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: psycopg2.errors.DeadlockDetected: deadlock detected
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: DETAIL:  Process 119422 waits for AccessExclusiveLock on relation 1620850 of database 585335; blocked by process 2915.
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Process 2915 waits for AccessShareLock on relation 585593 of database 585335; blocked by process 119422.
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: HINT:  See server log for query details.
2023-07-03 10:59:39 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: