Hit same issue on upgrading 6.2.12 -> 6.3 snap25

Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088) Traceback (most recent call last):
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     R = retval = fun(*args, **kwargs)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     return super(Task, self).__call__(*args, **kwargs)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     return super(PulpTask, self).__call__(*args, **kwargs)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     return self.run(*args, **kwargs)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py", line 107, in regenerate_applicability_for_consumers
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     manager.regenerate_applicability(profile_hash, content_type, profile_id, repo_id)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py", line 272, in regenerate_applicability
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     applicability)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py", line 384, in create
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     applicability.save()
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib/python2.7/site-packages/pulp/server/db/model/consumer.py", line 234, in save
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     self._id = self.get_collection().insert(new_document)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 2200, in insert
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     check_keys, manipulate, write_concern)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 530, in _insert
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     check_keys, manipulate, write_concern, op_id, bypass_doc_val)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 513, in _insert_one
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     _check_write_command_response([(0, result)])
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)   File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 300, in _check_write_command_response
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088)     raise DuplicateKeyError(error.get("errmsg"), 11000, error)
Nov 17 16:00:01 qe-sat6-upgrade-rhel7 pulp: celery.worker.job:ERROR: (4270-61088) DuplicateKeyError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: pulp_database.repo_profile_applicability.$profile_hash_-1_repo_id_-1  dup key: { : "0b583821e55e8f92bb3c7b932740bb02b0819726d7c2dcfb835a4c44b226ed21", : "Default_Organization-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6Server" }

The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.

VERIFIED
on sat6.2.14-1


- the duplicate key error is now handled properly


- prepare 2 consumers (register them to a sat, have some repos with packages and errata)
- possibly tweak the pulp server.managers.consumer.applicability code to add some time.sleep() just after the regenerate_applicability function call that would make your opportunity window wider.
- also add some _logger.info() messages into the `expect DuplicateKeyError` branch in order to make sure, you're replicated the behavior correctly
(you might need to tweak pulp.server.logs to whitelist the logger)

- now send out a package profile for a host1 (that would contain package version, triggering the errata status)
- while you're hooked on the sleep(), trigger package profile upload from the other host

- after the sleep() from the first package profile request expires, pulp should create an Applicability Object.
- after the sleep() from the second package profile request expires, it should follow the same route of creating the applicability object, however it should raise the DuplicateKeyError, due this one already exists. However, the error is properly caught and handled.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0273