Cancellation of node sync tasks is not propagated to the agent. As a result, the task on the parent has a state of canceled but continues to run on the agent/node. This means that Katello things the task was cancelled when it was not. It prevents new tasks to be run by the agent from being run until the previous task finishes normally. This can result in a backlog of agent requests and tasks on the node.
Since this issue was entered in Red Hat Bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release.
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Can I get some suitable text for a rel note for this? thanks
Brad, Can you help with rel note text? Need at least Issue and Workaround, Cause is optional, examples optional (but nice) :) thanks
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.
The Pulp upstream bug status is at VERIFIED. Updating the external tracker on this bug.
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
In trying to cancel a capsule sync in Sat6 (by way of canceling the task, since there is no 'normal' way via UI/cli to do it), I get the following upon cancel: Oct 5 16:37:53 rhsm-qe-3 goferd: [ERROR][worker-0] pulp_node.handlers.model:324 - Task [a321949c-3215-4a9c-a78e-589e16cc5124] cancellation failed http=200 Oct 5 16:37:53 rhsm-qe-3 pulp: celery.worker.control:INFO: Terminating a321949c-3215-4a9c-a78e-589e16cc5124 (15) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) Default_Organization-Library-cv_Capsule-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) Traceback (most recent call last): Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib/python2.7/site-packages/pulp_node/importers/strategies.py", line 130, in synchronize Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) self._synchronize(request) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib/python2.7/site-packages/pulp_node/importers/strategies.py", line 374, in _synchronize Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) unit_inventory = self._unit_inventory(request) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib/python2.7/site-packages/pulp_node/importers/strategies.py", line 218, in _unit_inventory Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) inventory = UnitInventory(base_URL, parent_units, child_units) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib/python2.7/site-packages/pulp_node/importers/inventory.py", line 76, in __init__ Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) self.parent_units = self._import_parent_units(parent_units) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib/python2.7/site-packages/pulp_node/importers/inventory.py", line 52, in _import_parent_units Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) for unit, ref in units: Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib/python2.7/site-packages/pulp_node/manifest.py", line 405, in next Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) return self.unit_generator.next() Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib/python2.7/site-packages/pulp_node/manifest.py", line 387, in get_units Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) unit = json.loads(json_unit) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) return _default_decoder.decode(s) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) obj, end = self.raw_decode(s, idx=_w(s, 0).end()) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib64/python2.7/json/decoder.py", line 381, in raw_decode Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) obj, end = self.scan_once(s, idx) Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 478, in sigterm_handler Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) handler() Oct 5 16:37:53 rhsm-qe-3 pulp: pulp_node.importers.strategies:ERROR: (16370-82688) TypeError: cancel_sync_repo() takes exactly 3 arguments (1 given)
submitted ^ too soon. This is in the logs on the capsule.
The traceback above is a known issue related to cancellation. It has been fixed and has a commit which could be cherry picked also. The known upstream Pulp issue is: https://pulp.plan.io/issues/1097
Verified on 6.1.3 2-Oct compose. Traceback above does not have any net effect on cancellation and can be fixed at a later date.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:1911