Bug 1279466 - Pulp fails to properly cancel tasks after a pulp worker dies. [NEEDINFO]
Pulp fails to properly cancel tasks after a pulp worker dies.
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Pulp (Show other bugs)
6.1.3
Unspecified Unspecified
unspecified Severity medium (vote)
: Beta
: --
Assigned To: satellite6-bugs
Jitendra Yejare
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-11-09 09:22 EST by Stuart Auchterlonie
Modified: 2016-07-27 05:20 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-07-27 05:20:19 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
tkolhar: needinfo? (sauchter)


Attachments (Terms of Use)

  None (edit)
Description Stuart Auchterlonie 2015-11-09 09:22:30 EST
Description of problem:

After a pulp_worker dies, attempts are made to cancel the tasks assigned
to that worker.

However the cancel is called with the wrong number of arguments
resulting in a traceback

Version-Release number of selected component (if applicable):

6.1.3

How reproducible:

Happens most times a pulp worker dies on my system

Steps to Reproduce:
1. Have a pulp worker die when it has work assigned to it
2.
3.

Actual results:

Tasks not correctly cancelled.


Expected results:

Tasks correctly cancelled

Additional info:
Nov 06 21:11:42 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.scheduler:ERROR: Workers 'resource_manager@stuarta-rhel7-sat6.usersys.redhat.com' has gone missing, removing from list of workers
Nov 06 21:11:42 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.tasks:ERROR: The worker named resource_manager@stuarta-rhel7-sat6.usersys.redhat.com is missing. Canceling the tasks in its queue.
Nov 06 21:11:43 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.scheduler:ERROR: Workers 'reserved_resource_worker-1@stuarta-rhel7-sat6.usersys.redhat.com' has gone missing, removing from list of workers
Nov 06 21:11:43 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@stuarta-rhel7-sat6.usersys.redhat.com is missing. Canceling the tasks in its queue.
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.worker_watcher:INFO: New worker 'reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com' discovered

Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224) Exception caught from plugin during publish for repo [org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server]
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224) Traceback (most recent call last):
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 127, in _do_publish
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 495, in wrap_f
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     return f(*args, **kwargs)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/distributors/http/distributor.py", line 127, in publish_repo
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     publisher.publish(units)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/distributors/http/publisher.py", line 51, in publish
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     manifest_path = super(self.__class__, self).publish(units)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/distributors/publisher.py", line 125, in publish
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     for unit in units:
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/conduit.py", line 111, in next
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     return self.unit_generator.next()
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/conduit.py", line 99, in get_units
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     for metadata in cursor:
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 814, in next
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     if len(self.__data) or self._refresh():
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 776, in _refresh
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     limit, self.__id))
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 700, in __send_message
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     **kwargs)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 991, in _send_message_with_response
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     return self.__send_and_receive(message, sock_info)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 969, in __send_and_receive
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     return self.__receive_message_on_socket(1, request_id, sock_info)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 954, in __receive_message_on_socket
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     header = self.__receive_data_on_socket(16, sock
_info)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 942, in __receive_data_on_socket
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     chunk = sock_info.sock.recv(length)
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 478, in sigterm_handler
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224)     handler()
Nov 06 21:11:44 stuarta-rhel7-sat6.usersys.redhat.com pulp[31754]: pulp.server.managers.repo.publish:ERROR: (31754-40224) TypeError: cancel_publish_repo() takes exactly 3 arguments (1 given)
Nov 06 21:11:45 stuarta-rhel7-sat6.usersys.redhat.com pulp[27151]: celery.worker.control:INFO: Terminating 85f0ab4b-ddf4-4562-b7c6-2f0329cec7e0 (15)
Nov 06 21:11:45 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.worker_watcher:INFO: New worker 'resource_manager@stuarta-rhel7-sat6.usersys.redhat.com' discovered
Nov 06 21:11:45 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.tasks:INFO: Task canceled: 85f0ab4b-ddf4-4562-b7c6-2f0329cec7e0.
Nov 06 21:11:45 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[cfed26b1-d7c8-42c0-8a30-af7510a0687f]
Nov 06 21:11:46 stuarta-rhel7-sat6.usersys.redhat.com pulp[27151]: celery.worker.control:INFO: Terminating c2732677-cb90-4791-bc78-c149254b4716 (15)
Nov 06 21:11:46 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.tasks:INFO: Task canceled: c2732677-cb90-4791-bc78-c149254b4716.
Nov 06 21:11:46 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.scheduler:ERROR: Workers 'reserved_resource_worker-0@stuarta-rhel7-sat6.usersys.redhat.com' has gone missing, removing from list of workers
Nov 06 21:11:46 stuarta-rhel7-sat6.usersys.redhat.com pulp[30873]: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@stuarta-rhel7-sat6.usersys.redhat.com is missing. Canceling the tasks in its queue.
Nov 06 21:11:46 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[d4f34a09-a793-4d7b-b598-5d0b2d60fcd9]
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224) Task pulp.server.managers.repo.publish.publish[b2a5e8b1-27ce-4580-9046-0511f384e23c] raised unexpected: TypeError('cancel_publish_repo() takes exactly 3 arguments (1 given)',)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224) Traceback (most recent call last):
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     R = retval = fun(*args, **kwargs)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 328, in __call__
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     return super(Task, self).__call__(*args, **kwargs)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     return self.run(*args, **kwargs)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 99, in publish
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     transfer_repo, conduit, call_config)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 127, in _do_publish
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 495, in wrap_f
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     return f(*args, **kwargs)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/distributors/http/distributor.py", line 127, in publish_repo
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     publisher.publish(units)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/distributors/http/publisher.py", line 51, in publish
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     manifest_path = super(self.__class__, self).publish(units)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/distributors/publisher.py", line 125, in publish
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     for unit in units:
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/conduit.py", line 111, in next
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     return self.unit_generator.next()
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp_node/conduit.py", line 99, in get_units
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     for metadata in cursor:
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 814, in next
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     if len(self.__data) or self._refresh():
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 776, in _refresh
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     limit, self.__id))
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 700, in __send_message
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     **kwargs)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 991, in _send_message_with_response
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     return self.__send_and_receive(message, sock_info)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 969, in __send_and_receive
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     return self.__receive_message_on_socket(1, request_id, sock_info)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 954, in __receive_message_on_socket
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     header = self.__receive_data_on_socket(16, sock_info)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 942, in __receive_data_on_socket
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     chunk = sock_info.sock.recv(length)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 478, in sigterm_handler
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224)     handler()
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:ERROR: (27155-40224) TypeError: cancel_publish_repo() takes exactly 3 arguments (1 given)
Nov 06 21:11:47 stuarta-rhel7-sat6.usersys.redhat.com pulp[27155]: celery.worker.job:INFO: Discarding revoked task: pulp.server.managers.repo.publish.publish[e4170d4b-aa35-4c83-b510-8ce2b89455a7]


-------

Corresponding dynflow error

17: Actions::Katello::Repository::Sync (error) [ 80.49s / 80.49s ]  Skip
Started at: 2015-11-06 21:25:49 UTC

Ended at: 2015-11-06 21:27:09 UTC

Real time: 80.49s

Execution time (excluding suspended state): 80.49s

Input:

---
repository:
  id: 53
  name: Red Hat OpenStack 6.0 for RHEL 7 RPMs x86_64 7Server
  label: Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
product:
  id: 66
  name: Red Hat OpenStack
  label: Red_Hat_OpenStack
  cp_id: '191'
provider:
  id: 4
  name: Red Hat
organization:
  id: 3
  name: org-stuarta
  label: org-stuarta
services_checked:
- pulp
- pulp_auth
- elasticsearch
id: 53
sync_result:
  pulp_tasks:
  - exception: 
    task_type: pulp.server.managers.repo.sync.sync
    _href: /pulp/api/v2/tasks/cbaa0aaa-1a2e-4d18-9bcd-02e3b62d6bbf/
    task_id: cbaa0aaa-1a2e-4d18-9bcd-02e3b62d6bbf
    tags:
    - pulp:repository:org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
    - pulp:action:sync
    finish_time: '2015-11-06T20:45:47Z'
    _ns: task_status
    start_time: '2015-11-06T20:40:07Z'
    traceback: 
    spawned_tasks:
    - _href: /pulp/api/v2/tasks/b2a5e8b1-27ce-4580-9046-0511f384e23c/
      task_id: b2a5e8b1-27ce-4580-9046-0511f384e23c
    - _href: /pulp/api/v2/tasks/e4170d4b-aa35-4c83-b510-8ce2b89455a7/
      task_id: e4170d4b-aa35-4c83-b510-8ce2b89455a7
    progress_report:
      yum_importer:
        content:
          size_total: 0
          items_left: 0
          items_total: 0
          state: FINISHED
          size_left: 0
          details:
            rpm_total: 0
            rpm_done: 0
            drpm_total: 0
            drpm_done: 0
          error_details: []
        comps:
          state: FINISHED
        distribution:
          items_total: 0
          state: FINISHED
          error_details: []
          items_left: 0
        errata:
          state: FINISHED
        metadata:
          state: FINISHED
    queue: reserved_resource_worker-0@stuarta-rhel7-sat6.usersys.redhat.com.dq
    state: finished
    worker_name: reserved_resource_worker-0@stuarta-rhel7-sat6.usersys.redhat.com
    result:
      importer_type_id: yum_importer
      importer_id: yum_importer
      exception: 
      repo_id: org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
      removed_count: 0
      started: '2015-11-06T20:40:09Z'
      _ns: repo_sync_results
      completed: '2015-11-06T20:45:44Z'
      traceback: 
      summary:
        content:
          state: FINISHED
        comps:
          state: FINISHED
        distribution:
          state: FINISHED
        errata:
          state: FINISHED
        metadata:
          state: FINISHED
      added_count: 0
      error_message: 
      updated_count: 77
      details:
        content:
          size_total: 0
          items_left: 0
          items_total: 0
          state: FINISHED
          size_left: 0
          details:
            rpm_total: 0
            rpm_done: 0
            drpm_total: 0
            drpm_done: 0
          error_details: []
        comps:
          state: FINISHED
        distribution:
          items_total: 0
          state: FINISHED
          error_details: []
          items_left: 0
        errata:
          state: FINISHED
        metadata:
          state: FINISHED
      id: 563d11789b3eb57b94f3d88c
      result: success
    error: 
    _id:
      $oid: 563d0738697e0dbab8d2a894
    id: 563d10279b3eb5587b70f8e9
  - exception: 
    task_type: pulp.server.managers.repo.publish.publish
    _href: /pulp/api/v2/tasks/b2a5e8b1-27ce-4580-9046-0511f384e23c/
    task_id: b2a5e8b1-27ce-4580-9046-0511f384e23c
    tags:
    - pulp:repository:org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
    - pulp:action:publish
    finish_time: '2015-11-06T21:11:45Z'
    _ns: task_status
    start_time: '2015-11-06T21:09:58Z'
    traceback: ! "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\",
      line 240, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\",
      line 328, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n
      \ File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in
      __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py\",
      line 99, in publish\n    transfer_repo, conduit, call_config)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py\",
      line 127, in _do_publish\n    publish_report = publish_repo(transfer_repo, conduit,
      call_config)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\",
      line 495, in wrap_f\n    return f(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp_node/distributors/http/distributor.py\",
      line 127, in publish_repo\n    publisher.publish(units)\n  File \"/usr/lib/python2.7/site-packages/pulp_node/distributors/http/publisher.py\",
      line 51, in publish\n    manifest_path = super(self.__class__, self).publish(units)\n
      \ File \"/usr/lib/python2.7/site-packages/pulp_node/distributors/publisher.py\",
      line 125, in publish\n    for unit in units:\n  File \"/usr/lib/python2.7/site-packages/pulp_node/conduit.py\",
      line 111, in next\n    return self.unit_generator.next()\n  File \"/usr/lib/python2.7/site-packages/pulp_node/conduit.py\",
      line 99, in get_units\n    for metadata in cursor:\n  File \"/usr/lib64/python2.7/site-packages/pymongo/cursor.py\",
      line 814, in next\n    if len(self.__data) or self._refresh():\n  File \"/usr/lib64/python2.7/site-packages/pymongo/cursor.py\",
      line 776, in _refresh\n    limit, self.__id))\n  File \"/usr/lib64/python2.7/site-packages/pymongo/cursor.py\",
      line 700, in __send_message\n    **kwargs)\n  File \"/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py\",
      line 991, in _send_message_with_response\n    return self.__send_and_receive(message,
      sock_info)\n  File \"/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py\",
      line 969, in __send_and_receive\n    return self.__receive_message_on_socket(1,
      request_id, sock_info)\n  File \"/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py\",
      line 954, in __receive_message_on_socket\n    header = self.__receive_data_on_socket(16,
      sock_info)\n  File \"/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py\",
      line 942, in __receive_data_on_socket\n    chunk = sock_info.sock.recv(length)\n
      \ File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line
      478, in sigterm_handler\n    handler()\nTypeError: cancel_publish_repo() takes
      exactly 3 arguments (1 given)\n"
    spawned_tasks: []
    progress_report: {}
    queue: reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com.dq
    state: error
    worker_name: reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com
    result: 
    error:
      code: PLP0000
      data: {}
      description: cancel_publish_repo() takes exactly 3 arguments (1 given)
      sub_errors: []
    _id:
      $oid: 563d117a697e0dbab8d2a8b0
    id: 563d171c9b3eb5587b717c0d
  - exception: 
    task_type: pulp.server.managers.repo.publish.publish
    _href: /pulp/api/v2/tasks/e4170d4b-aa35-4c83-b510-8ce2b89455a7/
    task_id: e4170d4b-aa35-4c83-b510-8ce2b89455a7
    tags:
    - pulp:repository:org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
    - pulp:action:publish
    finish_time: 
    start_time: 
    traceback: 
    spawned_tasks: []
    progress_report: {}
    queue: reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com.dq
    state: canceled
    worker_name: reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com
    result: 
    error: 
    _id:
      $oid: 563d117a697e0dbab8d2a8b1
    id: 563d17219b3eb5587b717c11
  poll_attempts:
    total: 75
    failed: 2
user_id: 1
current_user_id: 1
locale: en
Output:

---
sync_result:
  pulp_tasks:
  - exception: 
    task_type: pulp.server.managers.repo.sync.sync
    _href: /pulp/api/v2/tasks/cbaa0aaa-1a2e-4d18-9bcd-02e3b62d6bbf/
    task_id: cbaa0aaa-1a2e-4d18-9bcd-02e3b62d6bbf
    tags:
    - pulp:repository:org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
    - pulp:action:sync
    finish_time: '2015-11-06T20:45:47Z'
    _ns: task_status
    start_time: '2015-11-06T20:40:07Z'
    traceback: 
    spawned_tasks:
    - _href: /pulp/api/v2/tasks/b2a5e8b1-27ce-4580-9046-0511f384e23c/
      task_id: b2a5e8b1-27ce-4580-9046-0511f384e23c
    - _href: /pulp/api/v2/tasks/e4170d4b-aa35-4c83-b510-8ce2b89455a7/
      task_id: e4170d4b-aa35-4c83-b510-8ce2b89455a7
    progress_report:
      yum_importer:
        content:
          size_total: 0
          items_left: 0
          items_total: 0
          state: FINISHED
          size_left: 0
          details:
            rpm_total: 0
            rpm_done: 0
            drpm_total: 0
            drpm_done: 0
          error_details: []
        comps:
          state: FINISHED
        distribution:
          items_total: 0
          state: FINISHED
          error_details: []
          items_left: 0
        errata:
          state: FINISHED
        metadata:
          state: FINISHED
    queue: reserved_resource_worker-0@stuarta-rhel7-sat6.usersys.redhat.com.dq
    state: finished
    worker_name: reserved_resource_worker-0@stuarta-rhel7-sat6.usersys.redhat.com
    result:
      importer_type_id: yum_importer
      importer_id: yum_importer
      exception: 
      repo_id: org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
      removed_count: 0
      started: '2015-11-06T20:40:09Z'
      _ns: repo_sync_results
      completed: '2015-11-06T20:45:44Z'
      traceback: 
      summary:
        content:
          state: FINISHED
        comps:
          state: FINISHED
        distribution:
          state: FINISHED
        errata:
          state: FINISHED
        metadata:
          state: FINISHED
      added_count: 0
      error_message: 
      updated_count: 77
      details:
        content:
          size_total: 0
          items_left: 0
          items_total: 0
          state: FINISHED
          size_left: 0
          details:
            rpm_total: 0
            rpm_done: 0
            drpm_total: 0
            drpm_done: 0
          error_details: []
        comps:
          state: FINISHED
        distribution:
          items_total: 0
          state: FINISHED
          error_details: []
          items_left: 0
        errata:
          state: FINISHED
        metadata:
          state: FINISHED
      id: 563d11789b3eb57b94f3d88c
      result: success
    error: 
    _id:
      $oid: 563d0738697e0dbab8d2a894
    id: 563d10279b3eb5587b70f8e9
  - exception: 
    task_type: pulp.server.managers.repo.publish.publish
    _href: /pulp/api/v2/tasks/b2a5e8b1-27ce-4580-9046-0511f384e23c/
    task_id: b2a5e8b1-27ce-4580-9046-0511f384e23c
    tags:
    - pulp:repository:org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
    - pulp:action:publish
    finish_time: '2015-11-06T21:11:45Z'
    _ns: task_status
    start_time: '2015-11-06T21:09:58Z'
    traceback: ! "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\",
      line 240, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\",
      line 328, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n
      \ File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in
      __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py\",
      line 99, in publish\n    transfer_repo, conduit, call_config)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py\",
      line 127, in _do_publish\n    publish_report = publish_repo(transfer_repo, conduit,
      call_config)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\",
      line 495, in wrap_f\n    return f(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp_node/distributors/http/distributor.py\",
      line 127, in publish_repo\n    publisher.publish(units)\n  File \"/usr/lib/python2.7/site-packages/pulp_node/distributors/http/publisher.py\",
      line 51, in publish\n    manifest_path = super(self.__class__, self).publish(units)\n
      \ File \"/usr/lib/python2.7/site-packages/pulp_node/distributors/publisher.py\",
      line 125, in publish\n    for unit in units:\n  File \"/usr/lib/python2.7/site-packages/pulp_node/conduit.py\",
      line 111, in next\n    return self.unit_generator.next()\n  File \"/usr/lib/python2.7/site-packages/pulp_node/conduit.py\",
      line 99, in get_units\n    for metadata in cursor:\n  File \"/usr/lib64/python2.7/site-packages/pymongo/cursor.py\",
      line 814, in next\n    if len(self.__data) or self._refresh():\n  File \"/usr/lib64/python2.7/site-packages/pymongo/cursor.py\",
      line 776, in _refresh\n    limit, self.__id))\n  File \"/usr/lib64/python2.7/site-packages/pymongo/cursor.py\",
      line 700, in __send_message\n    **kwargs)\n  File \"/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py\",
      line 991, in _send_message_with_response\n    return self.__send_and_receive(message,
      sock_info)\n  File \"/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py\",
      line 969, in __send_and_receive\n    return self.__receive_message_on_socket(1,
      request_id, sock_info)\n  File \"/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py\",
      line 954, in __receive_message_on_socket\n    header = self.__receive_data_on_socket(16,
      sock_info)\n  File \"/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py\",
      line 942, in __receive_data_on_socket\n    chunk = sock_info.sock.recv(length)\n
      \ File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line
      478, in sigterm_handler\n    handler()\nTypeError: cancel_publish_repo() takes
      exactly 3 arguments (1 given)\n"
    spawned_tasks: []
    progress_report: {}
    queue: reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com.dq
    state: error
    worker_name: reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com
    result: 
    error:
      code: PLP0000
      data: {}
      description: cancel_publish_repo() takes exactly 3 arguments (1 given)
      sub_errors: []
    _id:
      $oid: 563d117a697e0dbab8d2a8b0
    id: 563d171c9b3eb5587b717c0d
  - exception: 
    task_type: pulp.server.managers.repo.publish.publish
    _href: /pulp/api/v2/tasks/e4170d4b-aa35-4c83-b510-8ce2b89455a7/
    task_id: e4170d4b-aa35-4c83-b510-8ce2b89455a7
    tags:
    - pulp:repository:org-stuarta-Red_Hat_OpenStack-Red_Hat_OpenStack_6_0_for_RHEL_7_RPMs_x86_64_7Server
    - pulp:action:publish
    finish_time: 
    start_time: 
    traceback: 
    spawned_tasks: []
    progress_report: {}
    queue: reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com.dq
    state: canceled
    worker_name: reserved_resource_worker-3@stuarta-rhel7-sat6.usersys.redhat.com
    result: 
    error: 
    _id:
      $oid: 563d117a697e0dbab8d2a8b1
    id: 563d17219b3eb5587b717c11
  poll_attempts:
    total: 75
    failed: 2
Error:

RuntimeError

There was an issue with the backend service pulp: Request Timeout

---
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/backend_services_check.rb:16:in
  `block in plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/backend_services_check.rb:14:in
  `each'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/backend_services_check.rb:14:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/remote_action.rb:20:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:29:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/keep_locale.rb:19:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/world.rb:30:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:441:in
  `block (2 levels) in execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:234:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:234:in
  `switch_flow'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:347:in
  `concurrence'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:440:in
  `block in execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `block in with_error_handling'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `catch'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `with_error_handling'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:439:in
  `execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:262:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/plan_step.rb:46:in
  `block in execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `with_meta_calculation'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/plan_step.rb:45:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:375:in
  `plan_action'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/katello/repository/node_metadata_generate.rb:19:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:442:in
  `block (3 levels) in execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:29:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/keep_locale.rb:19:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/world.rb:30:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:441:in
  `block (2 levels) in execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:234:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:234:in
  `switch_flow'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:347:in
  `concurrence'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:440:in
  `block in execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `block in with_error_handling'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `catch'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `with_error_handling'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:439:in
  `execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:262:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/plan_step.rb:46:in
  `block in execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `with_meta_calculation'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/plan_step.rb:45:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:375:in
  `plan_action'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/katello/repository/capsule_generate_and_sync.rb:23:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:442:in
  `block (3 levels) in execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:29:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/keep_locale.rb:19:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/world.rb:30:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:441:in
  `block (2 levels) in execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:234:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:234:in
  `switch_flow'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:347:in
  `concurrence'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:440:in
  `block in execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `block in with_error_handling'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `catch'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `with_error_handling'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:439:in
  `execute_plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:262:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/plan_step.rb:46:in
  `block in execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `with_meta_calculation'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/plan_step.rb:45:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:166:in
  `block (3 levels) in plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:234:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:234:in
  `switch_flow'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:224:in
  `with_planning_scope'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:165:in
  `block (2 levels) in plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:37:in
  `plan_phase'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:37:in
  `plan_phase'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/world.rb:30:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:164:in
  `block in plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in
  `transaction'
- /opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:208:in
  `transaction'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/transaction_adapters/active_record.rb:5:in
  `transaction'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan.rb:163:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/world.rb:129:in
  `block in plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/world.rb:127:in
  `tap'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/world.rb:127:in
  `plan'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/world.rb:100:in
  `trigger'
- /opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.15.7/lib/foreman_tasks.rb:17:in
  `trigger'
- /opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.15.7/lib/foreman_tasks.rb:23:in
  `trigger_task'
- /opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.15.7/lib/foreman_tasks.rb:35:in
  `async_task'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/katello/repository/sync.rb:56:in
  `run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:487:in
  `block (3 levels) in execute_run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.15.7/app/lib/actions/middleware/keep_current_user.rb:23:in
  `block in run'
- /opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.15.7/app/lib/actions/middleware/keep_current_user.rb:34:in
  `restore_curent_user'
- /opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.15.7/app/lib/actions/middleware/keep_current_user.rb:23:in
  `run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action/progress.rb:30:in
  `with_progress_calculation'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action/progress.rb:16:in
  `run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in
  `pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/keep_locale.rb:23:in
  `block in run'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/keep_locale.rb:34:in
  `with_locale'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.73/app/lib/actions/middleware/keep_locale.rb:23:in
  `run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/world.rb:30:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:486:in
  `block (2 levels) in execute_run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:485:in
  `catch'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:485:in
  `block in execute_run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `block in with_error_handling'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `catch'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in
  `with_error_handling'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:480:in
  `execute_run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:262:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in
  `block (2 levels) in execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in
  `with_meta_calculation'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in
  `block in execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in
  `open_action'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/executors/parallel/worker.rb:20:in
  `block in on_message'
- /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in
  `block in assigns'
- /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in
  `tap'
- /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in
  `assigns'
- /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in
  `match_value'
- /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in
  `block in match'
- /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in
  `each'
- /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in
  `match'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/executors/parallel/worker.rb:17:in
  `on_message'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:82:in
  `on_envelope'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:72:in
  `receive'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in
  `block (2 levels) in run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in
  `loop'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in
  `block in run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in
  `catch'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in
  `run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:13:in
  `block in initialize'
- /opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in
  `block in create_with_logging_context'
Comment 3 Michael Hrivnak 2016-01-04 13:02:26 EST
This appears to be fixed in pulp 2.8, and probably earlier than that. I can chase down the exact fix if desired, or we can just call it fixed in 2.8.
Comment 8 Tazim Kolhar 2016-04-05 02:21:29 EDT
Hi, 

    Please, guide me here for the steps:
    
    1.)Create a product
    2.) Add a Repository
         Is it specific to
         (Red Hat OpenStack 6.0 for RHEL 7 RPMs x86_64 7Server) ?

    3.) Sync it
    4.) Stop the pulp_celerybeat.service
    5.) Check /var/log/messages 

Thanks and Regards,
Tazim
Comment 10 Jitendra Yejare 2016-04-27 03:01:09 EDT
Verified !
@ Sat 6.2 Snap 9.2

After pulp worker dies the task has been canceled without throwing any errors as mentioned in Description. The proper info message of pulp workers shutdown is displayed in /var/log/messages as below:

Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Celery Workers...
Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #4...
Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #2...
Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #6...
Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #5...
Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #3...
Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #1...
Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #7...
Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #0...
Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess)
Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess)
Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess)
Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess)
Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess)
Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess)
Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess)
Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess)
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-1@qeblade36.rhq.lab.eng.bos.redhat.com' shutdown
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-0@qeblade36.rhq.lab.eng.bos.redhat.com' shutdown
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-7@qeblade36.rhq.lab.eng.bos.redhat.com' shutdown
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-5@qeblade36.rhq.lab.eng.bos.redhat.com' shutdown
Apr 27 02:53:05 qeblade36 pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.tasks:INFO: Task canceled: 0fba90b2-eabb-446e-ade7-30da9f433dce.
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-2@qeblade36.rhq.lab.eng.bos.redhat.com' shutdown
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-4@qeblade36.rhq.lab.eng.bos.redhat.com' shutdown
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-6@qeblade36.rhq.lab.eng.bos.redhat.com' shutdown
Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-3@qeblade36.rhq.lab.eng.bos.redhat.com' shutdown
Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-4@qeblade36.rhq.lab.eng.bos.redhat.com ready.
Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-6@qeblade36.rhq.lab.eng.bos.redhat.com ready.
Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-3@qeblade36.rhq.lab.eng.bos.redhat.com ready.
Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-1@qeblade36.rhq.lab.eng.bos.redhat.com ready.
Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-7@qeblade36.rhq.lab.eng.bos.redhat.com ready.
Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-0@qeblade36.rhq.lab.eng.bos.redhat.com ready.
Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-2@qeblade36.rhq.lab.eng.bos.redhat.com ready.
Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #7.
Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #3.
Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #6.
Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #0.
Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #4.
Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #2.
Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #1.
Apr 27 02:53:06 qeblade36 celery: reserved_resource_worker-5@qeblade36.rhq.lab.eng.bos.redhat.com ready.
Apr 27 02:53:07 qeblade36 systemd: Stopped Pulp Worker #5.
Apr 27 02:53:07 qeblade36 systemd: Stopped Pulp Celery Workers.


So moving this Bz to verified.
Comment 12 errata-xmlrpc 2016-07-27 05:20:19 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501

Note You need to log in before you can comment on or make changes to this bug.