Bug 1279466
Summary: | Pulp fails to properly cancel tasks after a pulp worker dies. | ||
---|---|---|---|
Product: | Red Hat Satellite | Reporter: | Stuart Auchterlonie <sauchter> |
Component: | Pulp | Assignee: | satellite6-bugs <satellite6-bugs> |
Status: | CLOSED ERRATA | QA Contact: | Jitendra Yejare <jyejare> |
Severity: | medium | Docs Contact: | |
Priority: | unspecified | ||
Version: | 6.1.3 | CC: | bbuckingham, bkearney, cwelton, mhrivnak, sauchter, sthirugn |
Target Milestone: | Unspecified | Keywords: | Triaged |
Target Release: | Unused | Flags: | sauchter:
needinfo-
|
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2016-07-27 09:20:19 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Stuart Auchterlonie
2015-11-09 14:22:30 UTC
This appears to be fixed in pulp 2.8, and probably earlier than that. I can chase down the exact fix if desired, or we can just call it fixed in 2.8. Hi, Please, guide me here for the steps: 1.)Create a product 2.) Add a Repository Is it specific to (Red Hat OpenStack 6.0 for RHEL 7 RPMs x86_64 7Server) ? 3.) Sync it 4.) Stop the pulp_celerybeat.service 5.) Check /var/log/messages Thanks and Regards, Tazim Verified ! @ Sat 6.2 Snap 9.2 After pulp worker dies the task has been canceled without throwing any errors as mentioned in Description. The proper info message of pulp workers shutdown is displayed in /var/log/messages as below: Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Celery Workers... Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #4... Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #2... Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #6... Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #5... Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #3... Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #1... Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #7... Apr 27 02:53:05 qeblade36 systemd: Stopping Pulp Worker #0... Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess) Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess) Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess) Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess) Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess) Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess) Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess) Apr 27 02:53:05 qeblade36 celery: worker: Cold shutdown (MainProcess) Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-1.lab.eng.bos.redhat.com' shutdown Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-0.lab.eng.bos.redhat.com' shutdown Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-7.lab.eng.bos.redhat.com' shutdown Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-5.lab.eng.bos.redhat.com' shutdown Apr 27 02:53:05 qeblade36 pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.tasks:INFO: Task canceled: 0fba90b2-eabb-446e-ade7-30da9f433dce. Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-2.lab.eng.bos.redhat.com' shutdown Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-4.lab.eng.bos.redhat.com' shutdown Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-6.lab.eng.bos.redhat.com' shutdown Apr 27 02:53:05 qeblade36 pulp: pulp.server.async.worker_watcher:INFO: Worker 'reserved_resource_worker-3.lab.eng.bos.redhat.com' shutdown Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-4.lab.eng.bos.redhat.com ready. Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-6.lab.eng.bos.redhat.com ready. Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-3.lab.eng.bos.redhat.com ready. Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-1.lab.eng.bos.redhat.com ready. Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-7.lab.eng.bos.redhat.com ready. Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-0.lab.eng.bos.redhat.com ready. Apr 27 02:53:05 qeblade36 celery: reserved_resource_worker-2.lab.eng.bos.redhat.com ready. Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #7. Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #3. Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #6. Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #0. Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #4. Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #2. Apr 27 02:53:06 qeblade36 systemd: Stopped Pulp Worker #1. Apr 27 02:53:06 qeblade36 celery: reserved_resource_worker-5.lab.eng.bos.redhat.com ready. Apr 27 02:53:07 qeblade36 systemd: Stopped Pulp Worker #5. Apr 27 02:53:07 qeblade36 systemd: Stopped Pulp Celery Workers. So moving this Bz to verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501 |