Hide Forgot
Description of problem: Warnings in the log when starting/restarting the celery services Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. systemctl restart pulp_workers 2. 3. Actual results: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: /usr/lib/python2.7/site-packages/celery/apps/worker.py:161: CDeprecationWarning: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: Starting from version 3.2 Celery will refuse to accept pickle by default. Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: The pickle serializer is a security concern as it may give attackers Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: the ability to execute any command. It's important to secure Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: your broker from unauthorized access when using pickle, so we think Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: that enabling pickle should require a deliberate action and not be Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: the default choice. Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: If you depend on pickle then you should set a setting to disable this Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: warning and to be sure that everything will continue working Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: when you upgrade to Celery 3.2:: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml'] Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: You must only enable the serializers that you will actually use. Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED)) Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com pulp[28633]: py.warnings:WARNING: Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com celery[28633]: -------------- resource_manager.lab.eng.bos.redhat.com v3.1.11 (Cipater) Oct 09 10:54:43 cloud-qe-15.idmqe.lab.eng.bos.redhat.com celery[28633]: ---- **** ----- Expected results: Additional info:
Here's the message with the log details removed: /usr/lib/python2.7/site-packages/celery/apps/worker.py:161: CDeprecationWarning: Starting from version 3.2 Celery will refuse to accept pickle by default. The pickle serializer is a security concern as it may give attackers the ability to execute any command. It's important to secure your broker from unauthorized access when using pickle, so we think that enabling pickle should require a deliberate action and not be the default choice. If you depend on pickle then you should set a setting to disable this warning and to be sure that everything will continue working when you upgrade to Celery 3.2:: CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml'] You must only enable the serializers that you will actually use. warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED)) -------------- resource_manager.lab.eng.bos.redhat.com v3.1.11 (Cipater) ---- **** -----
I've added a troubleshooting note [0] to the 2.5.0 docs indicating this warning is part of normal operation. [0]: https://github.com/pulp/pulp/pull/1232
Putting this back to NEW because I only documented the bug I did not actually fix it. To fix this bug I think you will need to: - Port over all celery tasks to using arguments that serialize correctly with json - Have Pulp configure celery to disable all pickling except json - test test test and update the unit tests - Decide if the consumer agent executes arbitrary code passed in via the message bus, and if so if we are OK with that. If we are not then we need to start using the json serializer with consumer tasks also.
Moved to https://pulp.plan.io/issues/575