Bug 1029067

Summary: python-celery has a runtime requirement on python-billiard < 3.0, but the spec file does not require any particular version of billiard
Product: [Fedora] Fedora Reporter: Randy Barlow <rbarlow>
Component: python-celeryAssignee: Matthias Runge <mrunge>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 19CC: andrew, beland, mhrivnak, mrunge, rbarlow, zkatsuragi
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: python-kombu-2.5.16-1.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-17 05:45:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Randy Barlow 2013-11-11 15:29:50 UTC
Description of problem:
I installed python-celery, but I cannot start celeryd because it has a runtime dependency on python-billiard, ensuring that <3.0 is installed. The Celery spec file should explicitly require billiard < 3.0, so as to avoid this problem.

Version-Release number of selected component (if applicable):
$ rpm -qa | grep cel
python-celery-3.0.15-3.fc19.noarch
$ rpm -qa | grep bill
python-billiard-3.3.0.0-1.fc19.x86_64

How reproducible:
Very

Steps to Reproduce:
1. yum install python-celery
2. celeryd

Actual results:
$ celeryd
Traceback (most recent call last):
  File "/usr/bin/celeryd", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2809, in <module>
    parse_requirements(__requires__), Environment()
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 594, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: billiard>=2.7.3.21,<3.0

Expected results:
Installing Celery should require billiard < 3.0 explicitly, instead of requiring any version of billiard.

Comment 1 Christopher Beland 2013-11-18 20:22:56 UTC
Also present in Fedora 20 beta (python-celery-3.0.19-4.fc20.noarch).

Comment 2 Matthias Runge 2013-11-21 10:04:19 UTC
Randy or Christopher,

sadly, I don't have currently a real testcase. May I ask you to try the following RPM?
http://www.matthias-runge.de/fedora/python-celery-3.0.19-5.fc20.noarch.rpm
(basically, I nuked the runtime-checks, since deps are handled via rpm)

SRPM is here:
http://www.matthias-runge.de/fedora/python-celery-3.0.19-5.fc20.src.rpm

If that works, I'd push that to f19, f20 to solve this annoying issue, which I fixed for f21.

Comment 3 Randy Barlow 2013-11-21 15:26:38 UTC
I don't have an F20 installation at the moment to try that package out there, and it appears that I don't have a new enough version of Kombu in F19:

Error: Package: python-celery-3.0.19-5.fc20.noarch (/python-celery-3.0.19-5.fc20.noarch)
           Requires: python-kombu >= 2.5.10
           Installed: python-kombu-2.5.8-1.fc19.noarch (@fedora)
               python-kombu = 2.5.8-1.fc19

I didn't take a look in the src RPM, but I am curious whether you added the <3.0 requirement to the src RPM? I believe that Celery developers intentionally put the check on billiards being under version 3, and so eliminating the check may lead to problems with Celery.

Respectfully, I would like to suggest that we should not get rid of the runtime-checks that the Celery developers put into place, as those checks helped us realize our library compatibility problems.

Thank you!

Comment 4 Michael Hrivnak 2013-11-21 15:38:45 UTC
I'd like to second Randy's suggestion that the runtime check not be removed, unless we have confirmation from the upstream project that celery will work with a newer version of billiard. If we do have such confirmation I would like to see a revised Requires declaration on the RPM that expresses what versions of billiard have been certified to work with this version of celery.

One way or another, we need to have confidence that 1) the versions of celery and billiard in F19 will definitely work together, and 2) that if celery does limit the versions we can use, that the limitation is enforced (preferably by the RPM).

Comment 5 Randy Barlow 2013-11-21 15:48:55 UTC
I had forgotten that I could just query the RPM for the answer to my question above. It looks like the RPM does not have the requirement on billiard<3.0:

$ rpm -qR -p python-celery-3.0.19-5.fc20.noarch.rpm
/usr/bin/python
pyparsing
python(abi) = 2.7
python-amqp
python-anyjson
python-billiard
python-dateutil
python-kombu >= 2.5.10
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(PartialHardlinkSets) <= 4.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsXz) <= 5.2-1

Comment 6 Fedora Update System 2014-01-08 11:18:15 UTC
python-kombu-2.5.16-1.fc20,python-celery-3.0.19-5.fc20,python-billiard-2.7.3.34-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/python-kombu-2.5.16-1.fc20,python-celery-3.0.19-5.fc20,python-billiard-2.7.3.34-1.fc20

Comment 7 Fedora Update System 2014-01-08 11:57:28 UTC
python-kombu-2.5.16-1.fc19,python-celery-3.0.15-4.fc19,python-billiard-2.7.3.34-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-kombu-2.5.16-1.fc19,python-celery-3.0.15-4.fc19,python-billiard-2.7.3.34-1.fc19

Comment 8 Fedora Update System 2014-01-09 07:37:02 UTC
Package python-kombu-2.5.16-1.fc20, python-celery-3.0.19-5.fc20, python-billiard-2.7.3.34-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-kombu-2.5.16-1.fc20 python-celery-3.0.19-5.fc20 python-billiard-2.7.3.34-1.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-0435/python-kombu-2.5.16-1.fc20,python-celery-3.0.19-5.fc20,python-billiard-2.7.3.34-1.fc20
then log in and leave karma (feedback).

Comment 9 Randy Barlow 2014-01-09 19:48:00 UTC
I can also confirm that the Celery RPM now has the correct requires:

$ rpm -q --requires python-celery | grep billiard
python-billiard >= 1:2.7.3.34
python-billiard < 1:3.0

Looks great, thanks for the fix!

Comment 10 Fedora Update System 2014-01-17 05:45:40 UTC
python-kombu-2.5.16-1.fc19, python-celery-3.0.15-4.fc19, python-billiard-2.7.3.34-1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 11 Fedora Update System 2014-01-17 05:48:55 UTC
python-kombu-2.5.16-1.fc20, python-celery-3.0.19-5.fc20, python-billiard-2.7.3.34-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 12 zkatsuragi 2014-01-17 20:24:21 UTC
I tried in fedora 19 today and still have the issue installing celery
# yum install python-celery
---> Package python-celery.noarch 0:3.0.15-4.fc19 will be installed
...
--> Finished Dependency Resolution
Error: Package: python-celery-3.0.15-4.fc19.noarch (updates)
           Requires: python-billiard > 1:2.7.3.34
           Installed: 1:python-billiard-2.7.3.34-1.fc19.x86_64 (@updates)
               python-billiard = 1:2.7.3.34-1.fc19
           Available: python-billiard-2.7.3.23-1.fc19.x86_64 (fedora)
               python-billiard = 2.7.3.23-1.fc19

# rpm -qa | grep billiard
python-billiard-2.7.3.34-1.fc19.x86_64