Bug 2019954 - Auto requirements explode on 'pytz>dev'
Summary: Auto requirements explode on 'pytz>dev'
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-rpm-generators
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-11-03 17:10 UTC by František Zatloukal
Modified: 2022-01-04 09:53 UTC (History)
9 users (show)

Fixed In Version: python-rpm-generators-12-11.fc36
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-01-04 09:53:10 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description František Zatloukal 2021-11-03 17:10:59 UTC
Description of problem:
Since python-rpm-generators-12-10.fc36 , I am unable to build python-celery because auto requirements generator explode on 'pytz>dev'. 

The same package builds just fine with python-rpm-generators-12-8.fc35.

Scratch build (with failure): https://koji.fedoraproject.org/koji/taskinfo?taskID=78282554

The requirement is from this file: https://github.com/celery/celery/blob/master/requirements/default.txt

Pulled in via setup.py by: https://github.com/celery/celery/blob/master/setup.py#L117

Relevant log:

Traceback (most recent call last):
  File "/usr/lib/rpm/pythondistdeps.py", line 259, in convert
    return OPERATORS[operator](name, operator, version_id)
  File "/usr/lib/rpm/pythondistdeps.py", line 242, in convert_ordered
    if operator == '>' and not version.pre and not version.dev and not version.post:
AttributeError: 'RpmVersion' object has no attribute 'pre'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/rpm/pythondistdeps.py", line 579, in <module>
    main()
  File "/usr/lib/rpm/pythondistdeps.py", line 564, in main
    spec_list.append(convert(name, spec[0], spec[1]))
  File "/usr/lib/rpm/pythondistdeps.py", line 261, in convert
    raise RuntimeError("Cannot process Python package version `{}` for name `{}`".
RuntimeError: Cannot process Python package version `dev` for name `python3.10dist(pytz)`

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/rpm/pythondistdeps.py", line 582, in <module>
    raise RuntimeError("Error: pythondistdeps.py generator encountered an unhandled exception and was terminated.") from exc
RuntimeError: Error: pythondistdeps.py generator encountered an unhandled exception and was terminated.
error: Dependency tokens must begin with alpha-numeric, '_' or '/': *** PYTHONDISTDEPS_GENERATORS_FAILED ***
Provides: python-celery = 5.2.0~rc2-1.fc36 python3-celery = 5.2.0~rc2-1.fc36 python3.10-celery = 5.2.0~rc2-1.fc36 python3.10dist(celery) = 5.2~rc2 python3dist(celery) = 5.2~rc2
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PartialHardlinkSets) <= 4.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires: (python3.10dist(billiard) < 4~~ with python3.10dist(billiard) >= 3.6.4) (python3.10dist(click) < 9~~ with python3.10dist(click) >= 8) (python3.10dist(kombu) < 6~~ with python3.10dist(kombu) >= 5.2~rc1) /usr/bin/python3 python(abi) = 3.10 python3.10dist(click-didyoumean) >= 0.0.3 python3.10dist(click-plugins) >= 1.1.1 python3.10dist(click-repl) >= 0.2

Comment 1 Miro Hrončok 2021-11-03 19:26:26 UTC
Thanks for the report.

Previously, this used to generate:

python3.10dist(pytz) > dev

Which seems to be a wrong result.  The requirement seems to behave like 'pytz>0.dev0', but it is parsed as LegacyVersion by packaging (which means "this version has a bogus value").


So this update just likely exposed this problem.


To restore the previous (somewhat working, but likely very wrong) behavior, we could define .pre .dev. and.post as None for legacy versions, but I believe that erroring might be a better choice, also considering that packaging warns to drop LegacyVersion altogether.


For the celery case, maybe let's try to figure out what do they actually want to achieve by pytz>dev? The commit https://github.com/celery/celery/commit/408835a3523a35d64a5a992fe00043c59f68df05 is from 2013. From the linked issue https://github.com/pypa/pip/issues/974 I believe they actually really want to say 'pytz>0.dev.0' but were told to use pytz>dev which behaves the same with pip, but is actually not a valid version specifier :/

Could they be told to use pytz>0.dev.0 instead?


>>> from packaging.version import parse
>>> parse('dev')
<LegacyVersion('dev')>
>>> parse('0.dev.0')
<Version('0.dev0')>


As a workaround, you should be able to sed/patch that to plain "pytz" when building RPM, the relevant issue with beta versions does not apply to RPM requirements.

Comment 2 Miro Hrončok 2021-11-03 19:45:48 UTC
In the meantime, I will rebuild everything in https://copr.fedorainfracloud.org/coprs/churchyard/generators-mass-rebuild/ to see how many packages are impacted by this and in https://copr.fedorainfracloud.org/coprs/churchyard/generators-mass-rebuild-fail-legacy/ I rebuild everything but with the generator that errors out immediately when a LegacyVersion is parsed.

Comment 3 Miro Hrončok 2021-11-04 10:24:13 UTC
We can deal with LegacyVersions later. Here is a proposed fix of this regression: https://src.fedoraproject.org/rpms/python-rpm-generators/pull-request/47

I've tested it on python-celery (with your workaround removed) and it produces the same results as before.


But will run all the builds in https://copr.fedorainfracloud.org/coprs/churchyard/generators-mass-rebuild-fix-regression-legacy anyway.

Comment 4 Miro Hrončok 2021-11-04 11:27:31 UTC
Current rawhide generatiors:

https://copr.fedorainfracloud.org/coprs/churchyard/generators-mass-rebuild/

0 failures with INVALID_REQUIREMENT_ERROR___SEE_STDERR
1 failure with PYTHONDISTDEPS_GENERATORS_FAILED (python-celery)


So the good news is this regression only impacts one package \o/





Generators failing with LegacyVersions:

https://copr.fedorainfracloud.org/coprs/churchyard/generators-mass-rebuild-fail-legacy/

0 failures with PYTHONDISTDEPS_GENERATORS_FAILED

5 failures with INVALID_REQUIREMENT_ERROR___SEE_STDERR

2 errors when processing requires:

python-celery -- Invalid version: dev
python-pvc  -- Invalid version: 5.5.0-2014.1.1

/usr/lib/python3.10/site-packages/celery-5.2.0rc1-py3.10.egg-info/requires.txt
pytz>dev

/usr/lib/python3.10/site-packages/pvc-0.3.0-py3.10.egg-info/requires.txt 
pyvmomi>=5.5.0-2014.1.1


3 errors when processing provides:

python-haversion -- Invalid version: main
python-ipmi -- Invalid version: unknown
python-lacrosse -- Invalid version: unknown

/usr/lib/python3.10/site-packages/pyhaversion-main-py3.10.egg-info/PKG-INFO
Version: main

/usr/lib/python3.10/site-packages/python_ipmi-unknown-py3.10.egg-info/PKG-INFO
Version: unknown

/usr/lib/python3.10/site-packages/pylacrosse-unknown-py3.10.egg-info/PKG-INFO
Version: unknown


Given the fact that some packages are regularly versioned with weird stuff, I am reluctant to error out on LegacyVersion right now, but definitively a good idea to monitor pypa/packaging closely for the removal.


I believe the main/unknown stuff is a bug in the 3 packages.

Maybe we should error out for provides by default and provide a way to force it (for versions like 5.5.0-2014.1.1). I'll start a discussion on python-devel.

Comment 6 František Zatloukal 2021-11-10 13:55:49 UTC
(In reply to Miro Hrončok from comment #1)
> Could they be told to use pytz>0.dev.0 instead?

Asked upstream: https://github.com/celery/celery/pull/7074

Thanks for the explanation!

Comment 7 Miro Hrončok 2022-01-04 09:53:10 UTC
https://src.fedoraproject.org/rpms/python-rpm-generators/pull-request/49

This has been fixed on Rawhide. Given the low impact of this, I will not actively backport to older branches until asked to.

----


Removing support for LegacyVersions is on my TODO, but it can happen after packaging does that.


Note You need to log in before you can comment on or make changes to this bug.