Description of problem:
The python3-astroid package is missing dependencies which were auto-generated properly till next release. Precisely our tests are failing thanks to the missing wrapt package which is a more than a year old dependency.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. mock -r fedora-rawhide-x86_64 --init
2. mock -r fedora-rawhide-x86_64 -i dnf
3. mock -r fedora-rawhide-x86_64 --shell
4. dnf repoquery --requires python3-astroid
4. dnf repoquery install python3-astroid
Only python3-six is an install dependency.
Should be python3-six, python3-wrapt and python3-lazy-object-proxy
The auto-generation was enabled here:
What bug are you reporting? Is it that the automatic dependencies generator is broken (python-rpm-generators) or that astroid is missing dependencies?
Let me check what happens here.
The requires.txt went from:
The asterisks are valid according to PEP 440, but easy to overlook -- I wouldn't be surprised if python-rpm-generators had a bug in handling them:
install_requires = [
'typed-ast>=1.4.0,<1.5;implementation_name== "cpython" and python_version<"3.8"',
It only requires:
python3.8dist(six) = 1.12
I think I know what the problem is. The 1.11.* versions with * is not handled properly.
Note that in the first place, we should probably fallback to generating at least unversioned dependency when we cannot parse/convert the version.
error: Illegal char '*' (0x2a) in: 1.4.*
error: Illegal char '*' (0x2a) in: 1.11.*
And ignores the dependency.
IMHO such error should stop the build, I've opened: https://github.com/rpm-software-management/rpm/issues/881
As an immediate workaround, either disable the dependency generator or patch the version in setup.py metadata.
We will fix this in python-rpm-generators and pyproject-rpm-macros, but we're low on resources.
I've submitted a PR to rpm to address this. It'll probably need to go through a few iterations of review.
.* is still broken with !=:
error: Illegal char '*' (0x2a) in: 0.42.*
.* is still broken with >=:
error: Illegal char '*' (0x2a) in: 2.3.*
The code explicitly only work with == ...* but not other specs:
elif spec == '==' and spec.endswith('.*'):
Since rpm now properly fails in those cases, this blocks the Python 3.9 rebuilds of the packages. I will come back with more as they present themselves, and we can figure out if we need to patch them as a workaround, or fix this.
Slightly unrelated, qtile has: error: Illegal char ']' (0x5d) in: 0.9[xcb]
Do you have a pointer to the specification or handling of "0.9[xcb]" as a version?
The input is: 'cairocffi>=0.9[xcb]'
I think it means cairocffi[xcb], with cairocffi>=0.9. Maybe it's just a quirk, https://www.python.org/dev/peps/pep-0508/ doesn't seem to mention it.
pip understands it:
$ pip install 'cairocffi>=0.9[xcb]'
Downloading https://files.pythonhosted.org/packages/f7/99/b3a2c6393563ccbe081ffcceb359ec27a6227792c5169604c1bd8128031a/cairocffi-1.1.0.tar.gz (68kB)
|████████████████████████████████| 71kB 1.6MB/s
Collecting cffi>=1.1.0 (from cairocffi>=0.9[xcb])
Downloading https://files.pythonhosted.org/packages/05/7d/723ceca757d8016850d1110df44b91121695f22f782f876b483ecb3928a9/cffi-1.13.2-cp38-cp38-manylinux1_x86_64.whl (439kB)
|████████████████████████████████| 440kB 3.4MB/s
Requirement already satisfied: setuptools>=39.2.0 in ./__venv__/lib/python3.8/site-packages (from cairocffi>=0.9[xcb]) (41.2.0)
Collecting xcffib>=0.3.2 (from cairocffi>=0.9[xcb])
Downloading https://files.pythonhosted.org/packages/fc/35/101babf6c90faf3e785e3b0fa4f5010a0fd6d20ec8f5d9395f0560263452/xcffib-0.9.0.tar.gz (83kB)
|████████████████████████████████| 92kB 7.8MB/s
Collecting pycparser (from cffi>=1.1.0->cairocffi>=0.9[xcb])
Collecting six (from xcffib>=0.3.2->cairocffi>=0.9[xcb])
Using cached https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Installing collected packages: pycparser, cffi, six, xcffib, cairocffi
Running setup.py install for xcffib ... done
Running setup.py install for cairocffi ... done
Successfully installed cairocffi-1.1.0 cffi-1.13.2 pycparser-2.19 six-1.14.0 xcffib-0.9.0
"cairocffi with the optional xcb component" makes sense, though I'm surprised that it can be specified in that order.
The rpm packages for python3-cairocffi.noarch don't seem to express that "xcb" is included, so I'm not sure if there's any options for handling extras other than to ignore them entirely.
I've also created: https://github.com/gordonmessmer/pyreq2rpm/issues/5
I've put some debugging statements in pip, and I'm *almost* sure that pip doesn't actually parse the "extras" part of that string:
DEBUG: req: cairocffi
DEBUG: req spec: >=0.9[xcb]
DEBUG: req extra: set()
This seems more like a bug in qtile than anything else.
Fixed in https://github.com/qtile/qtile/commit/749dfc9bc615b1a3c0cc15e7958d958711b0c2d6
I'll open a qtile bug.
The dependency generator doesn't handle extras right now because nobody agrees on how we would handle them. It doesn't help that this particular part of Python module packaging does not appear to be defined well, and I'm pretty sure the behavior handling this is specific to the dependency manager (pip vs pipenv vs poetry, etc).
Personally, we should probably just filter those out.
I think "cairocffi[xcb] >= 0.9.0" should be trated as "cairocffi >= 0.9.0" at least. However the qtile case, "cairocffi>=0.9[xcb]", that is a qtile bug: https://bugzilla.redhat.com/show_bug.cgi?id=1793402
OK, let's discuss how to handle extras: https://github.com/rpm-software-management/rpm/pull/1014
Making sure we are on the same page:
Handling extras is nice to have. Handling issues from comment #10 (!=x.y.* and >=x.y.*) seems like the real problem.
I'm seeing this in F31 too: https://koji.fedoraproject.org/koji/packageinfo?packageID=27750
It didn't fail in F30 or in F32 though. Have they received fixes already, or is this a regression in F31?
> It didn't fail in F30 or in F32 though
F30 has RPM version that doesn't fail, but the error is in the log:
error: Illegal char '*' (0x2a) in: 1.*
error: Illegal char '*' (0x2a) in: 1.*
error: Illegal char '*' (0x2a) in: 4.*
F31 and F32 have RPM version that fails.
F31 has not yet received the fix for the == x.y.* case.
Ah, no worries. I'll tweak the requirements file to use the ">=x.y, <x+1" type specs for the time being.
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.
Chances are * in != still blows up:
Fixed upstream. I'll backport it a bit later with some other changes (unless somebody else does it).
Backported in https://src.fedoraproject.org/rpms/python-rpm-generators/c/783dcc71471c6544e4cb89cef35b78be735f0a20?branch=master