Bug 1758141

What bug are you reporting? Is it that the automatic dependencies generator is broken (python-rpm-generators) or that astroid is missing dependencies?

Let me check what happens here.

The requires.txt went from:

lazy_object_proxy
six
wrapt

to:

lazy_object_proxy==1.4.*
six==1.12
wrapt==1.11.*


The asterisks are valid according to PEP 440, but easy to overlook -- I wouldn't be surprised if python-rpm-generators had a bug in handling them:
https://www.python.org/dev/peps/pep-0440/#version-matching

install_requires = [
    "lazy_object_proxy==1.4.*",
    "six==1.12",
    "wrapt==1.11.*",
    'typed-ast>=1.4.0,<1.5;implementation_name== "cpython" and python_version<"3.8"',
]

It only requires:

python3.8dist(six) = 1.12




I think I know what the problem is. The 1.11.* versions with * is not handled properly.

Note that in the first place, we should probably fallback to generating at least unversioned dependency when we cannot parse/convert the version.

RPMbuild says:

error: Illegal char '*' (0x2a) in: 1.4.*
error: Illegal char '*' (0x2a) in: 1.11.*


And ignores the dependency.

IMHO such error should stop the build, I've opened: https://github.com/rpm-software-management/rpm/issues/881

As an immediate workaround, either disable the dependency generator or patch the version in setup.py metadata.
We will fix this in python-rpm-generators and pyproject-rpm-macros, but we're low on resources.

I've submitted a PR to rpm to address this.  It'll probably need to go through a few iterations of review.

https://github.com/rpm-software-management/rpm/pull/951

https://src.fedoraproject.org/rpms/python-rpm-generators/pull-request/4

.* is still broken with !=:


python-kubernetes:

python-10.0.1/requirements.txt
websocket-client>=0.32.0,!=0.40.0,!=0.41.*,!=0.42.* 

error: Illegal char '*' (0x2a) in: 0.42.*


.* is still broken with >=:

python-fsleyes:

fsleyes-0.32.0/fsleyes.egg-info/requires.txt
nibabel>=2.3.*

error: Illegal char '*' (0x2a) in: 2.3.*


The code explicitly only work with == ...* but not other specs:

    elif spec[0] == '==' and spec[1].endswith('.*'):

Since rpm now properly fails in those cases, this blocks the Python 3.9 rebuilds of the packages. I will come back with more as they present themselves, and we can figure out if we need to patch them as a workaround, or fix this.

Slightly unrelated, qtile has: error: Illegal char ']' (0x5d) in: 0.9[xcb]

Do you have a pointer to the specification or handling of "0.9[xcb]" as a version?

The input is: 'cairocffi>=0.9[xcb]'

I think it means cairocffi[xcb], with cairocffi>=0.9. Maybe it's just a quirk, https://www.python.org/dev/peps/pep-0508/ doesn't seem to mention it.

pip understands it:

$ pip install 'cairocffi>=0.9[xcb]'
Collecting cairocffi>=0.9[xcb]
  Downloading https://files.pythonhosted.org/packages/f7/99/b3a2c6393563ccbe081ffcceb359ec27a6227792c5169604c1bd8128031a/cairocffi-1.1.0.tar.gz (68kB)
     |████████████████████████████████| 71kB 1.6MB/s 
Collecting cffi>=1.1.0 (from cairocffi>=0.9[xcb])
  Downloading https://files.pythonhosted.org/packages/05/7d/723ceca757d8016850d1110df44b91121695f22f782f876b483ecb3928a9/cffi-1.13.2-cp38-cp38-manylinux1_x86_64.whl (439kB)
     |████████████████████████████████| 440kB 3.4MB/s 
Requirement already satisfied: setuptools>=39.2.0 in ./__venv__/lib/python3.8/site-packages (from cairocffi>=0.9[xcb]) (41.2.0)
Collecting xcffib>=0.3.2 (from cairocffi>=0.9[xcb])
  Downloading https://files.pythonhosted.org/packages/fc/35/101babf6c90faf3e785e3b0fa4f5010a0fd6d20ec8f5d9395f0560263452/xcffib-0.9.0.tar.gz (83kB)
     |████████████████████████████████| 92kB 7.8MB/s 
Collecting pycparser (from cffi>=1.1.0->cairocffi>=0.9[xcb])
Collecting six (from xcffib>=0.3.2->cairocffi>=0.9[xcb])
  Using cached https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Installing collected packages: pycparser, cffi, six, xcffib, cairocffi
  Running setup.py install for xcffib ... done
  Running setup.py install for cairocffi ... done
Successfully installed cairocffi-1.1.0 cffi-1.13.2 pycparser-2.19 six-1.14.0 xcffib-0.9.0

"cairocffi with the optional xcb component" makes sense, though I'm surprised that it can be specified in that order.

The rpm packages for python3-cairocffi.noarch don't seem to express that "xcb" is included, so I'm not sure if there's any options for handling extras other than to ignore them entirely.

I've also created: https://github.com/gordonmessmer/pyreq2rpm/issues/5

I've put some debugging statements in pip, and I'm *almost* sure that pip doesn't actually parse the "extras" part of that string:

DEBUG: req: cairocffi
DEBUG: req spec: >=0.9[xcb]
DEBUG: req extra: set()

This seems more like a bug in qtile than anything else.

Fixed in https://github.com/qtile/qtile/commit/749dfc9bc615b1a3c0cc15e7958d958711b0c2d6

I'll open a qtile bug.

The dependency generator doesn't handle extras right now because nobody agrees on how we would handle them. It doesn't help that this particular part of Python module packaging does not appear to be defined well, and I'm pretty sure the behavior handling this is specific to the dependency manager (pip vs pipenv vs poetry, etc).

Personally, we should probably just filter those out.

I think "cairocffi[xcb] >= 0.9.0" should be trated as "cairocffi >= 0.9.0" at least. However the qtile case, "cairocffi>=0.9[xcb]", that is a qtile bug: https://bugzilla.redhat.com/show_bug.cgi?id=1793402

OK, let's discuss how to handle extras: https://github.com/rpm-software-management/rpm/pull/1014

Making sure we are on the same page:

Handling extras is nice to have. Handling issues from comment #10 (!=x.y.* and >=x.y.*) seems like the real problem.

https://github.com/rpm-software-management/rpm/pull/1015

Hello,

I'm seeing this in F31 too: https://koji.fedoraproject.org/koji/packageinfo?packageID=27750

It didn't fail in F30 or in F32 though. Have they received fixes already, or is this a regression in F31?

Cheers,
Ankur

> It didn't fail in F30 or in F32 though

F30 has RPM version that doesn't fail, but the error is in the log:

error: Illegal char '*' (0x2a) in: 1.*
error: Illegal char '*' (0x2a) in: 1.*
error: Illegal char '*' (0x2a) in: 4.*


F31 and F32 have RPM version that fails.

F31 has not yet received the fix for the == x.y.* case.

Ah, no worries. I'll tweak the requirements file to use the ">=x.y, <x+1" type specs for the time being.

Thanks.

This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Chances are * in != still blows up:

astropy>=2.0.12,!=3.0.*,!=3.1,!=3.1.1

Fixed upstream. I'll backport it a bit later with some other changes (unless somebody else does it).

Backported in https://src.fedoraproject.org/rpms/python-rpm-generators/c/783dcc71471c6544e4cb89cef35b78be735f0a20?branch=master

Thanks.