Bug 1661309 - Enabling Python Generators by default
Summary: Enabling Python Generators by default
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: Changes Tracking
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Igor Raits
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-20 19:07 UTC by Ben Cotton
Modified: 2019-11-19 15:34 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-19 15:34:19 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Ben Cotton 2018-12-20 19:07:53 UTC
This is a tracking bug for Change: Enabling Python Generators by default
For more details, see: https://fedoraproject.org/wiki/Changes/EnablingPythonGeneratorsByDefault

This change enables the Python module dependency generator for packages that provide Python Egg/Wheel metadata by default (this was opt-in since Fedora 28).

Comment 1 Avram Lubkin 2018-12-28 14:41:07 UTC
This doesn't appear to be supported in EPEL builds. I recommend we hold off on this until it's supported in EPEL. It is definitely useful, but unless it can be used across branches it is of limited use.

Comment 2 Igor Raits 2018-12-28 15:46:43 UTC
Sorry, but this is Fedora change and I am not interested in any EPEL work.

In any case, it has been supported since F28 so you can use it for all Fedora branches.

And also, this has been already turned on so there is nothing to hold off.

Comment 3 Igor Raits 2018-12-28 15:52:19 UTC
People who care about EPEL had almost a year to implement it, but no one did it. So delaying Fedora by EPEL is no-go.

Comment 4 Avram Lubkin 2018-12-28 16:10:09 UTC
You championed this change so I think it falls on you to make it work across all branches. EPEL is part of the Fedora project, so it's included. You're seeing duplicate requires in some packages and have not provided a mechanism for people to streamline their branches. So I recommend turning it off or porting it to EPEL. I looked at it, but pythondistdeps.py doesn't even have tests. How is this ready for prime time?

Comment 5 Neal Gompa 2018-12-28 16:36:14 UTC
(In reply to Avram Lubkin from comment #4)
> You championed this change so I think it falls on you to make it work across
> all branches. EPEL is part of the Fedora project, so it's included. You're
> seeing duplicate requires in some packages and have not provided a mechanism
> for people to streamline their branches. So I recommend turning it off or
> porting it to EPEL. I looked at it, but pythondistdeps.py doesn't even have
> tests. How is this ready for prime time?

While it is true that EPEL is a subproject of Fedora, the main Fedora Linux distribution is the upstream for RHEL. If it's never activated in Fedora, it can never make its way into RHEL. Consequently, it would never become available for EPEL branches either.

It is trivial enough to deal with branches that don't have the generator, as packages like pagure do so.

This is a change for Fedora 30, and obviously it means it doesn't apply to older Fedora releases, or even RHEL releases that EPEL builds on.

Comment 6 Avram Lubkin 2018-12-28 17:39:32 UTC
It's a change to how things are done in Fedora. It has no tests and can not be opted-in in other branches. It's just not ready yet.

Comment 7 Igor Raits 2018-12-28 17:44:46 UTC
(In reply to Avram Lubkin from comment #6)
> can not be opted-in in other branches

That's not true. It works perfectly fine even in F28.

Comment 8 Jason Montleon 2019-01-17 18:35:48 UTC
One of our packages was updated to use this.

Now when trying to install the python3 package I get:
python3-kubernetes-8.0.0-5.fc30.noarch.rpm      318 kB/s | 1.0 MB     00:03    
Error: 
 Problem: conflicting requests
  - nothing provides python3.7dist(ipaddress) >= 1.0.17 needed by python3-kubernetes-8.0.0-5.fc30.noarch


The python 2 ipaddress package is a port for the python3.3+ functionality, and no package seems to provide this for python 3 on Fedora 29 or 30 that I can see, like is done for python 2: https://koji.fedoraproject.org/koji/rpminfo?rpmID=14769581

This leaves the package uninstallable.

Comment 9 Igor Raits 2019-01-17 18:42:43 UTC
(In reply to Jason Montleon from comment #8)
> One of our packages was updated to use this.
> 
> Now when trying to install the python3 package I get:
> python3-kubernetes-8.0.0-5.fc30.noarch.rpm      318 kB/s | 1.0 MB     00:03 
> 
> Error: 
>  Problem: conflicting requests
>   - nothing provides python3.7dist(ipaddress) >= 1.0.17 needed by
> python3-kubernetes-8.0.0-5.fc30.noarch
> 
> 
> The python 2 ipaddress package is a port for the python3.3+ functionality,
> and no package seems to provide this for python 3 on Fedora 29 or 30 that I
> can see, like is done for python 2:
> https://koji.fedoraproject.org/koji/rpminfo?rpmID=14769581
> 
> This leaves the package uninstallable.

In this case you should not add "ipaddress" in "install_requires" in setup.py (or wherever you generate .egg-info).

Comment 10 Jason Montleon 2019-01-17 18:53:20 UTC
How should that be done to satisfy both python 2 and 3 then. Leaving it out will not work on python 2.

https://github.com/kubernetes-client/python/blob/master/requirements.txt#L8

Comment 11 Neal Gompa 2019-01-17 18:57:28 UTC
You can use environment markers. Here's an example from Pagure's requirements.txt file: https://pagure.io/pagure/blob/master/f/requirements.txt#_13

Comment 12 Jason Montleon 2019-01-17 19:15:54 UTC
Something very similar is being used: ipaddress>=1.0.17;python_version=="2.7"

Why isn't this being honored?

Comment 13 Jason Montleon 2019-01-17 19:20:28 UTC
Based on the examples in the documentation python_version=="2.7" is perfectly valid.
https://www.python.org/dev/peps/pep-0496/#id7

I don't own the python-kubernetes repo. I'm willing to ask the devs to update it if it's incorrect, but from what I see there is nothing wrong with what they've defined.

Comment 14 Igor Raits 2019-01-17 19:23:04 UTC
setup(
    name=PACKAGE_NAME,
    version=CLIENT_VERSION,
    description="Kubernetes python client",
    author_email="",
    author="Kubernetes",
    license="Apache License Version 2.0",
    url="https://github.com/kubernetes-client/python",
    keywords=["Swagger", "OpenAPI", "Kubernetes"],
    install_requires=['certifi>=14.05.14', 'six>=1.9.0', 'python-dateutil>=1.5', 
                      'setuptools>=0.9.8', 'urllib3>=1.10', 'pyyaml>=3.10', 
                      'google-auth>=1.0.1', 'ipaddress>=1.0.17',
                      'websocket-client>=0.32.0',
                      'requests', 'requests-oauthlib' , 'adal>=1.0.2'],


it's defined in setup.py

Comment 15 Avram Lubkin 2019-02-18 08:49:46 UTC
This still doesn't have any test code.

Comment 16 Igor Raits 2019-02-18 09:14:21 UTC
(In reply to Avram Lubkin from comment #15)
> This still doesn't have any test code.

There are much more things which do not have test code inside RPM and related things…

Feel free to send a pull request. Unfortunately I don't have time for this :(

Comment 17 Avram Lubkin 2019-02-18 09:36:57 UTC
(In reply to Igor Gnatenko from comment #16)
> There are much more things which do not have test code inside RPM and
> related things…

That's not really an excuse. If the code is well written and the problem set understood, it shouldn't take very long to put tests together. They really should have been included from the start and you agreed almost two months ago they were needed.


> Feel free to send a pull request. Unfortunately I don't have time for this :(
Then hold the feature until you have time to implement it correctly or ask on fedora-devel or fedora-python-devel for volunteers.
I'd be tempted to put something together , but unfortunately I'm on the road for a while and the spare cycles I have are already fill with open source commitments.

Comment 18 Igor Raits 2019-02-18 09:54:26 UTC
(In reply to Avram Lubkin from comment #17)
> (In reply to Igor Gnatenko from comment #16)
> > There are much more things which do not have test code inside RPM and
> > related things…
> 
> That's not really an excuse. If the code is well written and the problem set
> understood, it shouldn't take very long to put tests together. They really
> should have been included from the start and you agreed almost two months
> ago they were needed.
> 
> 
> > Feel free to send a pull request. Unfortunately I don't have time for this :(
> Then hold the feature until you have time to implement it correctly or ask
> on fedora-devel or fedora-python-devel for volunteers.
> I'd be tempted to put something together , but unfortunately I'm on the road
> for a while and the spare cycles I have are already fill with open source
> commitments.

The problem is not much with Python, but rather to get it as part of testing infrastructure in RPM which is… autotest. Also the code has been written back in 2010-2015 and is in use by openSUSE and Mageia for some years.

Comment 19 Neal Gompa 2019-02-18 12:03:53 UTC
(In reply to Jason Montleon from comment #13)
> Based on the examples in the documentation python_version=="2.7" is
> perfectly valid.
> https://www.python.org/dev/peps/pep-0496/#id7
> 
> I don't own the python-kubernetes repo. I'm willing to ask the devs to
> update it if it's incorrect, but from what I see there is nothing wrong with
> what they've defined.

The logic in the setup.py strips the environment markers from the requirements.txt when populating install_requires, presumably to support versions of setuptools that don't handle it (such as the version in RHEL 7): https://github.com/kubernetes-client/python/blob/master/setup.py#L34-L43

That's why it's doing the wrong thing.

Comment 20 Neal Gompa 2019-11-19 15:34:19 UTC
This has been long-done.


Note You need to log in before you can comment on or make changes to this bug.