Bug 1639030 - Please provide a complete Python 3 stack for porting from Python 2
Summary: Please provide a complete Python 3 stack for porting from Python 2
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: python3
Version: 7.7
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: pre-dev-freeze
: ---
Assignee: Python Maintainers
QA Contact: BaseOS QE - Apps
URL:
Whiteboard:
Depends On: 1672102 1719978 1597718 1660563 1660573 1660574 1660578 1660579 1691402 1719977
Blocks: 1630906 1661173
TreeView+ depends on / blocked
 
Reported: 2018-10-14 14:54 UTC by Neal Gompa
Modified: 2019-07-01 08:05 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-07-01 08:05:38 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1597718 None None None 2019-07-25 12:36:30 UTC

Internal Links: 1597718

Description Neal Gompa 2018-10-14 14:54:10 UTC
Description of problem:
With the Red Hat Enterprise Linux 7.6 Beta, Python 2 has been deprecated and is expected to be removed for the next major version of Red Hat Enterprise Linux[1].

However, the advice to use the Python 3 SCL is not going to work for me, because I rely on Python bindings of system components and leverage the modules that are included in the distribution (including rpm, libvirt, yum, etc.).

In order for provide an effective means of supporting porting to Python 3, please provide us with a Python 3 stack that covers everything included in RHEL 7 today.

I understand that YUM 3 does not currently have a Python 3 variant, but YUM 4[2] can be built to provide Python 3 interfaces.

In doing so, this will make it possible for me (and others) to more aggressively work on porting to Python 3 to support a Python 3-only future in the next major version of Red Hat Enterprise Linux.

[1]: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7-beta/html-single/7.6_release_notes/index#idm140579843349392

[2]: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7-beta/html-single/7.6_release_notes/index#BZ1461652

Version-Release number of selected component (if applicable):
N/A (there's no python3 package in RHEL, even though this component exists)

Comment 2 Neal Gompa 2018-10-14 14:58:09 UTC
To be clear, I am not requesting that python2 modules be replaced with python3 modules, only that python3 modules for everything currently provided for python2 to be additionally available.

Comment 3 Miro Hrončok 2018-10-14 17:58:22 UTC
This boils down to two things:

1) introduce python3 package, possibly with setuptools and pip (fairly easy form developer pov IMHO)

2) introduce everything that we ship for python2 (not that easy)


I think if we start with just 1) users/customers can request other python3 packages and the requests can be evaluated one by one. A request for everything is IMHO unlikely to be fulfilled.

(Note that those are just my opinions as a developer, it's not me who make those decisions.)

Comment 4 Neal Gompa 2018-10-15 03:52:31 UTC
(In reply to Miro Hrončok from comment #3)
> This boils down to two things:
> 
> 1) introduce python3 package, possibly with setuptools and pip (fairly easy
> form developer pov IMHO)
> 

I would seriously hope setuptools and pip would be included, otherwise it's going to be hard to use Python 3 for development...

> 2) introduce everything that we ship for python2 (not that easy)
> 
> 
> I think if we start with just 1) users/customers can request other python3
> packages and the requests can be evaluated one by one. A request for
> everything is IMHO unlikely to be fulfilled.
> 
> (Note that those are just my opinions as a developer, it's not me who make
> those decisions.)

Ordinarily, I would agree with you, but there are two problems:

1. A number of Python modules provided in RHEL are simply just not available through PyPI and cannot be made available that way for various reasons. These include, but are not limited to: rpm, hawkey, solv, libvirt, and so on. 

2. Most of the Python modules in the distribution have Python 3 subpackages in the upstream Fedora packaging, and it would be trivial to sync the packaging (even if you kept the version and patchset of the modules currently in RHEL 7). If I was asking for actual porting work, I think this would be much harder to do. But pretty much all the modules in question even had Python 3 modules back when Fedora was forked into RHEL 7 (when the Python 3 packaging was stripped). And a number of modules have been updated since then, even in RHEL. Some of the newer ones even have conditionals to disable Python 3 builds for RHEL 7, but enable them in Fedora. 

I feel like it's unfair to expect us to bend over backwards this way and have to do our own repackaging of the modules you already provide just so we can start porting to Python 3 (which you guys want us to do to support a Python 3-only environment in the next major version of RHEL).

Believe me, I *want* to port *everything* to Python 3. But not giving us a full stack for this is a major disincentive. I could live with some of it being introduced sooner, and the rest coming later, but I'm specifically requesting the whole enchilada because otherwise it's not reasonable to expect everything to be able to switch before Python 2 is gone.

Comment 6 Honza Horak 2018-10-15 18:46:19 UTC
Moving back to python3 component, since having it on distribution component does not allow having it public.

Comment 8 Carl George 2018-11-14 18:29:46 UTC
I love this idea.  It would be a welcome addition for EPEL packagers like myself.

Comment 11 Blaise Pabon 2019-02-12 15:08:44 UTC
Fwiw, Ansible seems to have a dependency on python3-dnf which is making trouble for a lot of roles running on fedora 29.
I understand that it may be easier to fix fc29 than RHEL7, but I wanted to illustrate yet another obstacle to migration.

Comment 14 Orion Poplawski 2019-04-12 14:51:06 UTC
It would be nice if there was some coordination with EPEL on this.

Comment 15 Miro Hrončok 2019-04-12 15:05:31 UTC
I believe that the entire 3.4 -> 3.6 switch was a coordination with EPEL.

I also plan to remove python36 and any other obsoleted package from EPEL. See also:

https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org/message/BE5FX3YVHGZAOAP7PWPYJPMHD2NDO7VW/

Comment 16 Orion Poplawski 2019-04-12 15:16:55 UTC
My apologies - I somehow missed that the 3.4->3.6 switch was in coordination with this.

Comment 17 Carl George 2019-04-12 15:32:17 UTC
> I also plan to remove python36 and any other obsoleted package from EPEL.

So you're saying that the RHEL packages that are on QA are version 3.6?  Also, can you share if these will be named python3 or python36?

Comment 18 Miro Hrončok 2019-04-12 16:21:15 UTC
I can only share what we plan to do as python-maint. I cannot make statements about what exactly will or will not be delivered in any future RHEL release.

> So you're saying that the RHEL packages that are on QA are version 3.6?

https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org/message/BE5FX3YVHGZAOAP7PWPYJPMHD2NDO7VW/

"We would like to add Python 3.6 to RHEL 7"

> Also, can you share if these will be named python3 or python36?

I don't know (whether I can share). Either way a proper upgrade path from EPEL is planed.

Comment 20 Petr Viktorin 2019-04-15 08:03:04 UTC
I'd prefer to call it `python3`. Given how late in the support cycle we are for RHEL 7, adding any other python3 is unlikely.
If it's python3, it should Provide/Obsolete `python36`, similar to what the Fedora package does. That would make it compatible with the EPEL stack.

Comment 21 Raphael Groner 2019-05-14 11:35:28 UTC
The whole doubled stack with python34 vs. 36 was and is a mess in EPEL7. Especially %python3_other - No good idea IMHO.

Comment 26 Carl George 2019-06-11 04:02:13 UTC
I was really hoping to see python3-createrepo_c and python3-libselinux added as part of this effort, but they are not in the RHEL 7.7 beta as far as I can tell.  Similar to the libraries mentioned in bug 1672102, these python bindings must be built from their RHEL packages (createrepo_c and libselinux).

Please enable the python3 subpackages of createrepo_c and libselinux to enhance this new python3 stack.

Comment 27 Miro Hrončok 2019-06-11 06:02:21 UTC
Carl, you have more chances if you open Bugzillas for the mentioned components.

Comment 29 Neal Gompa 2019-06-22 06:13:22 UTC
Honza, 

I see that you marked this bug as "ON_QA" for RHEL 7.7, but the RHEL 7.7 Beta release notes don't have any reference to this bug.

What's going on here? I do see that the Python 3 interpreter, pip, and setuptools were introduced (with a private rhbz as reference for that), but this bug hasn't been addressed. Is it actually being worked on or not?

Comment 30 Petr Viktorin 2019-06-26 08:54:00 UTC
Speaking for python-maint engineers: We got in all we could push for in 7.7. Since this isn't a customer request, our power is limited. (Here on Bugzilla, you're not reaching the people who make the decisions.)
Any more packages that need enterprise support will need to be added to 7.8, or to EPEL/CentOS.

Comment 31 Honza Horak 2019-07-01 08:05:38 UTC
As indicated in comments above, the solution we're working on for a next minor version of RHEL-7 now is not "a complete Python 3 stack" as required specifically in this ticket, it's rather a reasonably usable minimum. Since it is already delivered in RHEL-7.7 Beta, we moved the bug to ON_QA and you can check what specific packages are in RHEL-7 by looking closely at RHEL-7.7 Beta.

To not set expectations wrongly, from my PoV it is quite unlikely that there will be many new packages added at this point to RHEL-7, given the state where RHEL-7 is now, the packages will be rather added into EPEL (by anybody who cares, not us proactively). That is also a reason why I believe it is ok to close this bug now with "NEXTRELEASE" resolution.

As mentioned above, Bugzilla is not a support tool, so let me comment here with an official guidance we provide:

If this issue is critical, please raise a ticket through the regular Red Hat support channels to ensure it receives the proper attention and prioritization to assure a timely resolution. 

For information on how to contact the Red Hat production support team, please visit:
    https://www.redhat.com/support/process/production/#howto


Note You need to log in before you can comment on or make changes to this bug.