Bug 1738814 - postgresql depends on Python 2
Summary: postgresql depends on Python 2
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: postgresql
Version: 31
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Patrik Novotný
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F31_PY2REMOVAL
TreeView+ depends on / blocked
 
Reported: 2019-08-08 08:36 UTC by Lumír Balhar
Modified: 2020-05-11 21:55 UTC (History)
13 users (show)

Fixed In Version: postgresql-11.5-5.fc31
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-22 01:20:46 UTC
Type: Bug


Attachments (Terms of Use)

Description Lumír Balhar 2019-08-08 08:36:46 UTC
Python 2.7 will reach end-of-life in January 2020, over 9 years after it was released. This falls within the Fedora 31 lifetime.
Packages that depend on Python 2 are being switched to Python 3 or removed from Fedora: https://fedoraproject.org/wiki/Changes/F31_Mass_Python_2_Package_Removal#Information_on_Remaining_Packages
Python 2 will be retired in Fedora 32: https://fedoraproject.org/wiki/Changes/RetirePython2

To help planning, we'd like to know the plans for postgresql's future. Specifically:


- What is the reason for the Python2 dependency? (Is it software written in Python, or does it just provide Python bindings, or use Python in the build system or test runner?) 

- What are the upstream/community plans/timelines regarding Python 3?

- What is the guidance for porting to Python 3? (Assuming that there is someone who generally knows how to port to Python 3, but doesn't know anything about the particular package, what are the next steps to take?)


This bug is filed semi-automatically, and might not have all the context specific to postgresql.
If you need anything from us, or something is unclear, please mention it here.

Thank you.

Comment 1 Ben Cotton 2019-08-13 16:52:27 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 2 Ben Cotton 2019-08-13 17:33:37 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Comment 3 Lumír Balhar 2019-08-15 08:14:45 UTC
Please answer the above questions. If you don't, the package can be orphaned: https://fedoraproject.org/wiki/Changes/F31_Mass_Python_2_Package_Removal#Information_on_Remaining_Packages

If you need any information or help, or if you need some more time, please let us know.

Comment 4 Patrik Novotný 2019-08-20 11:34:58 UTC
- What is the reason for the Python2 dependency? (Is it software written in Python, or does it just provide Python bindings, or use Python in the build system or test runner?)

The reason is PL/Python bindings for Python2.


- What are the upstream/community plans/timelines regarding Python 3?

As both PL/Python2 and PL/Python3 are provided, upstream plans to support bindings for Python2 for distant future. However, Python3-only builds are possible.


- What is the guidance for porting to Python 3? (Assuming that there is someone who generally knows how to port to Python 3, but doesn't know anything about the particular package, what are the next steps to take?)

Only custom scripts using PL/Python would be needed to be ported to Python3.


More information:
https://www.postgresql.org/docs/current/plpython-python23.html

Comment 5 Tom Lane 2019-08-20 13:36:35 UTC
FWIW, if Fedora in general is dropping Python 2 support, I think you should just stop building the plpython2 language.

It'd greatly simplify the specfile to build only one plpython variant ;-)

Comment 6 Pavel Raiskup 2019-08-20 14:37:11 UTC
Last time I checked this 'plpython' was an equivalent to 'plpython2', so the only
think I'm afraid of is the migration path from Fedora N (including plpython2) to
Fedora N+1 (not including plpython2).

Not many users probably have plpython loaded, but if they do - they most likely
have the python2 version now.

Comment 7 Petr Viktorin 2019-08-20 15:08:03 UTC
Could you ask upstream to revisit [the documentation]? It would be good to clarify when is the "distant future" where plpythonu is switched to Python 3. The "migration to Python 3 in the Python community" is finishing. Also, PEP 394 changed ([PEP 1] says Informational PEPs can be updated, and PEP 394 itself says that it will be updated).
Upstream support for Python 2 is ending in a few months. From January 2020, upstream will stop releasing even security updates. 


As for Fedora, it's possible to keep postgresql on Python 2 in Fedora 32+, you'll just need a FESCo exception. An example is here: https://pagure.io/fesco/issue/2208
If upstream comes up with a more concrete plan than "distant future", we should try to align to it.  Otherwise, we need to find and justify a better answer to "when to drop Python 2" than the default, f32.


[the documentation]: https://www.postgresql.org/docs/current/plpython-python23.html
[PEP 1]: https://www.python.org/dev/peps/pep-0001/#pep-maintenance

Comment 8 Petr Kubat 2019-08-22 06:26:02 UTC
From a recent discussion on the pgsql-hackers ML it seems like the change of default python version is not coming sooner than in Postgresql 13. Not sure if upstream already has a schedule for 13 in mind, but it will definitely come after Python upstream drops support for 2.7.

Since Tom is already watching this bug he might be able to provide more info on the decision.

[1] https://www.postgresql.org/message-id/CAKmB1PGDAy9mXxSTqUchYEi4iJAA6NKVj4P5BtAzvQ9wSDUwJw@mail.gmail.com

Comment 9 Tom Lane 2019-08-22 15:12:18 UTC
Yeah, so ... we should have done something about this for PG v12, but we didn't, and nobody is going to want to break v12 feature freeze to the extent that would be needed to fix it.  So nothing is going to happen before v13, which will be out next fall if we hold to the standard schedule (which seems highly likely).

I have now started on some of the infrastructure work that would be needed to have more flexibility in this area [1].  But it seems like the really key questions remain to be answered: is it okay to let packagers decide whether "plpythonu" means python2 or python3; if not, is PG upstream ever going to change that; while if so, how would we make it work exactly?  I'm not sure how long it will take to reach consensus on such points.

In the meantime, it's worth pointing out that users of plpythonu probably don't care at all about whether Python upstream stops maintaining the language.  They're not looking for new features (else they'd be on python3), and they don't care about security issues because plpythonu is already a superuser-only procedural language, thanks to the lack of any reliable sandboxing feature in python2.  (OT: did python3 fix that?)  Only if they actively trip over a python bug, which seems unlikely, would they feel any need to migrate.  So if you want to get out of the business of shipping python2, you're going to have to push them.

Perhaps a reasonable approach is to announce that support for plpythonu and plpython2u will be dropped in F32, and to include some sort of migration tool in F31, if there's still time for that.  My first thoughts about a migration tool would be a script that finds every plpython(2)u function in the database, runs 2to3 over the body, and updates the function with the new body text and language plpython3u.  This might not be quite sufficient for everybody --- I have no idea how thorough or reliable 2to3 is --- but it'd be a creditable starting point for user testing.

[1[ https://www.postgresql.org/message-id/5889.1566415762@sss.pgh.pa.us

Comment 10 Tom Lane 2019-08-23 18:35:04 UTC
Also, an attempt to move things along in a different direction here:

https://www.postgresql.org/message-id/11546.1566584867%40sss.pgh.pa.us

Comment 11 Petr Viktorin 2019-08-26 09:31:03 UTC
Please file a FESCo exception with that info. It seems reasonable to keep the Python 2 support for now. But do consider at least pointing "plpythonu" to Python 3 in v13 (upstream, preferably).

While we're aggressively trimming the python2 stack in f32, we will need to keep the interpreter itself for some more time, and postgresql doesn't require any additional Python libraries.
It would be much better to move with upstream here.


> OT: did python3 fix [the lack of any reliable sandboxing feature]?

No, AFAIK Python devs would consider a sandbox out of scope for the language/runtime. I'm not really aware of requests/efforts to add one.

Comment 12 Petr Kubat 2019-08-29 06:45:35 UTC
I agree it would be best to align with upstream on this. Opened up a ticket asking for exception from FESCO: https://pagure.io/fesco/issue/2217

Comment 13 Miro Hrončok 2019-08-29 08:14:00 UTC
As a side note, can you please rename postgresql-plpython to postgresql-plpython2 and virtually provide postgresql-plpython from either postgresql-plpython2 or postgresql-plpython3 as you see fit? That way, people installing postgresql-plpython will immediately see what Python version that is.

Comment 14 Miro Hrončok 2019-08-29 09:03:17 UTC
What about postgresql-upgrade?

Comment 15 Petr Kubat 2019-08-29 09:41:56 UTC
>As a side note, can you please rename postgresql-plpython to postgresql-plpython2 and virtually provide postgresql-plpython from either postgresql-plpython2 or postgresql-plpython3 as you see fit?

Makes sense, will do.

>What about postgresql-upgrade?

Same thing really - postgresql-upgrade contains a stripped down version of the postgresql server from which the upgrade is performed (so for postgresql v11, postgresql-upgrade contains v10). This version of the postgresql server has to be able to work with all extensions and language handlers we have shipped in the previous version of the rpm so that the upgrade goes through as smoothly as possible for the user performing the upgrade.
The tricky thing will be (as mentioned by Pavel in comment 6) how to perform the upgrade from plpython2 to future default plpython3.

Comment 16 Fedora Update System 2019-09-02 16:28:00 UTC
FEDORA-2019-bad65b3890 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-bad65b3890

Comment 18 Pavel Raiskup 2019-09-02 19:06:04 UTC
Though I'm not sure the change is OK without explicit Obsoletes?

Comment 19 Miro Hrončok 2019-09-02 23:21:45 UTC
(In reply to Pavel Raiskup from comment #18)
> Though I'm not sure the change is OK without explicit Obsoletes?

No, it should be done with explicit Obsoletes: https://docs.fedoraproject.org/en-US/packaging-guidelines/#renaming-or-replacing-existing-packages

Comment 20 Fedora Update System 2019-09-02 23:56:56 UTC
postgresql-11.5-3.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-bad65b3890

Comment 21 Fedora Update System 2019-09-03 11:25:44 UTC
FEDORA-2019-6324af709b has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-6324af709b

Comment 22 Patrik Novotný 2019-09-03 11:26:58 UTC
Thanks for pointing that out. I've submitted new update with explicit Obsoletes.

Comment 23 Fedora Update System 2019-09-03 18:45:53 UTC
postgresql-11.5-4.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-6324af709b

Comment 24 Fedora Update System 2019-09-05 15:43:07 UTC
FEDORA-2019-815b0dee67 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-815b0dee67

Comment 25 Fedora Update System 2019-09-06 00:22:39 UTC
postgresql-11.5-5.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-815b0dee67

Comment 26 Fedora Update System 2019-09-22 01:20:46 UTC
postgresql-11.5-5.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.