Bug 1738814

This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Please answer the above questions. If you don't, the package can be orphaned: https://fedoraproject.org/wiki/Changes/F31_Mass_Python_2_Package_Removal#Information_on_Remaining_Packages

If you need any information or help, or if you need some more time, please let us know.

- What is the reason for the Python2 dependency? (Is it software written in Python, or does it just provide Python bindings, or use Python in the build system or test runner?)

The reason is PL/Python bindings for Python2.


- What are the upstream/community plans/timelines regarding Python 3?

As both PL/Python2 and PL/Python3 are provided, upstream plans to support bindings for Python2 for distant future. However, Python3-only builds are possible.


- What is the guidance for porting to Python 3? (Assuming that there is someone who generally knows how to port to Python 3, but doesn't know anything about the particular package, what are the next steps to take?)

Only custom scripts using PL/Python would be needed to be ported to Python3.


More information:
https://www.postgresql.org/docs/current/plpython-python23.html

FWIW, if Fedora in general is dropping Python 2 support, I think you should just stop building the plpython2 language.

It'd greatly simplify the specfile to build only one plpython variant ;-)

Last time I checked this 'plpython' was an equivalent to 'plpython2', so the only
think I'm afraid of is the migration path from Fedora N (including plpython2) to
Fedora N+1 (not including plpython2).

Not many users probably have plpython loaded, but if they do - they most likely
have the python2 version now.

Could you ask upstream to revisit [the documentation]? It would be good to clarify when is the "distant future" where plpythonu is switched to Python 3. The "migration to Python 3 in the Python community" is finishing. Also, PEP 394 changed ([PEP 1] says Informational PEPs can be updated, and PEP 394 itself says that it will be updated).
Upstream support for Python 2 is ending in a few months. From January 2020, upstream will stop releasing even security updates. 


As for Fedora, it's possible to keep postgresql on Python 2 in Fedora 32+, you'll just need a FESCo exception. An example is here: https://pagure.io/fesco/issue/2208
If upstream comes up with a more concrete plan than "distant future", we should try to align to it.  Otherwise, we need to find and justify a better answer to "when to drop Python 2" than the default, f32.


[the documentation]: https://www.postgresql.org/docs/current/plpython-python23.html
[PEP 1]: https://www.python.org/dev/peps/pep-0001/#pep-maintenance

From a recent discussion on the pgsql-hackers ML it seems like the change of default python version is not coming sooner than in Postgresql 13. Not sure if upstream already has a schedule for 13 in mind, but it will definitely come after Python upstream drops support for 2.7.

Since Tom is already watching this bug he might be able to provide more info on the decision.

[1] https://www.postgresql.org/message-id/CAKmB1PGDAy9mXxSTqUchYEi4iJAA6NKVj4P5BtAzvQ9wSDUwJw@mail.gmail.com

Yeah, so ... we should have done something about this for PG v12, but we didn't, and nobody is going to want to break v12 feature freeze to the extent that would be needed to fix it.  So nothing is going to happen before v13, which will be out next fall if we hold to the standard schedule (which seems highly likely).

I have now started on some of the infrastructure work that would be needed to have more flexibility in this area [1].  But it seems like the really key questions remain to be answered: is it okay to let packagers decide whether "plpythonu" means python2 or python3; if not, is PG upstream ever going to change that; while if so, how would we make it work exactly?  I'm not sure how long it will take to reach consensus on such points.

In the meantime, it's worth pointing out that users of plpythonu probably don't care at all about whether Python upstream stops maintaining the language.  They're not looking for new features (else they'd be on python3), and they don't care about security issues because plpythonu is already a superuser-only procedural language, thanks to the lack of any reliable sandboxing feature in python2.  (OT: did python3 fix that?)  Only if they actively trip over a python bug, which seems unlikely, would they feel any need to migrate.  So if you want to get out of the business of shipping python2, you're going to have to push them.

Perhaps a reasonable approach is to announce that support for plpythonu and plpython2u will be dropped in F32, and to include some sort of migration tool in F31, if there's still time for that.  My first thoughts about a migration tool would be a script that finds every plpython(2)u function in the database, runs 2to3 over the body, and updates the function with the new body text and language plpython3u.  This might not be quite sufficient for everybody --- I have no idea how thorough or reliable 2to3 is --- but it'd be a creditable starting point for user testing.

[1[ https://www.postgresql.org/message-id/5889.1566415762@sss.pgh.pa.us

Also, an attempt to move things along in a different direction here:

https://www.postgresql.org/message-id/11546.1566584867%40sss.pgh.pa.us

Please file a FESCo exception with that info. It seems reasonable to keep the Python 2 support for now. But do consider at least pointing "plpythonu" to Python 3 in v13 (upstream, preferably).

While we're aggressively trimming the python2 stack in f32, we will need to keep the interpreter itself for some more time, and postgresql doesn't require any additional Python libraries.
It would be much better to move with upstream here.


> OT: did python3 fix [the lack of any reliable sandboxing feature]?

No, AFAIK Python devs would consider a sandbox out of scope for the language/runtime. I'm not really aware of requests/efforts to add one.

I agree it would be best to align with upstream on this. Opened up a ticket asking for exception from FESCO: https://pagure.io/fesco/issue/2217

As a side note, can you please rename postgresql-plpython to postgresql-plpython2 and virtually provide postgresql-plpython from either postgresql-plpython2 or postgresql-plpython3 as you see fit? That way, people installing postgresql-plpython will immediately see what Python version that is.

What about postgresql-upgrade?

>As a side note, can you please rename postgresql-plpython to postgresql-plpython2 and virtually provide postgresql-plpython from either postgresql-plpython2 or postgresql-plpython3 as you see fit?

Makes sense, will do.

>What about postgresql-upgrade?

Same thing really - postgresql-upgrade contains a stripped down version of the postgresql server from which the upgrade is performed (so for postgresql v11, postgresql-upgrade contains v10). This version of the postgresql server has to be able to work with all extensions and language handlers we have shipped in the previous version of the rpm so that the upgrade goes through as smoothly as possible for the user performing the upgrade.
The tricky thing will be (as mentioned by Pavel in comment 6) how to perform the upgrade from plpython2 to future default plpython3.

FEDORA-2019-bad65b3890 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-bad65b3890

https://src.fedoraproject.org/rpms/postgresql/c/5c5f24923929d12a86604e03774fca9a9ad1d623?branch=master
https://src.fedoraproject.org/rpms/postgresql/c/3de7e5024ce09bce9d346ef6e40898a101b46b54?branch=master

Though I'm not sure the change is OK without explicit Obsoletes?

(In reply to Pavel Raiskup from comment #18)
> Though I'm not sure the change is OK without explicit Obsoletes?

No, it should be done with explicit Obsoletes: https://docs.fedoraproject.org/en-US/packaging-guidelines/#renaming-or-replacing-existing-packages

postgresql-11.5-3.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-bad65b3890

FEDORA-2019-6324af709b has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-6324af709b

Thanks for pointing that out. I've submitted new update with explicit Obsoletes.

postgresql-11.5-4.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-6324af709b

FEDORA-2019-815b0dee67 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-815b0dee67

postgresql-11.5-5.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-815b0dee67

postgresql-11.5-5.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.