Bug 1287556 - Downstream only patches in Python3
Summary: Downstream only patches in Python3
Keywords:
Status: CLOSED DUPLICATE of bug 2133847
Alias: None
Product: Fedora
Classification: Fedora
Component: python3.11
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Python Maintainers
QA Contact:
URL:
Whiteboard:
Depends On: 1287566 1291325 1292461 1292462 1292467 1301033 1318565 1874803
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-12-02 10:46 UTC by Robert Kuska
Modified: 2022-10-11 15:20 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-10-11 15:13:44 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Python 43103 0 None None None 2021-02-02 16:22:07 UTC
Red Hat Bugzilla 2133847 0 low CLOSED Downstream-only patch in Python 3: #001 rpath 2022-11-17 00:41:35 UTC
Red Hat Bugzilla 2133849 0 low NEW Downstream-only patch in Python 3: #251 Change user install location 2023-08-02 12:07:04 UTC
Red Hat Bugzilla 2133850 0 unspecified CLOSED Downstream-only patch in Python 3: #328 Restore pyc to TIMESTAMP invalidation mode as default in rpmbuild 2023-01-06 14:23:24 UTC
Red Hat Bugzilla 2133853 0 unspecified CLOSED Downstream-only patch in Python 3: #371 Revert "bpo-1596321: Fix threading._shutdown() for the main thread (GH-28549) (G... 2024-05-02 19:03:28 UTC

Internal Links: 2133847 2133849 2133850

Description Robert Kuska 2015-12-02 10:46:24 UTC
This is a tracking bug for status of downstream only patches in Python3 in Fedora.


I've decided to go through all patches we use in Python3 and weren't yet upstreamed to figure out if there is still an usage for them.

Comment 1 Jan Kurik 2016-02-24 14:04:51 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 24 development cycle.
Changing version to '24'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora24#Rawhide_Rebase

Comment 3 Fedora End Of Life 2017-07-25 19:35:44 UTC
This message is a reminder that Fedora 24 is nearing its end of life.
Approximately 2 (two) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 24. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '24'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 24 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 4 Jan Kurik 2017-08-15 07:00:16 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle.
Changing version to '27'.

Comment 5 Ben Cotton 2018-11-27 17:26:12 UTC
This message is a reminder that Fedora 27 is nearing its end of life.
On 2018-Nov-30  Fedora will stop maintaining and issuing updates for
Fedora 27. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora  'version' of '27'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 27 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 6 Ben Cotton 2019-02-19 17:11:21 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 30 development cycle.
Changing version to '30.

Comment 7 Petr Viktorin (pviktori) 2020-01-07 14:18:40 UTC
BTW, Some info on patches might be found on https://fedoraproject.org/wiki/SIGs/Python/PythonPatches

Comment 8 Victor Stinner 2020-02-04 13:28:43 UTC
The python39 package currently (python39-3.9.0~a3-1.fc32) have 9 patches. 2 of them are just temporary, fixing 3.9.0a3 regressions:

* 00340-makefile-missing-test.patch
* 00341-bpo-39460.patch

The interesting part are these 7 patches:

* 00001-rpath.patch: "Fixup distutils/unixccompiler.py to remove standard library path from rpath"
  => Can be removed, but blocked by a Fedora change.
  This patch is basically useless, but removing it would require to ensure that no Fedora package use rpath.
  See: https://pagure.io/packaging-committee/issue/886
  Added by: https://src.fedoraproject.org/rpms/python/c/f5df1f834310948b32407933e3b8713e1121105b?branch=master

* 00102-lib64.patch: Use /usr/lib64 instead of /usr/lib on 64-bit systems
  => Should be made upstream.
  Upstream issue: https://bugs.python.org/issue1294959
  Upstream PR 1: https://github.com/python/cpython/pull/11755 by Charalampos Stratakis
  Upstream PR 2: https://github.com/python/cpython/pull/8068 by Matěj Cepl
  Added by: https://src.fedoraproject.org/rpms/python/c/60b7a5f84fa3f6ea3d0d96fa99030a67a2aa018b

* 00111-no-static-lib.patch: Don't try to build a libpythonMAJOR.MINOR.a
  "Downstream only: not appropriate for upstream."

* 00189-use-rpm-wheels.patch: instead of bundled wheels, use our RPM packaged wheels. We keep them in /usr/share/python-wheels.
  => downstream only patch

* 00251-change-user-install-location.patch: Change user install location to /usr/local if executable is /usr/bin/python* and RPM build is not detected to make pip and distutils install into separate location.
  => downstream only patch
  Added by: https://src.fedoraproject.org/rpms/python3/c/332b947dfc8d0f0d3a4525864b121d0f239beb4d

* 00274-fix-arch-names.patch: Upstream uses Debian-style architecture naming, change to match Fedora
  => downstream only patch
  Added by: https://src.fedoraproject.org/rpms/python3/c/9423d36fb4847cc6117daea82490a188cc78ebdf

* 00328-pyc-timestamp-invalidation-mode.patch: Restore pyc to TIMESTAMP invalidation mode as default in rpmbuild
  => downstream only patch
  Rationale: https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/57#comment-27426
  Added by: https://src.fedoraproject.org/rpms/python3/c/9213c03924aeb612d1b754dbe0b4397782c6b3fb

Comment 9 Miro Hrončok 2020-02-04 13:45:21 UTC
* 00102-lib64.patch: Use /usr/lib64 instead of /usr/lib on 64-bit systems
  => Should be made upstream.
  Upstream issue: https://bugs.python.org/issue1294959
  Upstream PR 1: https://github.com/python/cpython/pull/11755 by Charalampos Stratakis
  Upstream PR 2: https://github.com/python/cpython/pull/8068 by Matěj Cepl
  Added by: https://src.fedoraproject.org/rpms/python/c/60b7a5f84fa3f6ea3d0d96fa99030a67a2aa018b

This was introduced much sooner, but is not in git sadly :( https://src.fedoraproject.org/rpms/python/c/d4a39597bc32ab2cf84b43a3b0c71a18d1a0e0ec





* 00189-use-rpm-wheels.patch: instead of bundled wheels, use our RPM packaged wheels. We keep them in /usr/share/python-wheels.
  => downstream only patch

* 00251-change-user-install-location.patch: Change user install location to /usr/local if executable is /usr/bin/python* and RPM build is not detected to make pip and distutils install into separate location.
  => downstream only patch

We can strive to have upstream support for those.




* 00328-pyc-timestamp-invalidation-mode.patch: Restore pyc to TIMESTAMP invalidation mode as default in rpmbuild
  => downstream only patch

I still need to figure out why this was chosen upstream. There are IMHO batter ways to handle reproducibility here than restoring to a slower hash checking.

Comment 10 Victor Stinner 2020-02-04 13:51:24 UTC
I created https://fedoraproject.org/wiki/SIGs/Python/UpstreamPythonPatches page to track downstream pages.

Comment 11 Victor Stinner 2021-02-02 16:22:07 UTC
> 00111-no-static-lib.patch: Don't try to build a libpythonMAJOR.MINOR.a
> "Downstream only: not appropriate for upstream."

I created bpo-43103 "Add configure option to disable build libpython.a and don't install python.o" to upstream this change:
https://bugs.python.org/issue43103

Comment 12 Victor Stinner 2021-06-23 09:43:15 UTC
I updated https://fedoraproject.org/wiki/SIGs/Python/UpstreamPythonPatches

In short, the Fedora package only has 3 downstream patches. 

* 00001-rpath.patch
* 00251-change-user-install-location.patch
* 00328-pyc-timestamp-invalidation-mode.patch

The first two are waiting for other changes. So for now, the most important one is 00328-pyc-timestamp-invalidation-mode.patch.

Comment 13 Petr Viktorin (pviktori) 2021-06-23 09:54:55 UTC
Removing 00001-rpath.patch makes numpy.distutils build Numpy (and some other packages) with wrong rpaths.
numpy.distutils patches distutils, and since distutils is going away I expect some upstream changes in this area. I'd wait the ~2 years before trying to remove patch 00001 again.

Comment 14 Petr Viktorin (pviktori) 2021-08-18 10:07:59 UTC
00251-change-user-install-location.patch
I think this should be the next focus in this patch. It needs to be handled together with distutils removal, upstream and with input from other distros.


00328-pyc-timestamp-invalidation-mode.patch
I think it would be best to drive a larger effort here: for code that's distributed & installed rather than currently being developed, Python should prefer .pyc files and only load .py when it needs the source (e.g. for showing tracebacks). This ties in with the minimization effort (there are many ideas, but IMO we should make it possible to have *.pyc with __pysource__/*.py, rather than the current *.py with __pycache__/*.pyc).
pyc invalidation mode should be part of that discussion, when we get time to tackle it; before that I think a patch is OK.

Comment 15 Petr Viktorin (pviktori) 2021-08-26 09:44:53 UTC
For 00251-change-user-install-location.patch:

PEP 668 ( https://www.python.org/dev/peps/pep-0668/ ) was written to handle cases like this; we should ensure it solves all our problems.

Comment 16 Miro Hrončok 2021-10-12 09:10:28 UTC
00251-change-user-install-location.patch was changed considerably: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/AAGUFQZ4RZDU7KUN4HA43KQJCMSFR3GW/

Comment 17 Petr Viktorin (pviktori) 2022-09-14 12:10:33 UTC
We now also have 00371-revert-bpo-1596321-fix-threading-_shutdown-for-the-main-thread-gh-28549-gh-28589.patch
which reverts https://github.com/python/cpython/pull/28549

Comment 18 Victor Stinner 2022-09-14 12:15:44 UTC
> We now also have 00371-revert-bpo-1596321-fix-threading-_shutdown-for-the-main-thread-gh-28549-gh-28589.patch

The revert was motivated by this FreeIPA+mod_wsgi issue: https://github.com/GrahamDumpleton/mod_wsgi/issues/730

mod_wsgi shutdown was modified in version 4.9.1 to better handle Python 3.9 shutdown: https://modwsgi.readthedocs.io/en/master/release-notes/version-4.9.1.html#features-changed

Comment 19 Tomas Orsava 2022-10-11 15:13:44 UTC
This bug has been split into the following issues for each specific downstream-only patch:
- #001 rpath: BZ#2133847
- #251 Change user install location: BZ#2133849
- #328 Restore pyc to TIMESTAMP invalidation mode as default in rpmbuild: BZ#2133850

*** This bug has been marked as a duplicate of bug 2133847 ***

Comment 20 Tomas Orsava 2022-10-11 15:20:20 UTC
(In reply to Tomas Orsava from comment #19)
> This bug has been split into the following issues for each specific
> downstream-only patch:
> - #001 rpath: BZ#2133847
> - #251 Change user install location: BZ#2133849
> - #328 Restore pyc to TIMESTAMP invalidation mode as default in rpmbuild:
> BZ#2133850

- #371 Revert "bpo-1596321: Fix threading._shutdown() for the main thread (GH-28549) (GH-28589)": BZ#2133853


Note You need to log in before you can comment on or make changes to this bug.