Bug 867962

A complaint I've heard at PyCon was that every distributor has its own way of subdividing python, and that this makes it difficult to know how to get all of python.  One suggestion there was that the "python" package should be a metapackage that brings in everything built from the source tarball (the full standard library).

This has the virtue of simplicity for an upstream: they can say "yum install python" and assume that everything is installed.

The current subdivision within the various %files stanzas of python.spec is:

  * python: /usr/bin/python (very small) and pydoc; manpages
  * python-libs: libpython.so.1 plus most of the standard library
  * python-devel: headers, pc files etc, the .exe template files for building standalone Windows executables from pure-Python code
  * python-tools: various /usr/bin things, including 2to3 and idle
  * tkinter
  * python-test: the entire test suite
  * python-debug:  all of the above is the optimized build; this has all of the above for a debug build

(There's also the implicit python-debuginfo, holding the DWARF debug data for both builds, and the source code)
  
So, for finer-grained dependencies, how about this subdivision:

  * "python" becomes a metapackage, requiring everything below (except "python-debug")
  * "python-core" is the core libpython.so libraries and interpreter, and some subset of the standard library, which could be e.g.:
     * "everything from the standard library that yum needs"
     * "everything from the standard library not listed below"
  * "python-libs" currently exists
  * "python-devel" as before (header files and .so symlinks)
  * "tkinter" as before
  * "python-idle" for /usr/bin/idle and its direct deps (maybe add a .desktop file?)
  * "python-2to3" for /usr/bin/2to3 and lib2to3
  * "python-test" as before (including tests for idle and 2to3)
  * "python-tools" for everything else that was in python-tools
  * "python-debug": all of the above is the optimized build; this has all of the above for a debug build

See also: http://lists.fedoraproject.org/pipermail/devel/2012-November/174170.html

Any changes to "python" should have corresponding changes to "python3": the two packages should have analogous subpackage organization.

Finer grained divisions may be okay -- from an end user perspective, I think they're fine.  From a packager perspective, they're going to require ongoing maintainance.  Whenever there's a new release, we're going to have to manually check whether subset A of the stdlib now requires subset B of the stdlib and whether that causes us to import them or not.

I have seen 2to3 used in the wild as an import so organizationally, a split that leaves it out in the cold doesn't seem right.  I don't have the same anecdotal evidence for libidle.

We can't put the python interpreter into the same package as the libpython.so library because of multilib.

If we're going to break the packages down, I'd do it by breaking python-libs from the stdlib again, like this:
  python-libs: libpython.so.1
  python-stdlib-SUBSET: subset of the stdlib

subsets could be organized along module boundaries or to optimize for the minimal environment.  I wouldn't mix those.  Module boundaries would look like this:
  python-stdlib-libidle
  python-stdlib-lib2to3
  python-stdlib-argparse
  python-stdlib-core

python-stdlib-core would be a catchall for every module that we didn't split out.

Optimized for the minimal environment would look like this:
  python-stdlib-minimal -- targeted inclusion of module required by yum and deps.
  python-stdlib -- everything else

I think this would be harder to achieve as it requires scanning a huge number of python modules in multiple packages.  python-stdlib would Require: python-stdlib-minimal and would be what is normally installed on systems.  Requires within the yum and python stacks would need to be careful to Require: python-stdlib-minimal instead.

Thinking about Requires... We'd have to be careful here.  We have several things we require now that we'd want to continue to mean more or less the same things and also some things that we wouldn't want to obey the standard behaviour:

* python(abi), python, python-devel, python2-devel, all currently drag in every runtime piece of the stdlib.  python(abi) is added automatically to any package that uses python.  We could write macros to disable that for yum and all of its deps or we could modify python(abi) to not include extra things.  Doing the latter, we probably would not want to use a python-stdlib-minimal strategy as that would be a very large change.  We could use the module-division strategy and just be very circumspect about which modules are left out of python-stdlib-core.
* Continuing to assume that we'd just break out a few packages into their own python-stdlib-module packages; some packages will need to be updated to either Require: or BuildRequire: python-stdlib-2to3.
* Metapackage: If we want "python" to be a metapackage that means everything we'll need to do several other thing:
  * Have a python-interpreter subpackage.
  * Find things that have an explicit dep on python and see if they can be removed.  (python(abi) will be better for libraries... possibly python-interpreter + python-stdlib-core for applications)
  * python(abi) will have to move to a different package since Requires: python will mean "everything" and Requires: python(abi) will mean "interpreter, libs, stdlib-core"
* We may want to have more deps than are strictly necessary.  technically, libpython.so can exist on its own.  /usr/bin/python only needs libpython.  But having the python-stdlib-core subset of the stdlib pulled in by one of those (currently, it's pulled whenever libpython is) would be more expected.

(In reply to comment #2)
> If we're going to break the packages down, I'd do it by breaking python-libs
> from the stdlib again, like this:
>   python-libs: libpython.so.1
>   python-stdlib-SUBSET: subset of the stdlib
> 
> subsets could be organized along module boundaries or to optimize for the
> minimal environment.  I wouldn't mix those.  Module boundaries would look
> like this:
>   python-stdlib-libidle
>   python-stdlib-lib2to3
>   python-stdlib-argparse
>   python-stdlib-core
> 
> python-stdlib-core would be a catchall for every module that we didn't split
> out.
> 
> Optimized for the minimal environment would look like this:
>   python-stdlib-minimal -- targeted inclusion of module required by yum and
> deps.
>   python-stdlib -- everything else

python-stdlib should also require all subsets so python-stdlib installs all python modules. Or how do you want to archive a full stdlib install?

I don't like the idea of "minimal is whatever yum needs". That can change from release and should be handled by proper requires by yum instead.

How about a python-minimal (or python-stdlib-minimal) to get a working interpreter with basic library stuff like "os/system" etc to get a program working without heavy stdlib dependencies?

I suggest looking for guidance of upstream. Perl upstream also didn't like how distribution break the whole tar ball into sub-packages. They are planning to provide guidelines for packaging. Someone will complain about minimal anyway...

This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Hello, new package maintainer! Any thoughts on this?

In python3-3.4, upstream is making clear that idle evolves separately from the stdlib (so that libidle is able to break api and idle can get bugfixes within a minor release that could not go into the stdlib).  With that philosophy defined upstream, it seems like /usr/bin/idle and libidle could be broken out into their own subpackage in our python2 packaging as well.  (perhaps one "idle" subpackage since it's primarily used as an application.)

Still highly against breaking out 2to3 all on its lonesome as there is actual code that expects to be able to import it.

FWIW iirc Debian have a .desktop file for /usr/bin/idle, so it appears in the menus

(In reply to comment #9)
> Still highly against breaking out 2to3 all on its lonesome as there is
> actual code that expects to be able to import it.

I can live with that -- still a win.

Created attachment 746295 [details]
.desktop file from the debian package

Here's the idle.desktop file from Debian.

Note that it references idle.xpm which is a symlink to python.xpm.  I haven't looked into what that is; it's not present in Fedora.

A quick web search doesn't  find an independent logo for idle.

On Debian, the idle subpackage is called "idle".  It makes sense for us to name it the same (as noted previously, idle is most prominently an application, rather than a library).

This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.