This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours
Bug 603937 - ipython bundles libraries
ipython bundles libraries
Status: ASSIGNED
Product: Fedora
Classification: Fedora
Component: ipython (Show other bugs)
rawhide
All Linux
low Severity medium
: ---
: ---
Assigned To: Thomas Spura
Fedora Extras Quality Assurance
: FutureFeature
Depends On: 858027 605993 605998 1016466 1016677 js-jquery1
Blocks:
  Show dependency treegraph
 
Reported: 2010-06-14 18:36 EDT by Toshio Ernie Kuratomi
Modified: 2014-09-06 10:24 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
First patch to get rid of a double bundling of Itpl.py (15.50 KB, patch)
2010-06-19 11:06 EDT, Toshio Ernie Kuratomi
no flags Details | Diff
Patch to make everything in external look for a system lib first (611.51 KB, patch)
2010-06-19 12:22 EDT, Toshio Ernie Kuratomi
no flags Details | Diff
Alternate patch for importing system libs (15.40 KB, patch)
2010-06-19 13:51 EDT, Toshio Ernie Kuratomi
no flags Details | Diff

  None (edit)
Description Toshio Ernie Kuratomi 2010-06-14 18:36:13 EDT
Description of problem:
ipython is bundling several libraries.  There do not seem to be major changes to the libraries but not all of them have been packaged for Fedora yet.

* argparse.py - in python-argparse package

* configobj.py - in python-configobj package

* guid.py - Not in fedora. Upstream website seems to be part of the activestate cookbook:
  http://code.activestate.com/recipes/163604-guid/

* Suggested to port to using the uuid module in python-2.5's stdlib and the python-uuid packages in python2.3+

* Itpl.py - This is both in IPython/Itpl.py and IPython/external/Itpl.py.  Not in Fedora. One file from here:
  http://lfw.org/python/
  * Was part of a rejected PEP. (python implemented this instead:
      http://docs.python.org/library/string.html#template-strings
    and also has this in py2.6 now:
      http://docs.python.org/library/string.html#format-string-syntax
  * This file has been modified a bit from the lfw.org version.  I'm not sure if there's a different upstream location with these changes.

* mglob.py - not built in Fedora. Available in pypi:
  http://pypi.python.org/pypi/mglob

* path.py - not built in Fedora. Available in pypi with a seemingly dead upstream:
  * http://pypi.python.org/pypi/path.py
  * I'd recommend porting to one of these replacements:
    - Based on path.py that is in Fedora and has a more active maintainer: python-unipath:
      http://pypi.python.org/pypi/Unipath
    - And another replacement, not in Fedora:
      http://pypi.python.org/pypi/path3

* pretty.py - not built in Fedora. Available in pypi:
  http://pypi.python.org/pypi/pretty/

* simplegeneric.py - in Fedora: python-simplegeneric package

* validate.py - in Fedora.  Part of python-configobj

I'll attach a few patches to this bug to make it so system versions of the libraries are preferred.  We'll still need to do some packaging to get the extra libraries into Fedora and decide whether to remove the things in externals when shipping the package.
Comment 1 Toshio Ernie Kuratomi 2010-06-19 11:06:40 EDT
Created attachment 425348 [details]
First patch to get rid of a double bundling of Itpl.py

Itpl.py is present in two places in the ipython-0.10 source.  IPython/Itpl.py and IPython/external/Itpl.py.  This patch updates all the code to use IPython/external/Itpl.py and removes IPython/Itpl.py.  Glancing at the github repo, upstream is doing something similar for their next release.  They've moved IPython/Itpl.py to the IPython/deathrow/ directory and updated the source to point at external.Itpl just like this patch.

This is a patch that sanitizes the tree so the next patches are effective.
Comment 2 Toshio Ernie Kuratomi 2010-06-19 12:22:00 EDT
Created attachment 425353 [details]
Patch to make everything in external look for a system lib first

This patch modifies the modules in the IPython/external directory.  With it, system libs are tried before the bundled copies.  With this patch in place upstream can distribute the bundled libs but they won't be used if the lib is available on the system.

We can choose to remove the bundled libraries and require the system packages in the rpm instead.
Comment 3 Toshio Ernie Kuratomi 2010-06-19 13:51:07 EDT
Created attachment 425361 [details]
Alternate patch for importing system libs

This is an alternate patch for importing system libs instead of bundled libs.  It can be applied in addition to the previous patch but only one of the two is necessary.

The advantage of the previous patch is that we are sure of catching any instance where the bundled library is used.  The advantage of this patch is that it requires less work when adding a new bundled library.  Instead, all the work is done in the code that imports and makes use of a library that could either be bundled or on the system.
Comment 4 Thomas Spura 2010-06-19 15:26:32 EDT
Thanks for working on this :)

Update to test will be there 'soon'.
Comment 5 Thomas Spura 2010-06-19 16:21:06 EDT
There is an update at:
https://admin.fedoraproject.org/updates/ipython-0.10-3.fc13

I want to leave this open, so I have a blocker for the new review requests of the not yet unbundled libraries.
Comment 6 Thomas Spura 2010-06-19 17:11:05 EDT
Still todo (for this version of ipython):
* guid.py - Not in fedora. Upstream website seems to be part of the activestate
cookbook:
  http://code.activestate.com/recipes/163604-guid/

-> Suggested to port to using the uuid module in python-2.5's stdlib and the
python-uuid packages in python2.3+

## This will be ported hopefully.

* path.py - not built in Fedora. Available in pypi with a seemingly dead
upstream:
  * http://pypi.python.org/pypi/path.py
  * I'd recommend porting to one of these replacements:
    - Based on path.py that is in Fedora and has a more active maintainer:
python-unipath:
      http://pypi.python.org/pypi/Unipath
    - And another replacement, not in Fedora:
      http://pypi.python.org/pypi/path3

## Will look at the porting before a review request (maybe you want to help here too) :)


This one will stay, because there is no real 'upstream'. This will be unbundled, once another program dublicates this file (currently there is none):
* Itpl.py - This is both in IPython/Itpl.py and IPython/external/Itpl.py.  Not
in Fedora. One file from here:
  http://lfw.org/python/
  * Was part of a rejected PEP. (python implemented this instead:
      http://docs.python.org/library/string.html#template-strings
    and also has this in py2.6 now:
      http://docs.python.org/library/string.html#format-string-syntax
  * This file has been modified a bit from the lfw.org version.  I'm not sure
if there's a different upstream location with these changes.
Comment 7 Toshio Ernie Kuratomi 2010-06-21 18:14:57 EDT
I noticed that the argparse bug in my patch that you pointed out on IRC was still in the new build.  I've fixed and submitted a new update with that fixed:
  https://admin.fedoraproject.org/updates/ipython-0.10-4.fc13
Comment 8 Thomas Spura 2010-06-22 16:37:34 EDT
(In reply to comment #7)
> I noticed that the argparse bug in my patch that you pointed out on IRC was
> still in the new build.  I've fixed and submitted a new update with that fixed:
>   https://admin.fedoraproject.org/updates/ipython-0.10-4.fc13    

Oh, yes I saw that in CVS... Thanks for looking...

I already pushed another one after that out named 0.10-5, which also deletes 2 other libraries (the two reviews were quite fast ;)).

   -> https://admin.fedoraproject.org/updates/ipython-0.10-5.fc13


Could you please unpush your update again?
Comment 9 Bug Zapper 2010-07-30 08:07:34 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 14 development cycle.
Changing version to '14'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 10 Thomas Spura 2011-04-09 05:01:56 EDT
The unbundle patch is now in upstream git, will look again at this, when the new version is available, what still needs to get unbundled...

(As a sidenote: python-pretty is inclueded and modified upstream now (so they basically adoped it)
Comment 11 Fedora Admin XMLRPC Client 2011-11-02 03:56:47 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 12 Fedora Admin XMLRPC Client 2011-11-02 07:11:27 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 13 Zbigniew Jędrzejewski-Szmek 2013-10-07 14:11:23 EDT
Update for IPython 1.1:

IPython/external/
├── argparse              -> upstream in python (2, 3)
│   ├── _argparse.py
│   └── __init__.py      
├── decorator             -> packaged (2, 3)
│   ├── _decorator.py
│   └── __init__.py      
├── decorators            -> part of numpy(2, 3)
│   ├── _decorators.py
│   ├── __init__.py
│   ├── _numpy_testing_noseclasses.py
│   └── _numpy_testing_utils.py
├── __init__.py
├── jsonpointer              -> packaged (2, 3)
│   ├── __init__.py
│   ├── _jsonpointer.py
│   └── VERSION.txt
├── jsonschema               -> packaged (2)
│   ├── COPYING
│   ├── __init__.py
│   ├── _jsonschema.py
│   └── VERSION
├── mathjax.py               -> not useful installer for mathjax
├── path                     -> packaged (2)
│   ├── __init__.py
│   └── _path.py
├── pexpect                  -> packaged (2, 3)
│   ├── __init__.py
│   └── _pexpect.py
├── simplegeneric            -> packaged (2, 3)
│   ├── __init__.py
│   └── _simplegeneric.py
├── qt_for_kernel.py         -> Those four are not bundling, just internal utilities, afaict.
├── qt_loaders.py            ->
├── qt.py                    ->
└── ssh                      ->
    ├── forward.py
    ├── __init__.py
    └── tunnel.py

Python-wise the situation looks OK.

mathjax itself is not packaged. Does this need to be packaged?

ipython-components:

jquery-ui: #858027
jquery: #857992
highlight:  ✔
requirejs:  ✔
marked:  ✔
less.js:  ✔
font-awesome: texlive-fontawesome is probably related, but it's not the same thing, i.e. missing
bootstrap: missing
codemirror: missing

I think that once we have jquery, the rest can be dealt with.
Comment 14 Thomas Spura 2013-10-08 04:42:26 EDT
Thanks! So all in all the missing dependencies are:

(In reply to Zbigniew Jędrzejewski-Szmek from comment #13)
> ├── jsonschema               -> packaged (2)
> │   ├── COPYING
> │   ├── __init__.py
> │   ├── _jsonschema.py
> │   └── VERSION

for python3 see your bug #1016207

> ├── path                     -> packaged (2)
> │   ├── __init__.py
> │   └── _path.py

for python3 see bug #1016466

> ├── qt_for_kernel.py         -> Those four are not bundling, just internal
> utilities, afaict.
> ├── qt_loaders.py            ->
> ├── qt.py                    ->
> └── ssh                      ->
>     ├── forward.py
>     ├── __init__.py
>     └── tunnel.py
> 
> Python-wise the situation looks OK.

Besides the two python3 subpackages yes.

> mathjax itself is not packaged. Does this need to be packaged?

It's fine to require the user to install it on their system before using it with:
from IPython.external.mathjax import install_mathjax
install_mathjax()

Once it is packaged, we can add a Requires on it later on.

> ipython-components:
> 
> jquery-ui: #858027
> jquery: #857992

> font-awesome: texlive-fontawesome is probably related, but it's not the same
> thing, i.e. missing
> bootstrap: missing
> codemirror: missing
> 
> I think that once we have jquery, the rest can be dealt with.

Yeah, let's see how it evolves, as soon as jquery is done...
Comment 15 Zbigniew Jędrzejewski-Szmek 2013-10-08 07:19:00 EDT
I have a mathjax package half-ready. I'll try to submit it for review soonish.
Comment 16 Thomas Spura 2014-02-11 06:23:50 EST
It seems there is only one python3 package missing and we have the python side ready for ipython-1.1.

But the notebook is bundling jquery a lot of less, codebirror, bootstrap etc...

I don't know if it is possible to get an exeption for that from fpc, but I don't *guess* so.

If that's indeed not the case, I'm afraid it will take some months (?years?), until we'll have a new ipython version available in the distribution...
Comment 17 Zbigniew Jędrzejewski-Szmek 2014-02-11 09:06:58 EST
Maybe some good soul could take on the review of zlib-js (bug #1024589)?

Note You need to log in before you can comment on or make changes to this bug.