Bug 603937

Created attachment 425348 [details]
First patch to get rid of a double bundling of Itpl.py

Itpl.py is present in two places in the ipython-0.10 source.  IPython/Itpl.py and IPython/external/Itpl.py.  This patch updates all the code to use IPython/external/Itpl.py and removes IPython/Itpl.py.  Glancing at the github repo, upstream is doing something similar for their next release.  They've moved IPython/Itpl.py to the IPython/deathrow/ directory and updated the source to point at external.Itpl just like this patch.

This is a patch that sanitizes the tree so the next patches are effective.

Created attachment 425353 [details]
Patch to make everything in external look for a system lib first

This patch modifies the modules in the IPython/external directory.  With it, system libs are tried before the bundled copies.  With this patch in place upstream can distribute the bundled libs but they won't be used if the lib is available on the system.

We can choose to remove the bundled libraries and require the system packages in the rpm instead.

Created attachment 425361 [details]
Alternate patch for importing system libs

This is an alternate patch for importing system libs instead of bundled libs.  It can be applied in addition to the previous patch but only one of the two is necessary.

The advantage of the previous patch is that we are sure of catching any instance where the bundled library is used.  The advantage of this patch is that it requires less work when adding a new bundled library.  Instead, all the work is done in the code that imports and makes use of a library that could either be bundled or on the system.

Thanks for working on this :)

Update to test will be there 'soon'.

There is an update at:
https://admin.fedoraproject.org/updates/ipython-0.10-3.fc13

I want to leave this open, so I have a blocker for the new review requests of the not yet unbundled libraries.

Still todo (for this version of ipython):
* guid.py - Not in fedora. Upstream website seems to be part of the activestate
cookbook:
  http://code.activestate.com/recipes/163604-guid/

-> Suggested to port to using the uuid module in python-2.5's stdlib and the
python-uuid packages in python2.3+

## This will be ported hopefully.

* path.py - not built in Fedora. Available in pypi with a seemingly dead
upstream:
  * http://pypi.python.org/pypi/path.py
  * I'd recommend porting to one of these replacements:
    - Based on path.py that is in Fedora and has a more active maintainer:
python-unipath:
      http://pypi.python.org/pypi/Unipath
    - And another replacement, not in Fedora:
      http://pypi.python.org/pypi/path3

## Will look at the porting before a review request (maybe you want to help here too) :)


This one will stay, because there is no real 'upstream'. This will be unbundled, once another program dublicates this file (currently there is none):
* Itpl.py - This is both in IPython/Itpl.py and IPython/external/Itpl.py.  Not
in Fedora. One file from here:
  http://lfw.org/python/
  * Was part of a rejected PEP. (python implemented this instead:
      http://docs.python.org/library/string.html#template-strings
    and also has this in py2.6 now:
      http://docs.python.org/library/string.html#format-string-syntax
  * This file has been modified a bit from the lfw.org version.  I'm not sure
if there's a different upstream location with these changes.

I noticed that the argparse bug in my patch that you pointed out on IRC was still in the new build.  I've fixed and submitted a new update with that fixed:
  https://admin.fedoraproject.org/updates/ipython-0.10-4.fc13

(In reply to comment #7)
> I noticed that the argparse bug in my patch that you pointed out on IRC was
> still in the new build.  I've fixed and submitted a new update with that fixed:
>   https://admin.fedoraproject.org/updates/ipython-0.10-4.fc13    

Oh, yes I saw that in CVS... Thanks for looking...

I already pushed another one after that out named 0.10-5, which also deletes 2 other libraries (the two reviews were quite fast ;)).

   -> https://admin.fedoraproject.org/updates/ipython-0.10-5.fc13


Could you please unpush your update again?

This bug appears to have been reported against 'rawhide' during the Fedora 14 development cycle.
Changing version to '14'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

The unbundle patch is now in upstream git, will look again at this, when the new version is available, what still needs to get unbundled...

(As a sidenote: python-pretty is inclueded and modified upstream now (so they basically adoped it)

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Update for IPython 1.1:

IPython/external/
├── argparse              -> upstream in python (2, 3)
│   ├── _argparse.py
│   └── __init__.py      
├── decorator             -> packaged (2, 3)
│   ├── _decorator.py
│   └── __init__.py      
├── decorators            -> part of numpy(2, 3)
│   ├── _decorators.py
│   ├── __init__.py
│   ├── _numpy_testing_noseclasses.py
│   └── _numpy_testing_utils.py
├── __init__.py
├── jsonpointer              -> packaged (2, 3)
│   ├── __init__.py
│   ├── _jsonpointer.py
│   └── VERSION.txt
├── jsonschema               -> packaged (2)
│   ├── COPYING
│   ├── __init__.py
│   ├── _jsonschema.py
│   └── VERSION
├── mathjax.py               -> not useful installer for mathjax
├── path                     -> packaged (2)
│   ├── __init__.py
│   └── _path.py
├── pexpect                  -> packaged (2, 3)
│   ├── __init__.py
│   └── _pexpect.py
├── simplegeneric            -> packaged (2, 3)
│   ├── __init__.py
│   └── _simplegeneric.py
├── qt_for_kernel.py         -> Those four are not bundling, just internal utilities, afaict.
├── qt_loaders.py            ->
├── qt.py                    ->
└── ssh                      ->
    ├── forward.py
    ├── __init__.py
    └── tunnel.py

Python-wise the situation looks OK.

mathjax itself is not packaged. Does this need to be packaged?

ipython-components:

jquery-ui: #858027
jquery: #857992
highlight:  ✔
requirejs:  ✔
marked:  ✔
less.js:  ✔
font-awesome: texlive-fontawesome is probably related, but it's not the same thing, i.e. missing
bootstrap: missing
codemirror: missing

I think that once we have jquery, the rest can be dealt with.

Thanks! So all in all the missing dependencies are:

(In reply to Zbigniew Jędrzejewski-Szmek from comment #13)
> ├── jsonschema               -> packaged (2)
> │   ├── COPYING
> │   ├── __init__.py
> │   ├── _jsonschema.py
> │   └── VERSION

for python3 see your bug #1016207

> ├── path                     -> packaged (2)
> │   ├── __init__.py
> │   └── _path.py

for python3 see bug #1016466

> ├── qt_for_kernel.py         -> Those four are not bundling, just internal
> utilities, afaict.
> ├── qt_loaders.py            ->
> ├── qt.py                    ->
> └── ssh                      ->
>     ├── forward.py
>     ├── __init__.py
>     └── tunnel.py
> 
> Python-wise the situation looks OK.

Besides the two python3 subpackages yes.

> mathjax itself is not packaged. Does this need to be packaged?

It's fine to require the user to install it on their system before using it with:
from IPython.external.mathjax import install_mathjax
install_mathjax()

Once it is packaged, we can add a Requires on it later on.

> ipython-components:
> 
> jquery-ui: #858027
> jquery: #857992

> font-awesome: texlive-fontawesome is probably related, but it's not the same
> thing, i.e. missing
> bootstrap: missing
> codemirror: missing
> 
> I think that once we have jquery, the rest can be dealt with.

Yeah, let's see how it evolves, as soon as jquery is done...

I have a mathjax package half-ready. I'll try to submit it for review soonish.

It seems there is only one python3 package missing and we have the python side ready for ipython-1.1.

But the notebook is bundling jquery a lot of less, codebirror, bootstrap etc...

I don't know if it is possible to get an exeption for that from fpc, but I don't *guess* so.

If that's indeed not the case, I'm afraid it will take some months (?years?), until we'll have a new ipython version available in the distribution...

Maybe some good soul could take on the review of zlib-js (bug #1024589)?

This needs a re-review.

The list here got completely irrelevant to ipython over time.

I'm closing this, feel free to reopen for notebook, but there we track bundled javascript libraries via Provides: bundled(...)