Bug 784947

Summary: ipython support for python3
Product: [Fedora] Fedora Reporter: Andrew McNabb <amcnabb>
Component: ipythonAssignee: Thomas Spura <tomspur>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 18CC: anthony.horton, cristian.ciupitu, jefflund, kalevlember, liling, mrunge, shahms, tomspur
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-04-18 02:32:02 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 785032, 785056, 785301, 837156, 838172, 878612, 928326, 947655    
Bug Blocks:    

Description Andrew McNabb 2012-01-26 18:47:43 UTC
Starting with version 0.12, ipython supports Python 3:

http://wiki.ipython.org/index.php?title=Python_3

It seems appropriate to have a bug open to track progress. The ipython spec file seems to have some preliminary support for Python 3, but apparently some package dependencies are preventing this from progressing. Specifically, python3-mglob, python3-simplegeneric, python3-tornado, and python3-matplotlib are unavailable.

It looks like the list of dependencies in the spec file isn't quite complete. For example, in the "%if 0%{?with_python3}" block, it mentions pyparsing, pexpect, pymongo, PyQt4 instead of python3 equivalents. There are available packages for python3-pyparsing and python3-PyQt4, but I don't see a python3-pexpect or python3-pymongo.

Is packaging for dependencies the main problem right now? Would it be helpful to create bug reports for individual required packages?

Comment 1 Thomas Spura 2012-01-27 02:12:29 UTC
(In reply to comment #0)
> Starting with version 0.12, ipython supports Python 3:
> 
> http://wiki.ipython.org/index.php?title=Python_3
> 
> It seems appropriate to have a bug open to track progress. The ipython spec
> file seems to have some preliminary support for Python 3, but apparently some
> package dependencies are preventing this from progressing. Specifically,
> python3-mglob, python3-simplegeneric, python3-tornado, and python3-matplotlib
> are unavailable.
> 
> It looks like the list of dependencies in the spec file isn't quite complete.
> For example, in the "%if 0%{?with_python3}" block, it mentions pyparsing,
> pexpect, pymongo, PyQt4 instead of python3 equivalents. There are available
> packages for python3-pyparsing and python3-PyQt4, but I don't see a
> python3-pexpect or python3-pymongo.

Yes, didn't fully look into it yet and (as you have mentioned already), there are python3 dependencies missing so, it's not possible to build it yet.

> Is packaging for dependencies the main problem right now? Would it be helpful
> to create bug reports for individual required packages?

Yes and yes :)

AFAIS these packages are not yet here:
python3-mglob
python3-simplegeneric
python3-tornado
python3-pexpect
python3-matplotlib
python3-pymongo

Could become reeealy difficult for a python3-matplotlib as it has quite difficult dependencies...

Comment 2 Andrew McNabb 2012-01-27 16:54:37 UTC
So it turns out that IPython's bundled copy of pexpect is not actually pexpect. In fact, the upstream pexpect project does not seem to show many signs of life. The last release was in January 2008 (4 years ago), and the last commit was in October 2010 (over a year ago).

IPython is actually using pexpect-u:

http://pypi.python.org/pypi/pexpect-u
https://github.com/ipython/ipython/commits/master/IPython/external/pexpect/_pexpect.py

I suspect that the best thing to do is to package pexpect-u (with a python3-pexpect-u subpackage) and have ipython depend on that instead of pexpect. Any thoughts?

Comment 3 Andrew McNabb 2012-01-27 17:37:40 UTC
I just added bug #785227 as a dependency. After looking into this, I'm convinced that the ideal solution is to have the pexpect subpackage switch to the pexpect-u source, since both provide a module named "pexpect". If pexpect-u were to be added as a separate package, then it would conflict with the current pexpect package.

Comment 4 Andrew McNabb 2012-01-27 21:22:40 UTC
It turns out that matplotlib has Python 3 support in trunk but not in release 1.1.0. When they say IPython 0.12 supports Python 3, I don't know whether that means that it requires a development version of matplotlib or that some functionality will be missing until the next version of matplotlib is released. Either way, it looks like it might be problematic.

Comment 5 Thomas Spura 2012-01-29 01:41:05 UTC
(In reply to comment #4)
> It turns out that matplotlib has Python 3 support in trunk but not in release
> 1.1.0. When they say IPython 0.12 supports Python 3, I don't know whether that
> means that it requires a development version of matplotlib or that some
> functionality will be missing until the next version of matplotlib is released.
> Either way, it looks like it might be problematic.

AFAIU, ipython works with the python3 interpreter, but there might be dependency problems...

When you build the latest commit from ipython [1], ipython3 "works" mostly, but when they don't, ipython3 fails badly, e.g. when typing "pylab" interactively.

I'm unsure, if it would be ok, to push a ipython3 package out there, without having all dependencies fullfilled...

[1] http://pkgs.fedoraproject.org/gitweb/?p=ipython.git;a=commit;h=e290d79b54335c070de2b49ad206fae80364be08

Comment 6 Thomas Spura 2012-01-29 01:45:07 UTC
(In reply to comment #2)
> I suspect that the best thing to do is to package pexpect-u (with a
> python3-pexpect-u subpackage) and have ipython depend on that instead of
> pexpect. Any thoughts?

+1 :)
But let's wait for an answer from the pexpect maintainer.

For now ipython3 will bundle pexpect or the testsuite will fail badly and completely...

Remaining deps for now:
- python3-tornado
  pycurl seems dead, but python3-tornado seems to work otherwise (bug #785301)
- python3-pexpect
  see above
- python3-matplotlib
  next release maybe needed
- python3-pymongo
  next release absolutely needed:
  https://jira.mongodb.org/browse/PYTHON-84

Comment 7 Ling Li 2012-07-06 15:07:04 UTC
pymongo 2.2 (released in Apr 2012) supports Python 3.
python3-matplotlib is tracked in Bug 753210 (?).

Comment 8 Thomas Spura 2012-07-06 22:28:10 UTC
(In reply to comment #6)
> pymongo 2.2 (released in Apr 2012) supports Python 3.

Thanks!
Tracked in bug #838172

> python3-matplotlib is tracked in Bug 753210 (?).

Yes.

It seems like pexpect and matplotlib are the last remaining ones.
I'd like to restructure the ipython package structure quite a bit, let's see, if it's possible to have python3-ipython nevertheless just with limited functionality. But could take a while, till I get to it...

Comment 9 Andrew McNabb 2012-11-20 17:07:33 UTC
I'll see if I can make some progress on pexpect today.

Comment 10 Andrew McNabb 2012-11-20 17:08:18 UTC
Also, it looks like matplotlib 1.2.0 was released on November 8 with support for Python 3.

Comment 11 Andrew McNabb 2012-11-20 19:06:03 UTC
I've now created a package review for a new python-pexpect module that supports Python 3. This package review is bug #878612. Any help in reviewing would be appreciated. Thanks.

Comment 12 Andrew McNabb 2012-11-21 14:41:09 UTC
The python3-pexpect should now be available, so I think the python3-pymongo and python3-matplotlib packages are the last two dependencies.

Comment 13 Andrew McNabb 2012-11-26 21:15:41 UTC
I have added a new review request for a new version of pymongo that supports Python 3. This review request is bug #880387. Any assistance in reviewing would be appreciated. Thanks.

Comment 14 Andrew McNabb 2012-11-27 23:50:58 UTC
There are now python3 packages available in koji for python3-pymongo and python3-matplotlib. The final packages should be pushed some time in a few days.

It should now be much easier for those who are working on python3-ipython to do local builds.

Comment 15 Andrew McNabb 2012-12-05 20:37:28 UTC
I have successfully built ipython for Python 3 in Fedora 18.

Spec file: http://aml.cs.byu.edu/~amcnabb/ipython.spec
Patch for spec file: http://aml.cs.byu.edu/~amcnabb/ipython.spec.patch
SRPM: http://aml.cs.byu.edu/~amcnabb/ipython-0.13.1-2.fc19.src.rpm

I also have this currently building in koji:

http://koji.fedoraproject.org/koji/taskinfo?taskID=4760855

If this build completes successfully, then I am not aware of any outstanding problems with this package.

Comment 16 Fedora End Of Life 2013-01-16 20:20:08 UTC
This message is a reminder that Fedora 16 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 16. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '16'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 16's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 16 is end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" and open it against that version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 17 Thomas Spura 2013-02-21 15:57:31 UTC
A few comments:
- There was a typo in "Requires:       python3-ipython-%{name} = %{version}-%{release}".
- Why do we need python-matplotlib-tk? Anything should be as expected without those dependency isn't it?
- I don't like patching X tests completely out.
  Furthermore, thermporarily disabling them when building is fine, but completely patching them out is not so good...
  I'd probably just ignore the failure

I uncommented matplotlib-tk, corrected the first one in git and skip the tests differently.

Furthermore, I restructured the python3-ipython subpackages to match the python2-ipython ones.




As these were quite huge changes, I'd like to have another pair of eyes on it to verify it once more. But we should be ready now to ship python3-ipython [soon]... :)

Comment 18 Andrew McNabb 2013-02-21 17:29:35 UTC
(In reply to comment #17)
> - Why do we need python-matplotlib-tk? Anything should be as expected
> without those dependency isn't it?

It's been long enough that I don't remember, so I'm fine with leaving it out if everything works.

> - I don't like patching X tests completely out.

I tried to exclude them individually on the command-line, but I never actually got exclusion to work. I'm pretty sure that the way that iptest is defined, this isn't possible, so patching them out was the only solution I could come up with.

In either case, it would probably be good to include a comment stating why the tests are expected to fail (i.e., the dependency on X) so that there's no mystery for people looking at the spec file in the future.

>   Furthermore, thermporarily disabling them when building is fine, but
> completely patching them out is not so good...
>   I'd probably just ignore the failure

You mean to specify that certain tests are expected to fail, right? This sounds okay, if you can figure out how to make it work. I never got this to work.

> As these were quite huge changes, I'd like to have another pair of eyes on
> it to verify it once more. But we should be ready now to ship
> python3-ipython [soon]... :)

I've looked through the changes, and everything looks good. I'll try installing it soon to see if everything works.

Comment 19 Kalev Lember 2013-02-27 15:39:29 UTC
I would love to have a Python 3 build of ipython for F19. Accerciser has been ported to Python 3 and requires ipython-console.

Comment 20 Kalev Lember 2013-03-27 00:10:52 UTC
I see that you've enabled Python 3 support in git, but the build has failed because of failed Python 3 tests. Could you disable the Python 3 tests for now, just to get somewhere?

For what it's worth, a locally rebuilt python3-ipython-console-0.13.1 seems to work fine here.

Comment 21 Thomas Spura 2013-04-03 00:23:48 UTC
I'd prefer to know, that it works too...

It seems we are pretty close at it :)

Comment 22 Fedora Update System 2013-04-06 16:55:59 UTC
ipython-0.13.2-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/ipython-0.13.2-1.fc19

Comment 23 Fedora Update System 2013-04-06 16:56:26 UTC
ipython-0.13.2-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/ipython-0.13.2-1.fc18

Comment 24 Kalev Lember 2013-04-06 18:00:23 UTC
Awesome, thanks for making it happen.

I'll give it a try when it hits updates-testing and leave karma to the F19 update (I only need it for F19+).

Comment 25 Fedora Update System 2013-04-07 00:34:23 UTC
Package ipython-0.13.2-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing ipython-0.13.2-1.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5070/ipython-0.13.2-1.fc18
then log in and leave karma (feedback).

Comment 26 Fedora Update System 2013-04-18 02:32:04 UTC
ipython-0.13.2-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 27 Fedora Update System 2013-04-25 14:04:09 UTC
ipython-0.13.2-1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.