Bug 735478 - python3 support for scipy (patch included)
Summary: python3 support for scipy (patch included)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: scipy
Version: 16
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Thomas Spura
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-09-02 18:34 UTC by Andrew McNabb
Modified: 2011-09-30 19:50 UTC (History)
5 users (show)

Fixed In Version: scipy-0.9.0-2.fc16
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-09-30 19:50:02 UTC


Attachments (Terms of Use)
patch to scipy.spec to support python3 (2.05 KB, patch)
2011-09-02 18:34 UTC, Andrew McNabb
no flags Details | Diff

Description Andrew McNabb 2011-09-02 18:34:13 UTC
Created attachment 521273 [details]
patch to scipy.spec to support python3

One of the things keeping me from using Python 3 on a daily basis is scipy, so I made a patch to scipy.spec to add a python3-scipy package. My change is based heavily on the Python 3 support in numpy.spec, so I hope there's nothing controversial here.

By the way, this does not work on Fedora 15.  It seems that python3-numpy-1.5.1 (which is in Fedora 15) fails to detect libraries correctly, but python3-numpy-1.6 (which is in Fedora 16) doesn't have any such problem.

I'm still testing (this package takes a long time to build), but I am posting it now because I would like to work on this quickly in the hope that it might make it in to Fedora 16. Please let me know if you have any thoughts or if there's anything else I should add.

Comment 1 Thomas Spura 2011-09-02 19:03:17 UTC
I'm just curious, what fails in numpy-1.5.1 so I'm building it on f15 to see the failure :)

%check is missing for python3 (currently untested):
%if 0%{?with_python3}
pushd %{py3dir}
mkdir test
cd test
PYTHONPATH=$RPM_BUILD_ROOT%{python3_sitearch} python3 -c "import scipy; scip    y.test('full')"
popd
%endif # with_python3

Comment 2 Andrew McNabb 2011-09-02 19:20:08 UTC
Stuff like this:

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib64/atlas
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib64/atlas
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib64/sse2
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib
  NOT AVAILABLE

The errors occur even though the libraries exist.  I found that I could produce the errors more directly by running the following after unpacking:

ATLAS=/usr/lib64/atlas BLAS=/usr/lib64 python3 setup.py config_fc --fcompiler=gnu95 --noarch build

I looked through the file /usr/lib64/python3.2/site-packages/numpy/distutils/system_info.py but I'm unfamiliar enough with this code that it would take a while to track down. Once I figured out that it's working in numpy 1.6, I figured it wasn't worth taking the time to track down a bug that's already been tracked down. :)

Comment 3 Andrew McNabb 2011-09-02 19:21:09 UTC
By the way, if to build on Fedora 15, you'll have to change the 15 to something else in the top line of the spec file, i.e.:

%if (0%{?fedora} > 14 || 0%{?rhel} > 5)

Comment 4 Thomas Spura 2011-09-02 20:09:30 UTC
(In reply to comment #2)
> /usr/lib64/python3.2/site-packages/numpy/distutils/system_info.py but I'm
> unfamiliar enough with this code that it would take a while to track down. Once
> I figured out that it's working in numpy 1.6, I figured it wasn't worth taking
> the time to track down a bug that's already been tracked down. :)

Yes, +1^^

Comments about your patch:
* python is called everywhere, so it would be better to just call python3 too.
* I added the test section for python3
* There was a problem with "summary" in the top line.

SPEC: http://tomspur.fedorapeople.org/scipy.spec
DIFF: http://tomspur.fedorapeople.org/scipy-diff2master

scratch build for f17:
http://koji.fedoraproject.org/koji/taskinfo?taskID=3321943

When the tests pass, I think it's ok to build it in rawhide and f16.
Andrew, thanks for testing on f16 :)

Other comments?

Comment 5 Andrew McNabb 2011-09-02 20:30:04 UTC
(In reply to comment #4)
> * python is called everywhere, so it would be better to just call python3 too.

As far as I can tell from the Python packaging guidelines, the %{__python3} macro is the preferred approach.  I think it might be better to switch the "python" calls to %{__python} than to get rid of the %{__python3} macros.

> * I added the test section for python3

Yes, I noticed this was missing after I posted the patch.  Your test section looks right to me.

> * There was a problem with "summary" in the top line.

I'm afraid I can't find the problem you're referring to.


> When the tests pass, I think it's ok to build it in rawhide and f16.
> Andrew, thanks for testing on f16 :)

Glad I can help.


> Other comments?

I can't think of anything else. Thanks for looking at this so quickly.

Comment 6 Thomas Spura 2011-09-02 21:16:34 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > * python is called everywhere, so it would be better to just call python3 too.
> 
> As far as I can tell from the Python packaging guidelines, the %{__python3}
> macro is the preferred approach.  I think it might be better to switch the
> "python" calls to %{__python} than to get rid of the %{__python3} macros.

I don't see where python3 is forbidden. It should be only a matter of taste like %{__rm} or rm (the first was needed, when /usr/bin was not in $PATH, but not anymore).

When Jef wants to change that, he's free to do it :)

> > * There was a problem with "summary" in the top line.
> 
> I'm afraid I can't find the problem you're referring to.

After applying your patch there was:
%else
Summary:
%endif

-> Summary was not always defined, but now is.

> 
> > When the tests pass, I think it's ok to build it in rawhide and f16.
> > Andrew, thanks for testing on f16 :)
> 
> Glad I can help.

Thanks! :)

There are several "ResourceWarning: unclosed file", where I'm unsure, if they'll cause problems.

This passes in f17:
http://koji.fedoraproject.org/koji/taskinfo?taskID=3321972

Here are the ResourceWarning from above:
http://koji.fedoraproject.org/koji/getfile?taskID=3321973&name=build.log

When you are sure, they are harmless, I'll go ahead and build it in f16 and f17, unless there is a objection of another (co-)maintainer.

SPEC: http://tomspur.fedorapeople.org/scipy.spec
DIFF: http://tomspur.fedorapeople.org/scipy-diff2master

Comment 7 Andrew McNabb 2011-09-03 10:43:42 UTC
(In reply to comment #6)
> 
> I don't see where python3 is forbidden. It should be only a matter of taste
> like %{__rm} or rm (the first was needed, when /usr/bin was not in $PATH, but
> not anymore).
> 
> When Jef wants to change that, he's free to do it :)

Fair enough. :)

> After applying your patch there was:
> %else
> Summary:
> %endif
> 
> -> Summary was not always defined, but now is.

The file I created the patch from doesn't have that problem, so I have no idea how that happened. Anyway, I'm glad you noticed it and it's fixed.


> There are several "ResourceWarning: unclosed file", where I'm unsure, if
> they'll cause problems.
> 
> This passes in f17:
> http://koji.fedoraproject.org/koji/taskinfo?taskID=3321972
> 
> Here are the ResourceWarning from above:
> http://koji.fedoraproject.org/koji/getfile?taskID=3321973&name=build.log
>
> When you are sure, they are harmless, I'll go ahead and build it in f16 and
> f17, unless there is a objection of another (co-)maintainer.

This seems to be a known problem with Scipy on Python 3.2. Scipy ticket #1385 has a comment saying that the warning is harmless, but it also has a patch: 

http://projects.scipy.org/scipy/ticket/1385

I suppose the two options are to ignore the warnings or to apply the patch.

Comment 8 Andrew McNabb 2011-09-03 10:44:25 UTC
(In reply to comment #7)
> 
> I suppose the two options are to ignore the warnings or to apply the patch.

By the way, my vote is to apply the patch, but I don't have strong feelings.

Comment 9 Thomas Spura 2011-09-03 11:36:41 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > Here are the ResourceWarning from above:
> > http://koji.fedoraproject.org/koji/getfile?taskID=3321973&name=build.log
> >
> > When you are sure, they are harmless, I'll go ahead and build it in f16 and
> > f17, unless there is a objection of another (co-)maintainer.
> 
> This seems to be a known problem with Scipy on Python 3.2. Scipy ticket #1385
> has a comment saying that the warning is harmless, but it also has a patch: 
> 
> http://projects.scipy.org/scipy/ticket/1385
> 
> I suppose the two options are to ignore the warnings or to apply the patch.

Thanks for the link.

The patch is already in 0.9, the ResourceWarnings are in nose, python3 but also some forgotten scipy files.

Will have a look soon.

Comment 10 Andrew McNabb 2011-09-08 18:00:28 UTC
Just pinging to see if you've had a chance to look at this or if you need any help.  By the way, it's not too late to get the package into Fedora 16, is it?  If it's in Fedora 16, I think I can get a bunch of people and stuff permanently moved over to Python 3, which would be great. :)

Comment 11 Thomas Spura 2011-09-09 07:47:58 UTC
(In reply to comment #10)
> Just pinging to see if you've had a chance to look at this or if you need any
> help.  By the way, it's not too late to get the package into Fedora 16, is it? 
> If it's in Fedora 16, I think I can get a bunch of people and stuff permanently
> moved over to Python 3, which would be great. :)

Yep, I think ResourceWarnings are ignorable and according to the python3 doc they should be disabled by default (but somewhere the warnings seem to be turned on).

I reported it to your scipy ticket above and that should be enought to get it fixed in a next release.

I plan to build & update it in devel and f16, just wanted to let co-maintainers a change to object, like already said in comment #6.

It's not to late for f16 ;)

Comment 12 Andrew McNabb 2011-09-09 16:10:01 UTC
(In reply to comment #11)
> 
> Yep, I think ResourceWarnings are ignorable and according to the python3 doc
> they should be disabled by default (but somewhere the warnings seem to be
> turned on).

I think this is just during the scipy.test in the %check section.


> I plan to build & update it in devel and f16, just wanted to let co-maintainers
> a change to object, like already said in comment #6.

So far it's been pretty quiet. :)


> It's not to late for f16 ;)

Great.  Thanks again for all of your help with this.

Comment 13 Thomas Spura 2011-09-09 17:54:58 UTC
(In reply to comment #12)
> > It's not to late for f16 ;)
> 
> Great.  Thanks again for all of your help with this.

You're welcome, happy testing and converting to python3 :)

Comment 14 Fedora Update System 2011-09-09 17:55:28 UTC
scipy-0.9.0-2.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/scipy-0.9.0-2.fc16

Comment 15 Fedora Update System 2011-09-10 18:59:25 UTC
Package scipy-0.9.0-2.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing scipy-0.9.0-2.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/scipy-0.9.0-2.fc16
then log in and leave karma (feedback).

Comment 16 Fedora Update System 2011-09-30 19:49:56 UTC
scipy-0.9.0-2.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.