Bug 513381 - octave uses reference blas instead of ATLAS
Summary: octave uses reference blas instead of ATLAS
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: octave
Version: 11
Hardware: x86_64
OS: Linux
low
medium
Target Milestone: ---
Assignee: Rakesh Pandit
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 491217
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-07-23 12:17 UTC by Dmitri A. Sergatskov
Modified: 2010-02-05 01:18 UTC (History)
4 users (show)

Fixed In Version: 3.2.3-4.fc12
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-02-05 01:18:44 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Patch against F-12 branch (1.42 KB, patch)
2010-01-06 10:19 UTC, Susi Lehtola
no flags Details | Diff

Description Dmitri A. Sergatskov 2009-07-23 12:17:53 UTC
Description of problem:

octave uses reference blas instead of ATLAS

Version-Release number of selected component (if applicable):

3.0.5-1

How reproducible:

100%

Steps to Reproduce:

At octave prompt run
a=randn(3000);
tic; a'*a; toc
  
Actual results:

It takes about 100 sec on 2.4GHz Opteron

Expected results:

About 14 seconds

Additional info:
 ldd /usr/bin/octave
	linux-vdso.so.1 =>  (0x00007fff072f5000)
	liboctinterp.so => /usr/lib64/octave-3.0.5/liboctinterp.so (0x0000003b6b000000)
	liboctave.so => /usr/lib64/octave-3.0.5/liboctave.so (0x0000003b68400000)
	libcruft.so => /usr/lib64/octave-3.0.5/libcruft.so (0x0000003b6ac00000)
	libumfpack.so.5 => /usr/lib64/libumfpack.so.5 (0x0000003b68000000)
	libamd.so.2 => /usr/lib64/libamd.so.2 (0x0000003b69e00000)
	libcamd.so.2 => /usr/lib64/libcamd.so.2 (0x0000003b67800000)
	libcolamd.so.2 => /usr/lib64/libcolamd.so.2 (0x0000003b69a00000)
	libcholmod.so.1 => /usr/lib64/libcholmod.so.1 (0x0000003b67000000)
	libccolamd.so.2 => /usr/lib64/libccolamd.so.2 (0x0000003b67c00000)
	libcxsparse.so.2 => /usr/lib64/libcxsparse.so.2 (0x0000003b6a200000)
	liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x0000003b69200000)
	libblas.so.3 => /usr/lib64/libblas.so.3 (0x00000030d0200000)
	libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00000037b2e00000)
	libreadline.so.5 => /lib64/libreadline.so.5 (0x00000037b3c00000)
	libncurses.so.5 => /lib64/libncurses.so.5 (0x00000037bae00000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00000037b1800000)
	libhdf5.so.6 => /usr/lib64/libhdf5.so.6 (0x00000032dbc00000)
	libz.so.1 => /lib64/libz.so.1 (0x00000037b2000000)
	libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00000037b2a00000)
	libm.so.6 => /lib64/libm.so.6 (0x00000037b1400000)
	libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000037b7000000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000037b5000000)
	libc.so.6 => /lib64/libc.so.6 (0x00000037b1000000)
	libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x0000003b67400000)
	libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x0000003b68e00000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00000037b1c00000)
	libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00000037c0600000)
	/lib64/ld-linux-x86-64.so.2 (0x00000037b0c00000)
	libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x0000003b6a600000)
##

After I rebuild rpm with modified configure command to:

%configure --enable-shared --disable-static --enable-64=%enable64 --with-f77=gfortran --with-blas="-L/usr/lib64/atlas -lf77blas -latlas" --with-lapack="-llapack"

I got:
ldd /usr/bin/octave
	linux-vdso.so.1 =>  (0x00007fff11767000)
	liboctinterp.so => /usr/lib64/octave-3.0.5/liboctinterp.so (0x00007fa40c096000)
	liboctave.so => /usr/lib64/octave-3.0.5/liboctave.so (0x00007fa40b6ef000)
	libcruft.so => /usr/lib64/octave-3.0.5/libcruft.so (0x00007fa40b48b000)
	libumfpack.so.5 => /usr/lib64/libumfpack.so.5 (0x0000003b68000000)
	libamd.so.2 => /usr/lib64/libamd.so.2 (0x0000003b69e00000)
	libcamd.so.2 => /usr/lib64/libcamd.so.2 (0x0000003b67800000)
	libcolamd.so.2 => /usr/lib64/libcolamd.so.2 (0x0000003b69a00000)
	libcholmod.so.1 => /usr/lib64/libcholmod.so.1 (0x0000003b67000000)
	libccolamd.so.2 => /usr/lib64/libccolamd.so.2 (0x0000003b67c00000)
	libcxsparse.so.2 => /usr/lib64/libcxsparse.so.2 (0x0000003b6a200000)
	liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x0000003b69200000)
	libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x0000003b68e00000)
	libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x0000003b6a600000)
	libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00000037b2e00000)
	libreadline.so.5 => /lib64/libreadline.so.5 (0x00000037b3c00000)
	libncurses.so.5 => /lib64/libncurses.so.5 (0x00000037bae00000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00000037b1800000)
	libhdf5.so.6 => /usr/lib64/libhdf5.so.6 (0x00000032dbc00000)
	libz.so.1 => /lib64/libz.so.1 (0x00000037b2000000)
	libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00000037b2a00000)
	libm.so.6 => /lib64/libm.so.6 (0x00000037b1400000)
	libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000037b7000000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000037b5000000)
	libc.so.6 => /lib64/libc.so.6 (0x00000037b1000000)
	libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x0000003b67400000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00000037b1c00000)
	libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00000037c0600000)
	/lib64/ld-linux-x86-64.so.2 (0x00000037b0c00000)

And benchmark times returns to expected ~14 seconds.

Comment 1 Alex Lancaster 2009-07-24 23:51:20 UTC
This is related to bug #491217.  There is an ambiguity in the deps that atlas should use lapack but .so names are provided by multiple packages.  Not sure if hardcoding the llapack in the %configure line would fix this or not.

Rakish may know.

Comment 2 Alex Lancaster 2009-07-24 23:53:58 UTC
(In reply to comment #1)

> Rakish may know.  

Sorry, Rakesh.

Comment 3 Dmitri A. Sergatskov 2009-07-25 01:41:18 UTC
I think the critical change is the --with-blas="-L/usr/lib64/atlas -lf77blas -latlas", rather than --with-lapack="-llapack"

Comment 4 Susi Lehtola 2009-08-17 21:59:24 UTC
By the way, as discussed a few days ago on fedora-devel, ATLAS isn't available for all processors at least on i386 (e.g. non-SSE ones), so changing from the reference implementation would currently mean that Octave doesn't work on older machines.

Comment 5 Alex Lancaster 2009-08-25 06:47:29 UTC
By the way, is it possible that compiling against atlas as per comment #3, might help with bug #510841 comment #16 ?

Comment 6 Fedora Update System 2009-10-20 00:07:41 UTC
plplot-5.9.5-1.fc11,pfstools-1.7.0-8.fc11,GMT-4.5.0-4.fc11,octave-forge-20090607-15.fc11,octave-3.2.3-1.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/plplot-5.9.5-1.fc11,pfstools-1.7.0-8.fc11,GMT-4.5.0-4.fc11,octave-forge-20090607-15.fc11,octave-3.2.3-1.fc11

Comment 7 Fedora Update System 2009-11-04 12:25:16 UTC
mathgl-1.9-6.fc11.1, plplot-5.9.5-1.fc11, pfstools-1.7.0-8.fc11, GMT-4.5.0-4.fc11, octave-forge-20090607-15.fc11, octave-3.2.3-1.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 8 Alex Lancaster 2009-11-04 18:23:23 UTC
bodhi isn't too smart about the version

Comment 9 Susi Lehtola 2010-01-02 12:07:11 UTC
Hmm,

$ rpm -q atlas
atlas-3.8.3-12.fc12.x86_64
$ rpm -q --requires octave|grep atlas
$

so it still isn't using ATLAS...

Comment 10 Alex Lancaster 2010-01-05 22:40:09 UTC
No, it does require ATLAS, it's just that requires are automatically generated, provided by shared libraries not explicit package, see:

$ rpm -q --provides atlas
config(atlas) = 3.8.3-12.fc12
libatlas.so.3()(64bit)  
libcblas.so.3()(64bit)  
libclapack.so.3()(64bit)  
libf77blas.so.3()(64bit)  
liblapack.so.3()(64bit)  
libptcblas.so.3()(64bit)  
libptf77blas.so.3()(64bit)  
atlas = 3.8.3-12.fc12
atlas(x86-64) = 3.8.3-12.fc12

$ rpm -q --requires octave|grep liblapack.so.3 
liblapack.so.3()(64bit)  


Also you can see that they are requires if you try:

yum remove atlas

that will also try to remove octave

Comment 11 Susi Lehtola 2010-01-06 00:08:09 UTC
(In reply to comment #10)
> No, it does require ATLAS, it's just that requires are automatically generated,
> provided by shared libraries not explicit package, see:
> 
> $ rpm -q --requires octave|grep liblapack.so.3 
> liblapack.so.3()(64bit)  

Umm, doesn't this just mean that it needs the LAPACK library: 
$ rpm -q --whatprovides "liblapack.so.3()(64bit)"
lapack-3.2.1-3.fc12.x86_64
atlas-3.8.3-12.fc12.x86_64

ATLAS just has the same ABI.


Actually, the funny thing is that even though the octave package doesn't require 

$ rpm -q octave
octave-3.2.3-2.fc12.1.x86_64

$ rpm -q --requires octave | grep atlas
$


For a further look, one can run

$ rpm -q --requires octave|grep .so|xargs rpm -q --whatprovides|grep atlas
atlas-3.8.3-12.fc12.x86_64

but this is still the same liblapack that is also provided by lapack.


However, the binary actually seems to be linked with atlas:
$ ldd /usr/bin/octave|grep atlas
	liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x0000003bb9e00000)
	libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x0000003bb9600000)
	libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x0000003bb8c00000)
	libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007f1882666000)

***

If you try "yum remove atlas" it will want to remove octave, since octave links against suitesparse, which in turn requires libcblas.so.3 that is provided (only by) ATLAS.

Comment 12 Alex Lancaster 2010-01-06 00:21:45 UTC
I would never expect that

$ rpm -q --requires octave | grep atlas

would return anything because all deps are supposed to be provided automatically not via explicit package names.

Right so the problem is that the same soname is provided by two different packages, which was the original problem.  What happens in this case (or at least it used to work this way) was that yum would install the package with the shortest package name, so "atlas" would "win" over "lapack".

I don't know how to ultimately resolve this except by somehow co-ordinating things with upstream atlas and lapack so their sonames don't conflict.  Ideas?

Comment 13 Alex Lancaster 2010-01-06 00:22:38 UTC
Would comment #3 (which isn't implemented in the spec file AFAIK) help?

Comment 14 Susi Lehtola 2010-01-06 09:58:48 UTC
(In reply to comment #13)
> Would comment #3 (which isn't implemented in the spec file AFAIK) help?  

No: tried it, didn't work - the requires turn out the same...

Comment 15 Susi Lehtola 2010-01-06 10:19:19 UTC
Created attachment 381944 [details]
Patch against F-12 branch

Comment 16 Susi Lehtola 2010-01-06 10:21:02 UTC
I figured it out - apply the patch in comment #15, then octave really builds against atlas:

$ rpm -qp --requires octave-3.2.3-2.fc12.1.x86_64.rpm |grep atl
libatlas.so.3()(64bit)

Comment 17 Susi Lehtola 2010-01-06 10:31:39 UTC
Reopening.

With the current F-12 package the operation in comment #1 takes 22 seconds on my Core 2 Quad, after application of the patch in comment #15 the time drops to 3 seconds.

Comment 18 Fedora Update System 2010-01-06 15:33:17 UTC
octave-3.2.3-3.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/octave-3.2.3-3.fc12

Comment 19 Fedora Update System 2010-01-07 00:58:14 UTC
octave-3.2.3-3.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update octave'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F12/FEDORA-2010-0239

Comment 20 Fedora Update System 2010-01-17 13:14:09 UTC
octave-3.2.3-4.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/octave-3.2.3-4.fc12

Comment 21 Fedora Update System 2010-01-19 00:59:27 UTC
octave-3.2.3-4.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update octave'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F12/FEDORA-2010-0722

Comment 22 Fedora Update System 2010-02-05 01:18:33 UTC
octave-3.2.3-4.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.