Bug 478856 - liblapack.so.3 ABI incompatible with the Lapack provided one
liblapack.so.3 ABI incompatible with the Lapack provided one
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: atlas (Show other bugs)
19
All Linux
medium Severity high
: ---
: ---
Assigned To: Frantisek Kluknavsky
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-01-05 12:43 EST by Jonathan Underwood
Modified: 2014-10-29 09:50 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-10-29 09:50:39 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jonathan Underwood 2009-01-05 12:43:20 EST
Description of problem:
/usr/lib64/liblapack.so.3 (provided by the lapack package) and /usr/lib64/atlas/liblapack.so.3 (provided by the atlas package) are not ABI compatible. One reason for that is that the atlas one prefixes symbols with clapack_. This breaks packages, because rpm (correctly) doesn't doesn't add paths to library dependendencies in packages. Hence a dependency on liblapack.so.3 is satisfied by both atlas and lapack packages, but symbols in the atlas liblapack are not present in the liblapack one. The only solution to this is to rename liblapack as provided by atlas to eg. libatlas-lapack.

This bug is an example:
https://bugzilla.redhat.com/show_bug.cgi?id=478435

See also the discussion on the fedora-devel mailing list:
https://www.redhat.com/archives/fedora-devel-list/2009-January/msg00164.html

Version-Release number of selected component (if applicable):
atlas-3.6.0-15.fc10.x86_64
Comment 1 Jonathan Underwood 2009-01-05 19:41:14 EST
Incidentally, the version of atlas currently shipped is quite out of date as well.
Comment 2 Deji Akingunola 2009-01-06 14:05:29 EST
(In reply to comment #1)
> Incidentally, the version of atlas currently shipped is quite out of date as
> well.

The atlas package has been updated to its latest version in rawhide. I'm hoping to find time to update F-10 (and possibly F-9) to the latest version while also taking care of the many packages that depends on it, in a co-ordinated fashion.
Comment 3 Deji Akingunola 2009-01-06 14:19:41 EST
(In reply to comment #0)
> Description of problem:
> /usr/lib64/liblapack.so.3 (provided by the lapack package) and
> /usr/lib64/atlas/liblapack.so.3 (provided by the atlas package) are not ABI
> compatible. One reason for that is that the atlas one prefixes symbols with
> clapack_. This breaks packages, because rpm (correctly) doesn't doesn't add
> paths to library dependendencies in packages. Hence a dependency on
> liblapack.so.3 is satisfied by both atlas and lapack packages, but symbols in
> the atlas liblapack are not present in the liblapack one. The only solution to
> this is to rename liblapack as provided by atlas to eg. libatlas-lapack.
> 
> This bug is an example:
> https://bugzilla.redhat.com/show_bug.cgi?id=478435
> 
While I haven't fully grasped what's going on at the referenced bugzilla, I don't think there is any problem with the atlas provided liblapack.so.3 . It should be fully compatible with liblapack.so.3 provided by the stock lapack. The atlas' liblapack.so.3 is a superset of the lapack's, which includes some symbols prefixed by clapack_ , cblas_ and ATL_ and all the other non-prefixed symbols that lapack itself includes.
If you're only interested in the ATLAS optimized lapack routines, then you should consider linking with liblapack_atlas.so.3 .
Comment 4 Jonathan Underwood 2009-01-06 16:07:52 EST
(In reply to comment #3)
> While I haven't fully grasped what's going on at the referenced bugzilla, I
> don't think there is any problem with the atlas provided liblapack.so.3 . It
> should be fully compatible with liblapack.so.3 provided by the stock lapack.
> The atlas' liblapack.so.3 is a superset of the lapack's, which includes some
> symbols prefixed by clapack_ , cblas_ and ATL_ and all the other non-prefixed
> symbols that lapack itself includes.
> If you're only interested in the ATLAS optimized lapack routines, then you
> should consider linking with liblapack_atlas.so.3 .

I think you've missed the point. This chain of events is currently possible:
1) A packager foo builds a package foo in the buildsystem against atlas, using one or more of the extra symbols that atlas has compared to normal lapack
2) During the package building RPM will work out library dependencies. It will see that foo requires liblapack.so.3, so it adds a Requires for that library to the package foo.
3) A user installs package foo with (say) yum. Foo requires liblapack.so.3, so yum (or the user) installs lapack, which also provides liblapack.so.3.
4) Foo doesn't work because liblapack.so.3 doesn't have the extra Atlas symbols.

Two libraries which are not ABI compatible should not have the same soname - this is sort of a fundamental assumption built into Fedora.

My recommendation would be to rename liblapack.so.3 from atlas to liblapack_atlas.so.3 and dropping the (rather pointless) present liblapack_atlas.so.3.
Comment 5 Jonathan Underwood 2009-01-06 16:13:55 EST
(In reply to comment #2)
> (In reply to comment #1)
> > Incidentally, the version of atlas currently shipped is quite out of date as
> > well.
> 
> The atlas package has been updated to its latest version in rawhide. I'm hoping
> to find time to update F-10 (and possibly F-9) to the latest version while also
> taking care of the many packages that depends on it, in a co-ordinated fashion.

Oh ok. Cool. I hadn't noticed that rawhide has the latest version, apologies.

[Incidentally, if you decide to, the rebuilding of dependent packages is made much easier if you ask rel-eng to make you a temporary tag in koji so that the package rebuilds can be done at your leisure using that tag without causing breakage to the main repository.]
Comment 6 Deji Akingunola 2009-01-06 16:37:37 EST
(In reply to comment #4)

> 
> I think you've missed the point. This chain of events is currently possible:
...

> Two libraries which are not ABI compatible should not have the same soname -
> this is sort of a fundamental assumption built into Fedora.
> 
> My recommendation would be to rename liblapack.so.3 from atlas to
> liblapack_atlas.so.3 and dropping the (rather pointless) present
> liblapack_atlas.so.3.

I think I now understand you. But unfortunately I can't readily accept your recommendation. Most people that uses atlas expect it to provide libblas and liblapack libraries (which is main of point of the package, to provide an optimized variant of blas and lapack). My own suggestion is to maybe entirely drop the stock lapack and blas from Fedora package collections and make atlas (or one of the other optimized versions) provide it.
Comment 7 Jonathan Underwood 2009-01-06 16:58:17 EST
(In reply to comment #6)
> I think I now understand you. But unfortunately I can't readily accept your
> recommendation. Most people that uses atlas expect it to provide libblas and
> liblapack libraries (which is main of point of the package, to provide an
> optimized variant of blas and lapack). My own suggestion is to maybe entirely
> drop the stock lapack and blas from Fedora package collections and make atlas
> (or one of the other optimized versions) provide it.

That has a certain appeal. But, remember, in order for Atlas to provide complete lapack coverage it needs to build against lapack. I suppose that could happen as part of the Atlas package build, such that the lapack tarball is part of the Atlas SRPM. Ugly though.

Another reason to keep the standard lapack (and blas) packages around is that they represent many years of hard won debugging and checking. Scientific researchers often want to build and run against such trusted libraries to cross check things. I'm not sure we can afford to drop lapack (and blas).

I will give some more thought as to whether there is a better 3rd option. Mind is blank right now.

[Aside: Also, bare in mind that Atlas isn't the only blas that is used with lapack.]

[Aside: Another point to note is that currently the Atlas package ensure its liblapack is linked to when it's installed by dropping a script into /etc/ld.so.conf.d.  However as the other bug I referenced demonstrates - this is a very fragile way of doing things, as it breaks whenever a user sets LD_LIBRARY_PATH himself, for whatever reason. The need for this script would go away if the Atlas library were renamed.]
Comment 8 Jonathan Underwood 2009-01-06 17:29:40 EST
Actually, Perhaps it would be enough to bump the soname of the Atlas liblapack
to always be 1 higher than the lapack liblapack so name against which it was
built - i.e. in the present case liblapack.so.4.

From a user perspective this would still amount to having -llapack on the
compile line as was always the case, and selecting which particular lapack
library to build against is done via -L /usr/lib/atlas if the Atlas lapack is
the desired one. No change in behaviour so far.

And this would fix rpm's automatic library dependency generation on building
package foo.

Yes, this to me seems like the best option on the table so far. Thoughts?
Comment 9 Bug Zapper 2009-11-18 05:39:55 EST
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '10'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 10's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 10 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 10 Jonathan Underwood 2009-11-18 08:51:08 EST
Still an open issue, moving to rawhide.
Comment 11 Caolan McNamara 2010-02-10 06:20:18 EST
atlas's lapack shows less symbols than the stock one, e.g.

[root@Nom tmp]# nm -D /usr/lib64/liblapack.so.3.2 | grep zgesdd_
0000003b7f5d45f0 T zgesdd_
[root@Nom tmp]# nm -D /usr/lib64/atlas/liblapack.so.3 | grep zgesdd_
<nada>

atlas-3.8.3-13.fc13.x86_64
lapack-3.2.1-4.fc13.x86_64

which breaks numpy which needs zgesdd_ when atlas gets installed
Comment 12 Deji Akingunola 2010-02-10 11:25:50 EST
(In reply to comment #11)
> atlas's lapack shows less symbols than the stock one, e.g.
> 
It is due to changes in lapack packaging that has not been followed up in atlas.
Now fixed in atlas-3.8.3-15.fc13.
Comment 13 Bug Zapper 2010-03-15 08:22:53 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 13 development cycle.
Changing version to '13'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 14 Steve Traylen 2011-04-30 16:15:40 EDT
Still an open issue, moving to rawhide.

Reading around this the situation arrises because of atlas wants a C rather than FORTRAN
API to liblpack and this was not previously available. [1]

However recently it looks like lapack does now have a standard C interface. [2]

So adding that C interface to the lapack package  or a new package, might this eventually
git rid of this situation. I realize there are some intermediate steps. The atlas C api and
official lapack C API may not be the same.


[1] http://math-atlas.sourceforge.net/faq.html#clapackStand

[2] http://www.netlib.org/lapack/#_standard_c_language_apis_for_lapack
Comment 15 Clément DAVID 2011-11-08 02:32:27 EST
Well, is this always applicable in F16 ?

objdump -T /usr/lib64/liblapack.so.3 |awk '$4 == ".text" {print $7}' |sort >liblapack.so.3.syms
objdump -T /usr/lib64/atlas/liblapack.so.3 |awk '$4 == ".text" {print $7}' |sort >liblapack-atlas.so.3.syms
diff -u liblapack.so.3.syms liblapack-atlas.so.3.syms |grep "-" |grep -v "@"

--- liblapack.so.3.syms	2011-11-08 08:27:46.418401257 +0100
+++ liblapack-atlas.so.3.syms	2011-11-08 08:27:46.466402019 +0100
-lsame_
-xerbla_


lsame : utility function not performance related http://www.netlib.org/lapack/util/lsame.f

xerbla : user provided error handler http://www.netlib.org/lapack/util/xerbla.f
Comment 16 Fedora End Of Life 2013-04-03 14:12:40 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19
Comment 17 Fedora Admin XMLRPC Client 2014-07-26 13:14:39 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 18 Fedora Admin XMLRPC Client 2014-07-28 03:46:23 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 19 Fedora Admin XMLRPC Client 2014-07-28 03:47:33 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 20 Frantisek Kluknavsky 2014-10-29 09:50:39 EDT
Each library has its own unique soname in atlas 3.10. 
I can not imagine a solution for older Fedoras without breaking things.

Note You need to log in before you can comment on or make changes to this bug.