Bug 1304402 - Fedora - Update Atlas from 3.10.2 to new 3.10.3 version
Fedora - Update Atlas from 3.10.2 to new 3.10.3 version
Status: NEW
Product: Fedora
Classification: Fedora
Component: atlas (Show other bugs)
27
All Linux
unspecified Severity medium
: ---
: ---
Assigned To: Jakub Martisko
Fedora Extras Quality Assurance
: Patch
Depends On:
Blocks: PPCTracker F-ExcludeArch-ppc64le/PPC64LETracker
  Show dependency treegraph
 
Reported: 2016-02-03 09:01 EST by IBM Bug Proxy
Modified: 2017-08-18 01:48 EDT (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Source package (11.73 MB, application/x-rpm)
2016-02-04 05:33 EST, IBM Bug Proxy
no flags Details
specfile (21.86 KB, text/x-rpm-spec)
2016-02-04 05:33 EST, IBM Bug Proxy
no flags Details
atlas-3.10.3-1.fc26.src.rpm (4.90 MB, application/octet-stream)
2016-08-02 04:58 EDT, Michel Normand
no flags Details
atlas.spec (34.56 KB, text/plain)
2016-08-02 04:59 EDT, Michel Normand
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
IBM Linux Technology Center 136497 None None None 2016-02-03 09:01 EST

  None (edit)
Description IBM Bug Proxy 2016-02-03 09:01:00 EST

    
Comment 1 IBM Bug Proxy 2016-02-03 09:01:06 EST
Even though the latest "Stable" upstream release of Atlas is 3.10.2, the upstream Changelog contains the following note:

   NOTE: 3.10.2 is terribly out of date, and was released only because the
      threading rewrite it taking too long.  If possible, you should use a
      developer release after testing that it works for your particular
      platform.  In particular, developer releases are *much* faster for any
      x86 that uses AVX or later SIMD ISA, or any machine with ncores >= 8.
      The developer release also supports ARM architectures better (though
      performance is not hugely better if you can get stable installed).


As a result, we have been carrying in Fedora several patches for architecture enablement and still lacking important optimizations like AVX on x86 and VSX on ppc64le.

We should follow upstream advise and use the latest developer release, which, as of now, is 3.11.38.
Comment 2 IBM Bug Proxy 2016-02-04 05:20:54 EST
------- Comment From thierry.fauck@fr.ibm.com 2016-02-04 05:19 EDT-------
The 3.11.38 fixed some bugs in the system probe and the specfile must be clean a bit.

Main changes are in the %build section where the %ifarch ppc64le section needs to be removed.
%build
for type in %{types}; do
if [ "$type" = "base" ]; then
libname=atlas
#%define pr_base %(echo ))
%define pr_base %{nil}
else
libname=atlas-${type}
fi

mkdir -p %{_arch}_${type}
pushd %{_arch}_${type}
../configure  %{mode} %{?threads_option} %{?arch_option} %{shared_option} -D c -DWALL -Fa alg '%{armflags} -g -fstack-protector-strong -Wa,--noexecstack -fPIC'\
--prefix=%{buildroot}%{_prefix}                 \
--incdir=%{buildroot}%{_includedir}             \
--libdir=%{buildroot}%{_libdir}/${libname}      \
--with-netlib-lapack-tarfile=%{SOURCE10}
Comment 3 IBM Bug Proxy 2016-02-04 05:33:02 EST
Created attachment 1121064 [details]
Source package
Comment 4 IBM Bug Proxy 2016-02-04 05:33:11 EST
Created attachment 1121065 [details]
specfile
Comment 5 Jan Kurik 2016-02-24 10:39:18 EST
This bug appears to have been reported against 'rawhide' during the Fedora 24 development cycle.
Changing version to '24'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora24#Rawhide_Rebase
Comment 6 Jakub Jelen 2016-03-23 18:01:28 EDT
Almost a year ago, there was announcement about ATLAS 3.11.35 [1]. I spend days building the atlas "from Fedora master" and wondering why the heck is it so slow and why it does not use the fancy vector instructions that Power8 provides.

Can we haz an update?

Or is there something blocking the update? I am especially interested in the Power features. I will certainly build my own tuned, but having Fedora as a leading platform with up-to date software would be cool.

[1] https://sourceforge.net/p/math-atlas/mailman/message/34330714/
Comment 7 Dan Horák 2016-03-24 04:22:43 EDT
My main concern about updating Atlas is the API/ABI compatibility. The 3.11 is a development version AFAIK and the questions are - does it have same API/ABI as our 3.10 packages? Will the final 3.12 be compatible with 3.11.x? We should avoid unnecessary breakages and rebuilds of dependent packages.
Comment 8 IBM Bug Proxy 2016-03-24 05:02:38 EDT
------- Comment From thierry.fauck@fr.ibm.com 2016-03-24 04:57 EDT-------
The current version 3.11.38 which includes optimizations for ppc64le support still have problems which are supposed to be fixed in the upcoming 3.11.39 version in dev. currently. There is no availability date yet.
V 3.11.38 uses vectors and VSX when basic support integrated to 3.10.2 was just using a generic system type with no optimization for ppc64le.
Comment 9 Jakub Jelen 2016-04-04 16:50:35 EDT
Thank you for the information. I understand the reasons. I tried myself to build ATLAS from upstream tarball, but without success. On the other hand your SRPM works just fine for my requirements. Thanks!
Comment 10 Jakub Jelen 2016-05-08 18:04:24 EDT
I tried to rebuild this package for more than the default 4 threads (with modified spec file) and the build was failing on Power8. There are still some issues with this development version, but I really appreciate the effort preparing this package.
I am not sure if you were aware of this problem. If you are interested, I guess I can provide some logs (I myself was not sure what are the important parts there).
Comment 11 IBM Bug Proxy 2016-05-10 11:54:55 EDT
------- Comment From thierry.fauck@fr.ibm.com 2016-05-10 11:47 EDT-------
Thanks for the report, I will double check current dev. version and if needed I will come back to you (but definitely knowing that changing spec is good to know).
Comment 12 Jan Kurik 2016-07-26 00:17:47 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 25 development cycle.
Changing version to '25'.
Comment 13 IBM Bug Proxy 2016-07-29 03:21:28 EDT
------- Comment From thierry.fauck@fr.ibm.com 2016-07-29 03:10 EDT-------
Folks,

I am happy to announce the release of ATLAS 3.10.3.  Please forward this
e-mail to any relevant mailing list, or to any parties that are
repackaging the stable release that you know about.

ATLAS 3.10.3 should be noticeably faster than 3.10.2 on modern hardware,
but the 3.11 series is almost always much faster on such systems.  While
I was able to backport support for modern architectures, and even
provide some reasonable kernels for modern ISA extensions, the 3.11
series allows for much larger block factors and improved storage formats
that are required to get decent performance many modern machines
(including all AVX-enabled Intel chips).  So, if you can use it, 3.11 is
still the best for modern machines by a long way.

I had hoped to have ATLAS 4.0 out by now, but various setbacks have
delayed the release, necessitating 3.10.3, since 3.10.2 was not
installing well on modern machines.

3.10.3 fixes these three bugs:
http://math-atlas.sourceforge.net/errata3.10.2.html#herkNaN
http://math-atlas.sourceforge.net/errata3.10.2.html#syr2kNaN
http://math-atlas.sourceforge.net/errata3.10.2.html#rotmg

I have tested 3.10.3 to work on the following OSes:
1. Linux
2. Windows64 (cygwin64 builds now work!)
3. AIX
4. OS X

For OSes 2-4, see special sections in the install guide for additional help:
http://math-atlas.sourceforge.net/atlas_install/node53.html
Hopefully other OSes (eg., Windows32, Solaris) still work from 3.10.2
testing.

Also note that clang can now be used to build ATLAS by adding:
--force-clang=/path/to/clang
to your configure line.  For the open version of clang, performance
still tends to lag gcc, but is strongly improved from last release.
Apple's clang appears to be substantially faster, but I may be mistaken.

New architecture support available in 3.10.3 includes:
1. ARM32: a7, a9, a15 (auto-detect of SOFT/HARD ABI)
2. ARM64: xgene1, a53, a57
3. Intel: Corei3 & Corei4 (skylake)
4. IBM: Z series, POWER8 (including little/big endian)

Support for modern vector extensions in atlas_simd.h:
1. Intel AVX2
2. IBM VSX & Z-series VX
3. ARM64 Advanced SIMD
4. ARM32 NEON (only if -Si ieee 0 flag is thrown)

Regards,
Clint

ATLAS 3.10.3 released 07/28/16, highlights of changes from 3.10.2
* Updated F77 L1BLAS testers to those used LAPACK3.6.1
* Fixed bug in rotmg revealed by LAPACK3.6.1 testers
* Fixed bug in hprk/sprk that could cause NaN propogation in
HERK/SYRK due
to reading uninitialized memory in BETA=0 case
* Fixed bug in threaded SYR2K/HER2K that could cause NaN propogation due
to reading uninitialized memory
* Extended matrix/vector norm functions to detect NaNs
* Extended configure:
+ --force-clang=/path/to/clang : will use clang for all C compilers,
even goodgcc (assumes gcc flag & inline-assembly compatibility)
+ --cripple-atlas-performance: install despite failing throttle check
+ Can now use arch string rather than enum # for -A arg
+ --force-tids now affects ATLrun.sh as well as threaded build
+ ARM32 autodetects SOFTFP/HARDFP ABI
* backport of config & archdefs for:
+ POWER[7,8]le, IBMz[10,13,19], Corei[3,4], ARM[7,9,15,17],
ARM64[xgene,a53,a57]
+ archdefs for NEON ARMa[7,15]
+ config support for IBM Z[9,196,12]
* backport & extension of atlas_simd.h & atlas_cplxsimd.h
+ New SIMD kernels for: VSX, VXZ, AVX2, AdvancedSIMD, NEON
* Fixed mflop test of PrintMMLine, that sometimes failed to print
valid mflop due to negative values from prior runs
* Removed ATL_dmm6x1x60_sse2_32.c from z index files (not valid cplx
kern)
* Forced MinGW comps to be ignored unless -Si nocygwin 1 is set
* Added support for WOW64 detection & basic use, numerous changes to
make
work on cygwin64
* Fixed uninit nM in s[1,2]nxtune.c's RecDoubleNX

--
**********************************************************************
** R. Clint Whaley, PhD * Assoc Prof, LSU * www.csc.lsu.edu/~whaley **
**********************************************************************
Comment 14 Michel Normand 2016-08-02 04:58 EDT
Created attachment 1186711 [details]
atlas-3.10.3-1.fc26.src.rpm

I am providing in attachment the new source rpm for atlas 3.10.3 stable release.
I validated it with scratch build on two koji:

http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3575870 for PowerPC  
http://koji.fedoraproject.org/koji/taskinfo?taskID=15100015 for x86_64

Question to package maintainers:
Could the atlas package be updated with this new source rpm ?
Comment 15 Michel Normand 2016-08-02 04:59 EDT
Created attachment 1186712 [details]
atlas.spec

spec file associated to the previous atlas-3.10.3-1.fc26.src.rpm
Comment 16 Orion Poplawski 2016-08-18 19:24:08 EDT
I'm seeing the ppc64 test failure: 
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3616883
http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/6883/3616883/build.log

Makefile:1692: recipe for target 'ssanity_test' failed

Anyone know what is causing it?
Comment 17 IBM Bug Proxy 2016-08-19 03:42:20 EDT
------- Comment From hannsj_uhl@de.ibm.com 2016-08-19 03:35 EDT-------
Comment from  Michel Normand 2016-08-19 03:06:36 EDT

(In reply to comment #20)
> I'm seeing the ppc64 test failure:
> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3616883
> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/6883/3616883/build.log
>
> Makefile:1692: recipe for target 'ssanity_test' failed
>
> Anyone know what is causing it?

No I do not know the cause.
That was previously addressed upstream in atlas-devel ML,
I am just creating an upstream issue to keep track  of the test failure.
https://sourceforge.net/p/math-atlas/support-requests/1032/
Comment 18 Fedora Admin XMLRPC Client 2017-05-29 07:10:20 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 19 IBM Bug Proxy 2017-05-29 07:16:48 EDT
#%define pr_base %(echo $((%{__isa_bits}+0)))
Comment 20 Jakub Martisko 2017-08-10 08:07:21 EDT
Ok, I've tried to rewrite the spec file a bit. All arches should now build and pass tests without any errors. I've tried to make the base package to be as least demanding (i.e. I am using more the "low end" cpus for the given arch) as possible.  This might lead to some performance regression but the package should be hopefully bit more stable and easier to build. To compensate, I've created few more optimized subpackages (power7, power8, ...). I've also reduced the number of threads for the threaded version to 2 for most of the arches, otherwise, there might be some failures on the machines with less cores than 4 (value used in the old builds). Any comments/hints/warnings are welcomed. 

https://koji.fedoraproject.org/koji/taskinfo?taskID=21085355

Note You need to log in before you can comment on or make changes to this bug.