Bug 716945 - Update fftw to 3.3 series
Summary: Update fftw to 3.3 series
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: fftw
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
Assignee: Susi Lehtola
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 726338 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-06-27 14:23 UTC by Steven G. Johnson
Modified: 2011-10-09 19:38 UTC (History)
3 users (show)

Fixed In Version: fftw-3.3-2.fc16
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-09-18 16:35:10 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Patch to update to 3.3 branch (7.48 KB, patch)
2011-07-07 17:45 UTC, Susi Lehtola
no flags Details | Diff
rpmlint output (42.13 KB, application/octet-stream)
2011-07-28 22:07 UTC, Susi Lehtola
no flags Details

Description Steven G. Johnson 2011-06-27 14:23:32 UTC
Description of problem:

There is a new upstream release, version 3.3 beta1, with the final 3.3 version scheduled for release on July 25; it might be good to start testing packages of 3.3 soon.

Some important changes that will affect packaging:

* Support for AVX instructions.  The i386 and amd64 packages should be configured with --enable-sse2 --enable-avx in order to include kernels with both of these types of SIMD instructions.  It is still safe to ship this binary on older processors, because the new instructions are automatically disabled at runtime if they are not supported on the CPU.

* Support for MPI parallelism (previously only available in buggy alpha form).  

* Support for quadruple precision (__float128) in gcc >= 4.6.  This is enabled with --enable-quad-precision, and produces a libfftw3q library.

* Support for a new Fortran 2003 interface, which works in gfortran and allows type-checked access to FFTW from Fortran.  (This only adds a single file to /usr/include, so the package modification is trivial.)

The new release is binary-compatible with the previous FFTW 3.3.2 release, however, so it is a drop-in replacement and Debian need not create a new package.

Comment 1 Susi Lehtola 2011-06-28 13:19:42 UTC
This is going to need some work since the package needs to be adapted for MPI.

At the same moment, we might also think about what flavors of fftw we want to ship. As someone who works on scientific computing, I believe that we should favor OpenMP instead of threads, since the former is a lot more commonly used, since it is way easier to program. Since we are currently shipping a threaded library, we should ship one in the future as well.

However, one does not always want to thread, since it's very often more efficient to just run many calculations in parallel. Thus, I believe the default package should be the serial version.

Of course, this all amounts to a huge explosion in the number of packages:
 - single, double, long and quadruple precision
 - serial version, threaded parallellism, OpenMP parallellism
 - non-MPI, OpenMPI, MPICH2
i.e. 4 x 3 x 3 = 36 compilations.

Not shipping MPI for now would make things simpler.

Comment 2 Susi Lehtola 2011-06-28 13:22:51 UTC
(In reply to comment #0)
> * Support for a new Fortran 2003 interface, which works in gfortran and allows
> type-checked access to FFTW from Fortran.  (This only adds a single file to
> /usr/include, so the package modification is trivial.)

/usr/include is the wrong place, the correct one is %{_libdir}/gfortran/modules.

> The new release is binary-compatible with the previous FFTW 3.3.2 release,
> however, so it is a drop-in replacement and Debian need not create a new
> package.

Excerpt from release notes:

"Temporarily removed MIPS paired-single support due to lack of available hardware for testing. We hope to add it back before the final FFTW 3.3 release; meanwhile, users who want this functionality should continue using FFTW 3.2.x."

If I understand correctly, this means for now that we can't ship 3.3 as a drop-in replacement..

Comment 3 Steven G. Johnson 2011-06-29 01:18:39 UTC
Regarding the number of compilations, some of those are combined since the serial, threads, and MPI versions can be done in a single build.  Also, the MPI versions don't support quadruple precision since MPI doesn't have this type.  But I agree that it is a lot of packages.  Regarding MPI, the main question is whether clusters (or code ported to/from clusters) are important to you.

> /usr/include is the wrong place, the correct one is
> %{_libdir}/gfortran/modules.

It's not a module (a .mod object file).  It's a Fortran source file that you include with an 'include' statement or an #include statement (for .F files that gfortran preprocesses with cpp).  This has the advantage of being compiler-independent, unlike .mod files.  Hence the natural place to put it is /usr/include

For Fortran include statements, as far as I can tell, gfortran has no default search path other than the current directory; see: http://gcc.gnu.org/ml/fortran/2006-06/msg00333.html  ...so there really isn't any preferred place, unless matters have changed.  For #include statements in cpp-preprocessed Fortran, /usr/include is the right place since you want it in the cpp search path.

That being said, if you want to build and install a .mod module file, it's a trivial matter of compiling a 4-line source file: see http://fftw.org/fftw3.3beta_doc/Defining-an-FFTW-module.html

>"Temporarily removed MIPS paired-single support due to lack of available
>hardware for testing. We hope to add it back before the final FFTW 3.3 release;
>meanwhile, users who want this functionality should continue using FFTW 3.2.x."
>
>If I understand correctly, this means for now that we can't ship 3.3 as a
>drop-in replacement..

It is binary compatible; at worst this means performance on MIPS will suffer in single precision.

However, we hope to add it back in by the final release; the main problem is that we don't have a MIPS machine to test on.  If MIPS is important to Fedora and you can help with that it would be much appreciated.

--SGJ

Comment 4 Susi Lehtola 2011-07-07 17:45:50 UTC
Created attachment 511765 [details]
Patch to update to 3.3 branch

This won't actually work with 3.3 beta 1, since openmp support has been fixed after that.

Comment 5 Susi Lehtola 2011-07-08 20:35:29 UTC
I had thought that linking with -lfftw would give the threaded version. This is not the case, since one has to (and has had to) link to -lfftw_threads to get the parallellized version.

There's actually only a few packages that use the threaded part, so I've branched it to a separate package yielding 3 MB savings per installation. And as you can see from the patch, there are no functional changes (except from the branching of the parallellized libraries), so I think it'll be safe to push the update in every branch.

I'll still think about the MPI part, which is more messy to handle.

Comment 6 Susi Lehtola 2011-07-26 19:22:26 UTC
Whenever fftw 3.3 is released, I'll update the branches.

Comment 7 Steven G. Johnson 2011-07-27 18:03:13 UTC
Version 3.3 is now released.

Comment 8 Susi Lehtola 2011-07-28 13:28:08 UTC
*** Bug 726338 has been marked as a duplicate of this bug. ***

Comment 9 Susi Lehtola 2011-07-28 22:07:16 UTC
Created attachment 515803 [details]
rpmlint output

Comment 10 Susi Lehtola 2011-07-28 22:18:02 UTC
(In reply to comment #9)
> Created attachment 515803 [details]
> rpmlint output

As you can see, there are quite a few errors in the source package, which have been reported upstream. I've already fixed the space vs tab error, and the rest are false warnings.

Before I ship the updated package on any distros other than rawhide, I'd like to confirm that separating the fftw{,f,l,q}-wisdom and fftw-wisdom-to-conf binaries and the runtime libraries doesn't lead into trouble.

I'm guessing that no application that uses fftw library calls uses the binaries, since IIRC one can use library calls to do the same thing.

Comment 11 Steven G. Johnson 2011-08-05 17:08:49 UTC
Separating these two programs into a separate package is probably okay.

fftw-wisdom-to-conf does something that cannot be accomplished by library calls.  However, it is only used when building a package (it produces C source code as output) and in practice I doubt it is used by any Fedora package.  Its only purpose is to reduce the size of binaries when static linking, mainly for embedded systems.  Thus it is not relevant to any Fedora package that dynamically links with FFTW.

Comment 12 Susi Lehtola 2011-08-05 18:41:25 UTC
Yeah, that's what I gathered from the documentation. fftw-devel will pull in fftw (and the -libs packages) in any case.

If you aren't yet aware, there was a quadruple precision rounding bug on i686 which has now been fixed, and I'm just waiting for the updated gcc packages to hit the build roots. This affects Fedora 15, 16 and rawhide.

The other packages, which don't have quadruple precision support since it isn't supported by the compiler version, built just fine.

Comment 13 Susi Lehtola 2011-09-18 16:35:10 UTC
fftw 3.3 is in F16 and rawhide, closing.

Comment 14 Fedora Update System 2011-09-29 10:40:08 UTC
fftw-3.3-2.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/fftw-3.3-2.fc16

Comment 15 Fedora Update System 2011-09-29 10:40:17 UTC
fftw-3.3-2.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/fftw-3.3-2.fc15

Comment 16 Fedora Update System 2011-10-09 06:49:44 UTC
fftw-3.3-2.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 17 Fedora Update System 2011-10-09 19:38:59 UTC
fftw-3.3-2.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.