Bug 562504

Read the MPI packaging guidelines, and try again. Make subpackages (compile against) at least for Open MPI and MPICH2.

http://fedoraproject.org/wiki/PackagingDrafts/MPI

(The guidelines were approved by FESCo in August, but spot still hasn't had time to integrate them with the rest of the guidelines.)

The example in the guidelines is nonsense for this python build :'(

I completely rewrote dobuildX and added doinstallX for both python2 and python3.
Don't know if this requires different guidelines :(


From the logs, it builds with
/usr/lib64/mpich2/bin/mpicc
/usr/lib64/openmpi/bin/mpicc
/usr/bin/mpicc

but there is a warning, that:
*** WARNING: identical binaries are copied, not linked:
        /usr/lib64/python2.6/site-packages/mpi4py-mpich2/MPE.so
   and  /usr/lib64/python2.6/site-packages/mpi4py/MPE.so
extracting debug info from /home/tom/rpmbuild/BUILDROOT/mpi4py-1.2-2.fc12.x86_64/usr/lib64/python2.6/site-packages/mpi4py-mpich2/MPI.so
*** WARNING: identical binaries are copied, not linked:
        /usr/lib64/python2.6/site-packages/mpi4py-mpich2/MPI.so
   and  /usr/lib64/python2.6/site-packages/mpi4py/MPI.so
extracting debug info from /home/tom/rpmbuild/BUILDROOT/mpi4py-1.2-2.fc12.x86_64/usr/lib64/python2.6/site-packages/mpi4py-mpich2/dl.so
*** WARNING: identical binaries are copied, not linked:
        /usr/lib64/python2.6/site-packages/mpi4py-mpich2/dl.so
Same for python3.1.

So I guess, grepping the right compiler did not work.

Do you have a solution for that?

(Note: Because did not work atm, I didn't add proper desctiptions.)

Builds in koji, with all python3-* subpackages:


###########################
current TODO list:
- properly build with mpi variants (don't know how)
- add good descriptions
- what to do with lam?
  can this left out?
- didn't look for rpmlint. There are enought other problems
- How to enable the testsuite "mpd&..."?


Spec URL: http://tomspur.fedorapeople.org/review/mpi4py.spec
SRPM URL: http://tomspur.fedorapeople.org/review/mpi4py-1.2-2.fc12.src.rpm

Forgot the koji build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=1967451

LAM has been obsoleted long ago and dropped from rawhide, so drop it.

**

The correct locations for the modules are
 %{python_sitearch}/openmpi/
for the Open MPI library and and
 %{python_sitearch}/mpich2/
for the MPICH2 library.

Is there *really* a *serial* version of the library?

**

The -mpi package must be named -openmpi.

**

Drop the optional compilers support, it's only meant for MPI compiler packages (e.g. openmpi and mpich2).

**

Is the egg independent of the MPI library used?

**

I don't like the use of
 %{_builddir}/$MPI_COMPILER
please create the temporary directories inside the same buildroot, e.g.
 mv build $MPI_COMPILER

Next try ;)

(In reply to comment #4)
> LAM has been obsoleted long ago and dropped from rawhide, so drop it.

Done

> The correct locations for the modules are
>  %{python_sitearch}/openmpi/
> for the Open MPI library and and
>  %{python_sitearch}/mpich2/
> for the MPICH2 library.
> 
> Is there *really* a *serial* version of the library?

No, I deleted that part again, just building the parallel versions.

> The -mpi package must be named -openmpi.

Done

> Drop the optional compilers support, it's only meant for MPI compiler packages
> (e.g. openmpi and mpich2).

Done

> Is the egg independent of the MPI library used?

Yes, but now it's shipped in every supbackage

> I don't like the use of
>  %{_builddir}/$MPI_COMPILER
> please create the temporary directories inside the same buildroot, e.g.
>  mv build $MPI_COMPILER    

Now I move build around, should be better now.

- koji build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=1967719

- rpmlint complains about devel-file-in-non-devel-package and no-documentation.
  Rest is ok.

Remaining 'maybe' TODO: enable testsuite

Spec URL: http://tomspur.fedorapeople.org/review/mpi4py.spec
SRPM URL: http://tomspur.fedorapeople.org/review/mpi4py-1.2-3.fc12.src.rpm

The subpackages need to Requires: mpi4py-common, since the license etc is in -common.

Move the large documentation stuff into -doc.

**

The names of the python3 versions are IMHO a bit odd, but this seems to be the way python3 is packaged in Fedora.

**

After you've added the docs and enabled running the testsuite I'll do the review.

(In reply to comment #6)
> The subpackages need to Requires: mpi4py-common, since the license etc is in
> -common.
> 
> Move the large documentation stuff into -doc.

Done

> The names of the python3 versions are IMHO a bit odd, but this seems to be the
> way python3 is packaged in Fedora.

Also don't know a better way. packages like mpi4py-openmpi-python3 are not better.


- Testsuite is enabled, and fails within 2 tests for openmpi. For mpich2 both tests pass. (Both for python2 and python3)
  The failing is reported upstream at:
  http://code.google.com/p/mpi4py/issues/detail?id=14

New koji build is failing, because mpd can not be started:
http://koji.fedoraproject.org/koji/taskinfo?taskID=1967879

Don't know how to to that part better. Here locally it works.
Maybe it's not possible to do that in the buildsystem.
Do you have an idea? Did you have a similar case?


Spec URL: http://tomspur.fedorapeople.org/review/mpi4py.spec
SRPM URL: http://tomspur.fedorapeople.org/review/mpi4py-1.2-4.fc12.src.rpm

 %check
 mpd --daemon
 %{_openmpi_load}
 PYTHONPATH=%{buildroot}%{python_sitearch}/openmpi \
     python test/runalltest.py
 %{_openmpi_unload}

mpd is the MPICH2 manager (Open MPI does not need one); move the mpd statement to the MPICH2 section. Looks like the build is failing since MPICH2 is not working for some reason.

Furthermore the %check is ran in the build dir, so PYTHONPATH should be set to
 PYTHONPATH=openmpi/
for the Open MPI version etc.

Doesn't work locally, either:

+ mpd --daemon
configuration file /builddir/.mpd.conf not found
A file named .mpd.conf file must be present in the user's home
directory (/etc/mpd.conf if root) with read and write access
only for the user, and must contain at least a line with:
MPD_SECRETWORD=<secretword>
One way to safely create this file is to do the following:
  cd $HOME
  touch .mpd.conf
  chmod 600 .mpd.conf
and then use an editor to insert a line like
  MPD_SECRETWORD=mr45-j9z
into the file.  (Of course use some other secret word than mr45-j9z.)
error: Bad exit status from /var/tmp/rpm-tmp.L5Y4a4 (%check)
    Bad exit status from /var/tmp/rpm-tmp.L5Y4a4 (%check)
RPM build errors:


You should run the mpd command right after %{_mpich2_load}. The MPICH2 package is still blatantly in violation of the guidelines, which is why you can run MPICH2 commands without loading the MPICH2 module.

(In reply to comment #8)
> Furthermore the %check is ran in the build dir, so PYTHONPATH should be set to
>  PYTHONPATH=openmpi/
> for the Open MPI version etc.    

If ever, than it should be PYTHONPATH=openmpi/lib.linux-x86_64-2.6, so maybe PYTHONPATH=openmpi/lib.linux-* ?

I don't like that '*' and would prefer running it, when installed.
Or how do you like the '*' from above?

(In reply to comment #9)
> Doesn't work locally, either:
> 
> + mpd --daemon
> configuration file /builddir/.mpd.conf not found
> A file named .mpd.conf file must be present in the user's home
> directory (/etc/mpd.conf if root) with read and write access
> only for the user, and must contain at least a line with:
> MPD_SECRETWORD=<secretword>
> One way to safely create this file is to do the following:
>   cd $HOME
>   touch .mpd.conf
>   chmod 600 .mpd.conf
> and then use an editor to insert a line like
>   MPD_SECRETWORD=mr45-j9z
> into the file.  (Of course use some other secret word than mr45-j9z.)
> error: Bad exit status from /var/tmp/rpm-tmp.L5Y4a4 (%check)
>     Bad exit status from /var/tmp/rpm-tmp.L5Y4a4 (%check)
> RPM build errors:

ok, I already have such a file here :(

I don't want to add such a file on the buildsystem and without it won't work.
How about disabling the mpich2 testsuite, till there is a 'sane' possibility?

> You should run the mpd command right after %{_mpich2_load}. The MPICH2 package
> is still blatantly in violation of the guidelines, which is why you can run
> MPICH2 commands without loading the MPICH2 module.    

Done, works for me locally, but in koji fails becaus openmpi can't be initialized...

+ PYTHONPATH=/builddir/build/BUILDROOT/mpi4py-1.2-5.fc13.x86_64/usr/lib64/python2.6/site-packages/openmpi
+ python test/runalltest.py
[x86-03.phx2.fedoraproject.org:18162] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file ess_hnp_module.c at line 161
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
  orte_plm_base_select failed
  --> Returned value Not found (-13) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
[x86-03.phx2.fedoraproject.org:18162] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file runtime/orte_init.c at line 132
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
  orte_ess_set_name failed
  --> Returned value Not found (-13) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
[x86-03.phx2.fedoraproject.org:18162] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file orted/orted_main.c at line 323
[x86-03.phx2.fedoraproject.org:18161] [[INVALID],INVALID] ORTE_ERROR_LOG: Unable to start a daemon on the local node in file ess_singleton_module.c at line 381
[x86-03.phx2.fedoraproject.org:18161] [[INVALID],INVALID] ORTE_ERROR_LOG: Unable to start a daemon on the local node in file ess_singleton_module.c at line 143
[x86-03.phx2.fedoraproject.org:18161] [[INVALID],INVALID] ORTE_ERROR_LOG: Unable to start a daemon on the local node in file runtime/orte_init.c at line 132
RPM build errors:
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
  orte_ess_set_name failed
  --> Returned value Unable to start a daemon on the local node (-128) instead of ORTE_SUCCESS


Don't know what to do here, except disabling the tests complete...

Disabled for now:
Spec URL: http://tomspur.fedorapeople.org/review/mpi4py.spec
SRPM URL: http://tomspur.fedorapeople.org/review/mpi4py-1.2-5.fc12.src.rpm

Does the check work in your local build?

Check what libraries the python modules link to. My guess is that you've linked also the Open MPI version against mpich2, and it fails due to that.

(In reply to comment #11)
> Does the check work in your local build?

Yes.

> 
> Check what libraries the python modules link to. My guess is that you've linked
> also the Open MPI version against mpich2, and it fails due to that.    

rpmbuild says nope:
Processing files: mpi4py-openmpi-1.2-5.fc12.x86_64
Provides: MPE.so()(64bit) MPI.so()(64bit) dl.so()(64bit) mpi4py-runtime = 1.2-5.fc12
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PartialHardlinkSets) <= 4.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(VersionedDependencies) <= 3.0.3-1
Requires: libc.so.6()(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libdl.so.2()(64bit) libdl.so.2(GLIBC_2.2.5)(64bit) libm.so.6()(64bit) libmpi.so.0()(64bit) libnsl.so.1()(64bit) libopen-pal.so.0()(64bit) libopen-rte.so.0()(64bit) libpthread.so.0()(64bit) libpython2.6.so.1.0()(64bit) libutil.so.1()(64bit) python(abi) = 2.6 rtld(GNU_HASH)
Processing files: mpi4py-mpich2-1.2-5.fc12.x86_64
Provides: MPE.so()(64bit) MPI.so()(64bit) dl.so()(64bit) mpi4py-runtime = 1.2-5.fc12
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PartialHardlinkSets) <= 4.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(VersionedDependencies) <= 3.0.3-1
Requires: libc.so.6()(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libmpich.so.1.2()(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) libpython2.6.so.1.0()(64bit) librt.so.1()(64bit) python(abi) = 2.6 rtld(GNU_HASH)


From opempi %build:
/usr/lib64/openmpi/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -I/usr/include/python2.6 -c _configtest.c -o _configtest.o

From mpich2 %build:
/usr/lib64/mpich2/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -I/usr/include/python2.6 -c _configtest.c -o _configtest.o

If there is a bug in /usr/lib64/openmpi/bin/mpicc that links against mpich2, rpm would not detect the R: libmpi.so.0()(64bit)

Hi, I'm the author of mpi4py and a Fedora user since the project started. I would like to make some comments on the way mpi4py is being packaged and propose some cleanups in the spec file.

First of all, I'm preparing a bug-fix release being tracked in https://mpi4py.googlecode.com/svn/branches/release-1.2 . Up to now, the accumulated fixes are all related to the custom distutils-based buildsystem and the removal of a hidden file in the documentation (I actually noticed this after looking at mpi4py.spec). Additionally, I've disabled some failing tests for Open MPI (after discussing about the failure in the Open MPI devel ML). Hopefully, I'll make a bug-fix release soon, and we will have time to get mpi4py-1.2.1 in Fedora 13.

1) Please DO NOT REMOVE all empty pyx/pxd files!!. In particular, the files '__init__.pyx' and '__init__.pxd' located at 'src/include/mpi4py' DO ARE REQUIRED for Cython support (i.e. for accessing mpi4py internals at the C level when using Cython's "cimport" statement). If these empty files cause any issue, they could be added a single comment line, let say "# placeholder" or something like that.

2) I would remove the whole 'docs/source/slides' directory. BTW, the contents of 'docs/source/usrman' are the reST sources from which the 'docs/usrman/*' HTML documentation (and 'docs/mpi4py.pdf') are generated (using Sphinx and Latex)... So perhaps the whole directory 'docs/source' could be removed, though the  almost-plain-text reST sources at 'docs/source/usrman' could be handy.

3) mpi4py's custom, distutils-based buildsystem (conf/mpidistutils.py) already handles MPI compiler wrappers mpicc/mpicxx (as long as they can be found in $PATH), so there is not need to "export CC=mpicc CXX=mpicxx".


4) mpi4py DO SUPPORT Python 3. Moreover, the testsuite should also run and all tests pass. Of course, about half the tests will not run because of missing numpy, but the other half will use Python's builtin "array.array" instances.

(In reply to comment #13)
> Hi, I'm the author of mpi4py and a Fedora user since the project started. I
> would like to make some comments on the way mpi4py is being packaged and
> propose some cleanups in the spec file.

I could not find you in FAS, so I assume, you are not in the packager group. If so, you could co-maintain this package (or I co-maintain it ;) ).

> 1) Please DO NOT REMOVE all empty pyx/pxd files!

Didn't know that about the cython support. Atm, I just use python.
I deleted that part.

> 2) I would remove the whole 'docs/source/slides' directory. BTW, the contents
> of 'docs/source/usrman' are the reST sources from which the 'docs/usrman/*'
> HTML documentation (and 'docs/mpi4py.pdf') are generated (using Sphinx and
> Latex)... So perhaps the whole directory 'docs/source' could be removed, though
> the  almost-plain-text reST sources at 'docs/source/usrman' could be handy.

docs/source deleted.

> 3) mpi4py's custom, distutils-based buildsystem (conf/mpidistutils.py) already
> handles MPI compiler wrappers mpicc/mpicxx (as long as they can be found in
> $PATH), so there is not need to "export CC=mpicc CXX=mpicxx".

According to http://fedoraproject.org/wiki/PackagingDrafts/MPI
that should be there. Furthermore this does not destroy something, so I leave it there, untill Jussi disagrees.

> 4) mpi4py DO SUPPORT Python 3. Moreover, the testsuite should also run and all
> tests pass. Of course, about half the tests will not run because of missing
> numpy, but the other half will use Python's builtin "array.array" instances.    

Yes, unfortunately there were some issues on the buildsystem with the tests.
The mpich tests can't be run, because of the mpd issue, so I'll run them here locally and try to run the rest olso on the buildsystem, but they failed completely:
"""
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
  orte_plm_base_select failed
  --> Returned value Not found (-13) instead of ORTE_SUCCESS
"""

-> Disabled completely. Furthermore the testsuite runs just fine on my system.

Thanks for your suggestions and infomations :)

Spec URL: http://tomspur.fedorapeople.org/review/mpi4py.spec
SRPM URL: http://tomspur.fedorapeople.org/review/mpi4py-1.2-6.fc12.src.rpm

(In reply to comment #14)
> > 3) mpi4py's custom, distutils-based buildsystem (conf/mpidistutils.py) already
> > handles MPI compiler wrappers mpicc/mpicxx (as long as they can be found in
> > $PATH), so there is not need to "export CC=mpicc CXX=mpicxx".
> 
> According to http://fedoraproject.org/wiki/PackagingDrafts/MPI
> that should be there. Furthermore this does not destroy something, so I leave
> it there, untill Jussi disagrees.

As long as the build uses mpicc and mpicxx everything is fine. Still, having the declarations explicitly doesn't hurt you at all..

Sorry, I've been busy for some time. Here's the review:

rpmlint output:
mpi4py-mpich2.x86_64: W: no-documentation
mpi4py-mpich2.x86_64: E: non-standard-executable-perm /usr/lib64/python2.6/site-packages/mpich2/mpi4py/dl.so 0775
mpi4py-mpich2.x86_64: E: non-standard-executable-perm /usr/lib64/python2.6/site-packages/mpich2/mpi4py/MPE.so 0775
mpi4py-mpich2.x86_64: E: zero-length /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/__init__.pxd
mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.MPI_api.h
mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.MPI.h
mpi4py-mpich2.x86_64: E: non-standard-executable-perm /usr/lib64/python2.6/site-packages/mpich2/mpi4py/MPI.so 0775
mpi4py-mpich2.x86_64: E: zero-length /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/__init__.pyx
mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.h
mpi4py-openmpi.x86_64: W: no-documentation
mpi4py-openmpi.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/openmpi/mpi4py/include/mpi4py/mpi4py.MPI_api.h
mpi4py-openmpi.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/openmpi/mpi4py/include/mpi4py/mpi4py.h
mpi4py-openmpi.x86_64: E: non-standard-executable-perm /usr/lib64/python2.6/site-packages/openmpi/mpi4py/MPI.so 0775
mpi4py-openmpi.x86_64: E: zero-length /usr/lib64/python2.6/site-packages/openmpi/mpi4py/include/mpi4py/__init__.pyx
mpi4py-openmpi.x86_64: E: non-standard-executable-perm /usr/lib64/python2.6/site-packages/openmpi/mpi4py/MPE.so 0775
mpi4py-openmpi.x86_64: E: non-standard-executable-perm /usr/lib64/python2.6/site-packages/openmpi/mpi4py/dl.so 0775
mpi4py-openmpi.x86_64: E: zero-length /usr/lib64/python2.6/site-packages/openmpi/mpi4py/include/mpi4py/__init__.pxd
mpi4py-openmpi.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/openmpi/mpi4py/include/mpi4py/mpi4py.MPI.h
python3-mpi4py-mpich2.x86_64: W: no-documentation
python3-mpi4py-mpich2.x86_64: E: non-standard-executable-perm /usr/lib64/python3.1/site-packages/mpich2/mpi4py/dl.so 0775
python3-mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python3.1/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.h
python3-mpi4py-mpich2.x86_64: E: non-standard-executable-perm /usr/lib64/python3.1/site-packages/mpich2/mpi4py/MPE.so 0775
python3-mpi4py-mpich2.x86_64: E: zero-length /usr/lib64/python3.1/site-packages/mpich2/mpi4py/include/mpi4py/__init__.pyx
python3-mpi4py-mpich2.x86_64: E: non-standard-executable-perm /usr/lib64/python3.1/site-packages/mpich2/mpi4py/MPI.so 0775
python3-mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python3.1/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.MPI.h
python3-mpi4py-mpich2.x86_64: E: zero-length /usr/lib64/python3.1/site-packages/mpich2/mpi4py/include/mpi4py/__init__.pxd
python3-mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python3.1/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.MPI_api.h
python3-mpi4py-openmpi.x86_64: W: no-documentation
python3-mpi4py-openmpi.x86_64: E: non-standard-executable-perm /usr/lib64/python3.1/site-packages/openmpi/mpi4py/dl.so 0775
python3-mpi4py-openmpi.x86_64: E: non-standard-executable-perm /usr/lib64/python3.1/site-packages/openmpi/mpi4py/MPE.so 0775
python3-mpi4py-openmpi.x86_64: E: non-standard-executable-perm /usr/lib64/python3.1/site-packages/openmpi/mpi4py/MPI.so 0775
python3-mpi4py-openmpi.x86_64: E: zero-length /usr/lib64/python3.1/site-packages/openmpi/mpi4py/include/mpi4py/__init__.pyx
python3-mpi4py-openmpi.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python3.1/site-packages/openmpi/mpi4py/include/mpi4py/mpi4py.MPI_api.h
python3-mpi4py-openmpi.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python3.1/site-packages/openmpi/mpi4py/include/mpi4py/mpi4py.MPI.h
python3-mpi4py-openmpi.x86_64: E: zero-length /usr/lib64/python3.1/site-packages/openmpi/mpi4py/include/mpi4py/__init__.pxd
python3-mpi4py-openmpi.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python3.1/site-packages/openmpi/mpi4py/include/mpi4py/mpi4py.h
8 packages and 0 specfiles checked; 20 errors, 16 warnings.

MUST: Fix the permissions, they should be 755 for libraries.


***

MUST: The package does not yet exist in Fedora. The Review Request is not a duplicate. OK

MUST: The spec file for the package is legible and macros are used consistently. OK
- I think the 0%{?rhel}>6 should be 0%{?rhel}>5, since RHEL 6 hasn't even been published yet. The python3 draft isn't really consistent in this.

MUST: The package must be named according to the Package Naming Guidelines. OK
MUST: The spec file name must match the base package %{name}. OK
MUST: The package must be licensed with a Fedora approved license and meet the  Licensing Guidelines. OK

MUST: The License field in the package spec file must match the actual license. OK
- Source code doesn't contain license headers.
- Attached LICENSE.txt does, however, specify the license to be BSD.

MUST: The sources used to build the package must match the upstream source, as provided in the spec URL. OK
MUST: The package MUST successfully compile and build into binary rpms. OK
MUST: The spec file MUST handle locales properly. N/A
MUST: Optflags are used and time stamps preserved. OK
MUST: Packages containing shared library files must call ldconfig. N/A
MUST: A package must own all directories that it creates or require the package that owns the directory. OK
MUST: Files only listed once in %files listings. OK
MUST: Debuginfo package is complete. OK

MUST: Permissions on files must be set properly. NEEDSWORK
- Fix the lib perms with chmod at the end of %install.

MUST: Clean section exists. OK
MUST: Large documentation files must go in a -doc subpackage. OK

MUST: All relevant items are included in %doc. Items in %doc do not affect runtime of application.  ~OK
- I suggest adding PKG-INFO and THANKS.txt to -common.

MUST: Header files must be in a -devel package. OK
- This doesn't really apply here.

MUST: Static libraries must be in a -static package. N/A
MUST: Packages containing pkgconfig(.pc) files must 'Requires: pkgconfig'.
MUST: If a package contains library files with a suffix then library files ending in .so must go in a -devel package. N/A
MUST: In the vast majority of cases, devel packages must require the base package using a fully versioned dependency. N/A
MUST: Packages does not contain any .la libtool archives. N/A
MUST: Desktop files are installed properly. N/A
MUST: No file conflicts with other packages and no general names. OK
MUST: Buildroot cleaned before install. OK
SHOULD: %{?dist} tag is used in release. OK
SHOULD: If the package does not include license text(s) as separate files from upstream, the packager should query upstream to include it. OK
SHOULD: The package builds in mock. OK

Fix the issues before import to CVS. This package has been

APPROVED

Huh? The permissions are ok here...

$ rpmls x86_64/mpi4py-mpich2-1.2-6.fc12.x86_64.rpm 
drwxr-xr-x  /usr/lib64/python2.6/site-packages/mpich2/mpi4py
-rw-r--r--  /usr/lib64/python2.6/site-packages/mpich2/mpi4py-1.2-py2.6.egg-info
-rwxr-xr-x  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/MPE.so
-rwxr-xr-x  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/MPI.so
-rw-r--r--  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/__init__.py
-rw-r--r--  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/__init__.pyc
-rw-r--r--  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/__init__.pyo
-rwxr-xr-x  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/dl.so
[snip]

755 should be -rwxr-xr-x AFAIK, or am I wrong here?

Example rpmlint on the same rpm:
$ rpmlint x86_64/mpi4py-mpich2-1.2-6.fc12.x86_64.rpm 
mpi4py-mpich2.x86_64: W: spelling-error %description -l en_US picklable -> pickle, pickerel, despicable
mpi4py-mpich2.x86_64: W: spelling-error %description -l en_US builtin -> built in, built-in, built
mpi4py-mpich2.x86_64: W: spelling-error %description -l en_US mpi -> moi, mp, mi
mpi4py-mpich2.x86_64: W: spelling-error %description -l en_US py -> pt, p, y
mpi4py-mpich2.x86_64: W: no-documentation
mpi4py-mpich2.x86_64: E: zero-length /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/__init__.pxd
mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.MPI_api.h
mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.MPI.h
mpi4py-mpich2.x86_64: E: zero-length /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/__init__.pyx
mpi4py-mpich2.x86_64: W: devel-file-in-non-devel-package /usr/lib64/python2.6/site-packages/mpich2/mpi4py/include/mpi4py/mpi4py.h
1 packages and 0 specfiles checked; 2 errors, 8 warnings.

only ignorable warnings/errors


I will request a cvs entry, but will only import, when I get another answer from you.

Thanks for the review.

______________________

New Package CVS Request
=======================
Package Name: mpi4py
Short Description: Python bindings of the Message Passing Interface (MPI)
Owners: tomspur
Branches: F-12 F-13
InitialCC:

(In reply to comment #18)
> Huh? The permissions are ok here...

That's what I got when I mock built the srpm in rawhide x86_64. How did you build yours?

> Example rpmlint on the same rpm:
> $ rpmlint x86_64/mpi4py-mpich2-1.2-6.fc12.x86_64.rpm 
> mpi4py-mpich2.x86_64: W: spelling-error %description -l en_US picklable ->
> pickle, pickerel, despicable

Hmm, I haven't seen a spell checker in rpmlint before!

Excerpt:

$ rpmls /var/mock/fedora-rawhide-x86_64/result/mpi4py-mpich2-1.2-6.fc13.x86_64.rpm 
drwxr-xr-x  /usr/lib64/python2.6/site-packages/mpich2/mpi4py
-rw-r--r--  /usr/lib64/python2.6/site-packages/mpich2/mpi4py-1.2-py2.6.egg-info
-rwxrwxr-x  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/MPE.so
-rwxrwxr-x  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/MPI.so
-rw-r--r--  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/__init__.py
-rw-r--r--  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/__init__.pyc
-rw-r--r--  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/__init__.pyo
-rwxrwxr-x  /usr/lib64/python2.6/site-packages/mpich2/mpi4py/dl.so

so the permissions really are incorrect.

I have no clue about why the permissions on the *.so files are wrong in Jussi's box.

I also saw that rpmlint complains (signaling an error, not a warning) about empty files. What's wrong with empty files to that being an error? If required, I could add a comment line at the start of these files, in order to 'fill' them and make rpmlint stop complaining.


BTW, Thomas... Could you make sure that the MPE extension module actually works on MPICH2+x86_64 ?? For example, after installing mpi4py (or adjusting PYTHONPATH to point to the build directory), run this: 

$ make -C demo/mpe-logging

If this does not work, that's surely related to libmpe.a from MPICH2 not being built with -fPIC.

(In reply to comment #22)
> I have no clue about why the permissions on the *.so files are wrong in Jussi's
> box.

I can reproduce this in my local mock + building for rawhide. But because this seems to be resolved on the buildsystem, so I will verify the build everytime in koji, but don't search here on my system for a bug.

> I also saw that rpmlint complains (signaling an error, not a warning) about
> empty files. What's wrong with empty files to that being an error? If required,
> I could add a comment line at the start of these files, in order to 'fill' them
> and make rpmlint stop complaining.

http://fedoraproject.org/wiki/Common_Rpmlint_issues#zero-length
I guess, this is a  packaging guidelines, because this broke something somewhere.
rpmlint sometimes shows false positives, so it would be ok to leave the errors.

> BTW, Thomas... Could you make sure that the MPE extension module actually works
> on MPICH2+x86_64 ??

$ PYTHONPATH=/usr/lib64/python2.6/site-packages/mpich2/ make -C demo/mpe-logging
make: Entering directory `/home/tom/rpmbuild/MOPENED/mpi4py/mpi4py-1.2/demo/mpe-logging'
mpiexec -n 8 python cpilog.py
Writing logfile....
Enabling the Default clock synchronization...
Finished writing logfile cpilog.clog2.
mpiexec -n 8 python ring.py
Writing logfile....
Enabling the Default clock synchronization...
Finished writing logfile ring.clog2.
mpiexec -n 8 python threads.py
Writing logfile....
Enabling the Default clock synchronization...
Finished writing logfile threads.clog2.
rm -f *.[cs]log2
make: Leaving directory `/home/tom/rpmbuild/MOPENED/mpi4py/mpi4py-1.2/demo/mpe-logging'

> If this does not work, that's surely related to libmpe.a from MPICH2 not being
> built with -fPIC.    

From the spec file of mpich2, you can see, that on x86_64, it's explicit build with -fPIC, but not on any other platform.
But because this fails on x86_64 in the buildsystem too, this should be something different :(

(In reply to comment #23)
> > I also saw that rpmlint complains (signaling an error, not a warning) about
> > empty files. What's wrong with empty files to that being an error? If required,
> > I could add a comment line at the start of these files, in order to 'fill' them
> > and make rpmlint stop complaining.
> 
> http://fedoraproject.org/wiki/Common_Rpmlint_issues#zero-length
> I guess, this is a  packaging guidelines, because this broke something
> somewhere.
> rpmlint sometimes shows false positives, so it would be ok to leave the errors.
> 

OK. After looking around in my F12 box, I can see many empty __init__.py files inside the system Python install. So I'll leave my *.pyx/*.pyd empty, too.

> > BTW, Thomas... Could you make sure that the MPE extension module actually works
> > on MPICH2+x86_64 ??
> 
> $ PYTHONPATH=/usr/lib64/python2.6/site-packages/mpich2/ make -C
> demo/mpe-logging
> make: Entering directory
> `/home/tom/rpmbuild/MOPENED/mpi4py/mpi4py-1.2/demo/mpe-logging'
> mpiexec -n 8 python cpilog.py
> Writing logfile....
> Enabling the Default clock synchronization...
> Finished writing logfile cpilog.clog2.
> mpiexec -n 8 python ring.py
> Writing logfile....
> Enabling the Default clock synchronization...
> Finished writing logfile ring.clog2.
> mpiexec -n 8 python threads.py
> Writing logfile....
> Enabling the Default clock synchronization...
> Finished writing logfile threads.clog2.
> rm -f *.[cs]log2
> make: Leaving directory
> `/home/tom/rpmbuild/MOPENED/mpi4py/mpi4py-1.2/demo/mpe-logging'
> 

OK. It works.

> > If this does not work, that's surely related to libmpe.a from MPICH2 not being
> > built with -fPIC.    
> 
> From the spec file of mpich2, you can see, that on x86_64, it's explicit build
> with -fPIC, but not on any other platform.
>

OK. I've mailed mpich2-dev about this issue some time ago. They more or less agreed that -fPIC should be used for building objects going to libmpe.a in the case of sharedlib-enabled MPICH2 build.

> But because this fails on x86_64 in the buildsystem too, this should be
> something different :(    

Sorry, could you point to any link where I can take a look?

(In reply to comment #24)
> > But because this fails on x86_64 in the buildsystem too, this should be
> > something different :(    
> 
> Sorry, could you point to any link where I can take a look?    

It's posted in comment #10.

That's the same fault in a new scratch build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=2011533

Link to build.log:
http://koji.fedoraproject.org/koji/getfile?taskID=2011534&name=build.log

  orte_plm_base_select failed
  --> Returned value Not found (-13) instead of ORTE_SUCCESS
and
  orte_ess_set_name failed
  --> Returned value Not found (-13) instead of ORTE_SUCCESS

Searching in the web didn't help /me...

Could you update the spec to run the testsuite like this:

mpiexec -n 1 python test/runalltest.py \
    --path=%{buildroot}%{python_sitearch}/openmpi


Additionally, you could run the testsuite in parallel, let say usoing 4 procs:


mpiexec -n 4 python test/runalltest.py \
    --path=%{buildroot}%{python_sitearch}/openmpi


BTW, you should do the same for the mpich2 and python3 stuff.

Given all of the additional discussion, I feel I should ask if this is ready
for CVS or if it needs more review.

(In reply to comment #27)
> Given all of the additional discussion, I feel I should ask if this is ready
> for CVS or if it needs more review.    

There are currently two issues:
- the permissions might be wrong (*.so files)
- the tests can't run in the buildsystem

I'll check that in the buildsystem, once build and adjust the spec, if nessesary.

I'll try to adjust the tests a bit and to run them in the future, but this is not really related to a review issue. Tests should be run, but in this case it's not that easily possible...
(The discussion about the tests should be maybe discussed somewhere else, maybe that's why it looks like needing more review.)

So, I'd say 'ready for CVS'.

CVS done (by process-cvs-requests.py).

The scratch build has right permissions:
http://koji.fedoraproject.org/koji/taskinfo?taskID=2015955

Now doing the right build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=2015966

Also with right permissions.

_____________________________


There are remaining problems with the testsuite, that will hopefully be shoot down in the future.

e.g. mpich2 currently can't be tested, because it needs a DNS lookup, what the buildsystem blocks. Maybe that's why openmpi also bracks.
(It seems, both are not designed to work in a non-network atm...)

mpi4py-1.2.1-1.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/mpi4py-1.2.1-1.fc12

mpi4py-1.2.1-1.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/mpi4py-1.2.1-1.fc13

mpi4py-1.2.1-2.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/F13/FEDORA-2010-3100

mpi4py-1.2.1-2.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/mpi4py-1.2.1-2.fc12

mpi4py-1.2.1-2.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.

mpi4py-1.2.1-2.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Package Change Request
======================
Package Name: mpi4py
New Branches: el6
Owners: tomspur

Git done (by process-git-requests).

mpi4py-1.3-5.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mpi4py-1.3-5.el6

mpi4py-1.3-5.el6 has been pushed to the Fedora EPEL 6 stable repository.