Bug 635742 - external program cannot find mpi4py
Summary: external program cannot find mpi4py
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mpich2
Version: 12
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Thomas Spura
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-09-20 15:40 UTC by David Jansen
Modified: 2010-10-28 06:00 UTC (History)
4 users (show)

Fixed In Version: mpich2-1.2.1p1-10.fc14
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-10-22 18:07:08 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description David Jansen 2010-09-20 15:40:36 UTC
Description of problem:
I'm not sure if this qualifies as a bug in the Fedora package, or that the program that I'm installing, doesn't follow standards. So maybe this is a request for help and clarification and not a bug report.

I'm installing an external, non-rpm, program (amuse, from www.amusecode.org) that requires mpich2 and mpi4py. So, I installed those packages from Fedora, but the program setup still doesn't find it.
basically, the program does:
  self.check_module("mpi4py", lambda module : module.__version__, '1.1.0')
to verify presence and version of mpi4py, and that fails.
This seems to be due to the fact that the fedora package installs its files in /usr/lib64/python2.6/site-packages/mpich2/mpi4py whereas installation from source installs in /usr/lib64/python2.6/site-packages/mpi4py (and, in that case, installation can proceed). 
If I do: 
  ln -s mpich2/mpi4py /usr/lib64/python2.6/site-packages/
the test succeeds and I can install the program, but of course this is an ugly workaround.

Any help?

Comment 1 Lisandro Dalcin 2010-09-20 15:58:05 UTC
I think this is actually a bug in the mpich2 package. Take a look at the module contents:

$ module show mpich2-x86_64
-------------------------------------------------------------------
/usr/share/Modules/modulefiles/mpich2-x86_64:

setenv		 MPI_BIN /usr/lib64/mpich2/bin 
setenv		 MPI_SYSCONFIG /etc/mpich2-x86_64 
setenv		 MPI_FORTRAN_MOD_DIR /usr/include/mpich2-x86_64 
setenv		 MPI_INCLUDE /usr/include/mpich2-x86_64 
setenv		 MPI_LIB /usr/lib64/mpich2/lib 
setenv		 MPI_MAN /usr/share/man/mpich2 
setenv		 MPI_PYTHON_SITEARCH /usr/lib64/python2.6/site-packages/mpich2 
setenv		 MPI_COMPILER mpich2-x86_64 
setenv		 MPI_SUFFIX _mpich2 
setenv		 MPI_HOME /usr/lib64/mpich2 
prepend-path	 PATH /usr/lib64/mpich2/bin 
prepend-path	 LD_LIBRARY_PATH /usr/lib64/mpich2/lib 
prepend-path	 MANPATH /usr/share/man/mpich2 
-------------------------------------------------------------------

I think a line is missing:
 
prepend-path	 PYTHONPATH /usr/lib64/python2.6/site-packages/mpich2

If you and that line at the very end of file "/usr/share/Modules/modulefiles/mpich2-x86_64", things should work. This fix would be more appropriate than your symlink workaround.

PS: Not sure how to ping MPICH2 package maintainers about this...

Comment 2 David Jansen 2010-09-20 16:18:21 UTC
Thanks, that is a better workaround anyway. I'm not sure about filing a bug with the mpich2 package however, after all, mpi4py is not a requirement for mpich2, so mpich2 has no obligation to set up the environment for mpi4py, i think.

Comment 3 Lisandro Dalcin 2010-09-20 16:28:30 UTC
That requirement is not only for mpi4py, but for EVERY Python package that also depends on MPI. Take a look at the MPI packaging guidelines: http://fedoraproject.org/wiki/Packaging/MPI. You will see this text:

    The module file MUST prepend ... PYTHONPATH to %{python_sitearch}/%{name}%{?_cc_name_suffix}/

IIUC, MPICH2 is violating the Fedora packaging guidelines for MPI implementations.

Comment 4 David Jansen 2010-09-20 16:30:18 UTC
Yes, you're right, so it is a MPICH2 packaging bug and I'll file it there. Thanks for your quick help!

Comment 5 Thomas Spura 2010-09-21 09:39:09 UTC
Lisandro, thanks for tracking this down.

____________________________________________


This issue is already "in progress" for F-13 in bug #542745, but there doesn't seem to be much progress anymore. :-(

Deji: ping?

This shouldn't be more than just:
$ git diff
diff --git a/mpich2-modules.patch b/mpich2-modules.patch
index 357b815..036ce19 100644
--- a/mpich2-modules.patch
+++ b/mpich2-modules.patch
@@ -1,6 +1,6 @@
 --- src/packaging/envmods/mpich2.module.in     2009-07-02 15:54:58.000000000 -0
 +++ src/packaging/envmods/mpich2.module.in.new 2009-11-09 10:37:03.524042574 -0
-@@ -4,11 +4,16 @@
+@@ -4,11 +4,17 @@
  # 
  
  # Define prefix so PATH and MANPATH can be updated.
@@ -19,6 +19,7 @@
  prepend-path  PATH          @bindir@
 +prepend-path  LD_LIBRARY_PATH @libdir@
  prepend-path  MANPATH       @mandir@
++prepend-path  PYTHONPATH    @pysitearch@/@MPILIBNAME@2
 -
 -# Undefine prefix and exec_prefix which are too generic environment variables.
 -unsetenv      prefix

Comment 6 Fedora Update System 2010-10-11 22:36:02 UTC
mpich2-1.2.1p1-3.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/mpich2-1.2.1p1-3.fc13

Comment 7 Fedora Update System 2010-10-11 22:36:08 UTC
mpich2-1.2.1p1-9.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/mpich2-1.2.1p1-9.fc14

Comment 8 Fedora Update System 2010-10-11 22:36:14 UTC
mpich2-1.2.1p1-2.fc12 has been submitted as an update for Fedora 12.
https://admin.fedoraproject.org/updates/mpich2-1.2.1p1-2.fc12

Comment 9 Fedora Update System 2010-10-12 02:39:32 UTC
mpich2-1.2.1p1-9.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mpich2'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/mpich2-1.2.1p1-9.fc14

Comment 10 Fedora Update System 2010-10-22 18:07:04 UTC
mpich2-1.2.1p1-3.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 11 Fedora Update System 2010-10-28 06:00:10 UTC
mpich2-1.2.1p1-10.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.