Bug 511099

Summary: install ld configuration file again
Product: [Fedora] Fedora Reporter: Milos Jakubicek <xjakub>
Component: openmpiAssignee: Doug Ledford <dledford>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: high    
Version: rawhideCC: dledford, susi.lehtola
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-07-21 16:39:59 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Milos Jakubicek 2009-07-13 16:33:05 UTC
Description of problem:

Prior to openmpi-1.3.1-2.fc12, an ld configuration file was installed under %{_sysconfdir}/ld.so.conf.d with openmpi-libs, but now there is not such file anymore which makes binaries compiled against openmpi crashing when trying to execute them (linker will not find the linked library). Please put the configuration file back as soon as possible.

Version-Release number of selected component (if applicable):

openmpi-1.3.1-3.fc12.x86_64

How reproducible:

Always

Steps to Reproduce:
1. echo "int main(){}" > test.cc
2. g++ -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -DHAVE_INLINE -m64 -I/usr/lib64/openmpi/1.3.1-gcc/include  -L/usr/lib64/openmpi/1.3.1-gcc/lib -lmpi test.cc
3. ./a.out
  
Actual results:

./a.out: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory

Comment 1 Doug Ledford 2009-07-21 16:39:59 UTC
This is caused by the need for multiple MPI implementations (currently, lam, openmpi, and mpich2, with mvapich and mvapic2 possibly coming as well).  Configuration of multiple MPI stacks is beyond what alternatives can do, especially given the need for man pages, runtimes, and other things to go along with the libraries.  In addition, alternatives is really intended for system wide defaults while MPI implementation selection is usually needed on a per user and sometimes a per application basis.

The recommended way to handle libraries or applications that might link against an MPI is to do a multi-stage build process with files installed in different locations.  For example, given libfoo that can be built with or without mpi support, do something like this:

Name: libfoo

%package openmpi
BuildRequires: openmpi-devel

%package mpich2
BuildRequires: mpich2-devel

%build
# Have to build in the install step to do multiple builds

%install
# Make the non-MPI version first
%configure
make
make DESTDIR=%{buildroot} install
make clean
. /etc/profile.d/modules.sh
module load openmpi-%{_arch}
%configure --with-mpi --prefix=$MPI_HOME --bindir=$MPI_BIN --libdir=$MPI_LIB
make CC=mpicc
make DESTDIR=%{buildroot} install
make clean
module unload openmpi-%{_arch}
module load mpich2-%{_arch}
%configure --with-mpi --prefix=$MPI_HOME --bindir=$MPI_BIN --libdir=$MPI_LIB
make CC=mpicc
make DESTDIR=%{buildroot} install

%files #All the normal files

%files openmpi #All openmpi linked files

%files mpich2 #All mpich2 linked files

By putting the files linked against a given mpi implementation in the same bin and lib directories as the mpi binaries themselves, you'll accomplish having a system where when a user uses the library without first loading an mpi implementation, then they'll get the standard non-mpi version.  If, instead, they load an mpi implementation first, then the files in MPI_BIN and MPI_LIB will override the system wide defaults and the user will get the mpi enabled version that goes with the implementation they loaded.

Comment 2 Milos Jakubicek 2009-07-22 14:09:02 UTC
Doug, thank you for your detailed explanation.

Comment 3 Susi Lehtola 2009-07-24 12:49:39 UTC
(In reply to comment #0)
> Steps to Reproduce:
> 1. echo "int main(){}" > test.cc
> 2. g++ -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
> -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -DHAVE_INLINE
> -m64 -I/usr/lib64/openmpi/1.3.1-gcc/include  -L/usr/lib64/openmpi/1.3.1-gcc/lib
> -lmpi test.cc
> 3. ./a.out

I hope you're not really running the above, but instead are using the 'mpicxx' wrapper which includes all of the necessary includes and libraries...

$ mpicxx --show
g++ -I/usr/lib/openmpi/1.3.3-gcc/include -pthread -m32 -L/usr/lib/openmpi/1.3.3-gcc/lib -lmpi_cxx -lmpi -lopen-rte -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl

Comment 4 Milos Jakubicek 2009-07-24 19:36:37 UTC
Yes of course, that was just to set an example. You can have a look here (any feedback appreciated):
https://koji.fedoraproject.org/koji/taskinfo?taskID=1497947