Bug 647147 - Violation of MPI packaging guidelines: automatic loading of module in main package
Summary: Violation of MPI packaging guidelines: automatic loading of module in main pa...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mpich2
Version: rawhide
Hardware: Unspecified
OS: Unspecified
low
medium
Target Milestone: ---
Assignee: Deji Akingunola
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-10-27 11:36 UTC by Susi Lehtola
Modified: 2012-01-11 06:05 UTC (History)
2 users (show)

Fixed In Version: mpich2-1.4.1p1-4.fc16
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-01-11 06:04:03 UTC


Attachments (Terms of Use)

Description Susi Lehtola 2010-10-27 11:36:29 UTC
Hi,

could you *pretty please* split the files

/etc/profile.d/mpich2-x86_64.csh
/etc/profile.d/mpich2-x86_64.sh

into a subpackage, such as mpich2-autoload, as instructed in the MPI packaging guidelines?

The current system is troublesome when multiple MPI runtimes are installed. Furthermore, having the MPI compiler commands available is not desired, when one wants to build MPI-aware programs in serial mode (which is also required by the guideline).

Comment 1 Susi Lehtola 2010-10-27 11:39:58 UTC
If you like, you can add 
 Obsoletes: mpich2 <= mpich2-1.2.1p1-8
in mpich2-autoload, so that the behavior stays the same by default on current installations, however new installs would get the normal behavior (unless -autoload is explicitly installed).

Comment 2 Deji Akingunola 2010-10-28 16:40:50 UTC
(In reply to comment #0)
> Hi,
> 
> could you *pretty please* split the files
> 
> /etc/profile.d/mpich2-x86_64.csh
> /etc/profile.d/mpich2-x86_64.sh
> 
> into a subpackage, such as mpich2-autoload, as instructed in the MPI packaging
> guidelines?
>
I'm sorry no. I don't see the need for creating such a subpackage, because I would have made it a hard dependency for the mpich2 main package.
I believe users installing mpich2 should 'just' have access to mpicc and friends and mpiexec.

> The current system is troublesome when multiple MPI runtimes are installed.

What specific troubles are we talking about?

> Furthermore, having the MPI compiler commands available is not desired, when
> one wants to build MPI-aware programs in serial mode (which is also required by
> the guideline).
You can simply unload the mpich2 module then, and continue with building in your serial model.

Comment 3 Erik Zeek 2010-10-28 17:17:07 UTC
(In reply to comment #2)
> (In reply to comment #0)
> > Hi,
> > 
> > could you *pretty please* split the files
> > 
> > /etc/profile.d/mpich2-x86_64.csh
> > /etc/profile.d/mpich2-x86_64.sh
> > 
> > into a subpackage, such as mpich2-autoload, as instructed in the MPI packaging
> > guidelines?
> >
> I'm sorry no. I don't see the need for creating such a subpackage, because I
> would have made it a hard dependency for the mpich2 main package.
> I believe users installing mpich2 should 'just' have access to mpicc and
> friends and mpiexec.

What about users of OpenMPI?  I have both OpenMPI and Mpich2 installed (for development reasons), but prefer OpenMPI.  These files check to see if an MPI implementation has been already loaded.  I unload the Mpich module and load the OpenMPI module in my .bash_profile, but /etc/bashrc re-sources all of the files in /etc/profile.d/ loading the Mpich module yet again (neither module file has a conflict line).  Since the above files are not marked noreplace, I have to manually edit them after every Mpich upgrade.

> > The current system is troublesome when multiple MPI runtimes are installed.
> 
> What specific troubles are we talking about?

See above.

> > Furthermore, having the MPI compiler commands available is not desired, when
> > one wants to build MPI-aware programs in serial mode (which is also required by
> > the guideline).
> You can simply unload the mpich2 module then, and continue with building in
> your serial model.

Comment 4 Susi Lehtola 2010-10-28 17:19:44 UTC
(In reply to comment #2)
> I'm sorry no. I don't see the need for creating such a subpackage, because I
> would have made it a hard dependency for the mpich2 main package.
> I believe users installing mpich2 should 'just' have access to mpicc and
> friends and mpiexec.
> 
> > The current system is troublesome when multiple MPI runtimes are installed.
> 
> What specific troubles are we talking about?

You are making it a PITA for packagers and sysadmins alike, who need to use multiple MPI implementations for a reason or another. There's no sense in not being able to disable automatic loading.

Would you be happy with a system where you could only have *one* editor installed, since every package insisted they be the default editor to be used? 

The resulting outcome would be totally unpredictable; the result would depend on what packages are installed and which module is loaded last.

For instance loading another MPI runtime leaves MPICH2 variables in existence in the PATH, which may cause trouble with e.g. build systems that sniff out what kind of a system is used.

> > Furthermore, having the MPI compiler commands available is not desired, when
> > one wants to build MPI-aware programs in serial mode (which is also required by
> > the guideline).
> You can simply unload the mpich2 module then, and continue with building in
> your serial model.

Once again, this is incompatible with the existence of multiple MPI implementations. We hope packages to be built for every one of them that are available in Fedora, not just the one the primary package maintainer is used to.

Comment 5 Deji Akingunola 2010-10-28 18:42:37 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #0)

> 
> What about users of OpenMPI?  I have both OpenMPI and Mpich2 installed (for
> development reasons), but prefer OpenMPI.  These files check to see if an MPI
> implementation has been already loaded.  I unload the Mpich module and load the
> OpenMPI module in my .bash_profile, but /etc/bashrc re-sources all of the files
> in /etc/profile.d/ loading the Mpich module yet again 


Put the commands in .bashrc instead (below the section that sources /etc/bashrc)

.bash_profile sources .bashrc, which sources /etc/bashrc, which in turn reads thru /etc/profile.d .

My .bashrc look like below and does what you're wanting to do above;

>>>>
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# User specific aliases and functions
if [ -f $HOME/.profile ]; then
   if [ -z $ARMNLIB ]; then
        . $HOME/.profile
        . /opt/intel/bin/ifortvars.sh intel64
   fi
fi

module unload mpich2-x86_64
module load openmpi-x86_64
<<<<

Comment 6 Deji Akingunola 2010-10-28 18:52:23 UTC
(In reply to comment #4)

> > 
> > What specific troubles are we talking about?
> 
> You are making it a PITA for packagers and sysadmins alike, who need to use
> multiple MPI implementations for a reason or another. There's no sense in not
> being able to disable automatic loading.
> 
You are again making broad instantiated claims. Experienced users who want to use multiple MPI implementations can always 'module unload mpich2'.

> Would you be happy with a system where you could only have *one* editor
> installed, since every package insisted they be the default editor to be used? 
> 

That is a wrong analogy to use. A system can have vim, pine, and emacs all installed, and one can use any one of them without mucking with PATHs
 
> The resulting outcome would be totally unpredictable; the result would depend
> on what packages are installed and which module is loaded last.
> 
> For instance loading another MPI runtime leaves MPICH2 variables in existence
> in the PATH, which may cause trouble with e.g. build systems that sniff out
> what kind of a system is used.
> 
I don't see why having MPICH2 variable deep down a PATH is a problem. Any build system that have issues with that needs to be fixed.

> > > Furthermore, having the MPI compiler commands available is not desired, when
> > > one wants to build MPI-aware programs in serial mode (which is also required by
> > > the guideline).
> > You can simply unload the mpich2 module then, and continue with building in
> > your serial model.
> 
> Once again, this is incompatible with the existence of multiple MPI
> implementations. We hope packages to be built for every one of them that are
> available in Fedora, not just the one the primary package maintainer is used
> to.

I'm not forcing mpich2 on anyone, just trying to make it easier to use.

Comment 7 Erik Zeek 2010-11-08 18:02:32 UTC
Please reread and rewrite opening paragraphs of
http://fedoraproject.org/wiki/Packaging:MPI

to indicate that Mpich2 is now the default compiler for Fedora and that the choices of MPI compiler should not be left to the user.

Comment 8 Susi Lehtola 2011-11-15 22:01:56 UTC
Still in breakage.

http://fedoraproject.org/wiki/Packaging/MPI#Packaging_of_MPI_compilers
"MUST: If the maintainer wishes for the environment module to load automatically by use of a scriptlet in /etc/profile.d or by some other mechanism, this MUST be done in a subpackage."

Comment 9 Fedora Update System 2012-01-02 15:11:20 UTC
mpich2-1.4.1p1-4.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/mpich2-1.4.1p1-4.fc16

Comment 10 Fedora Update System 2012-01-02 15:11:30 UTC
mpich2-1.4.1p1-4.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/mpich2-1.4.1p1-4.fc15

Comment 11 Fedora Update System 2012-01-02 21:54:29 UTC
Package mpich2-1.4.1p1-4.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing mpich2-1.4.1p1-4.fc15'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-0033/mpich2-1.4.1p1-4.fc15
then log in and leave karma (feedback).

Comment 12 Fedora Update System 2012-01-11 06:04:03 UTC
mpich2-1.4.1p1-4.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 13 Fedora Update System 2012-01-11 06:05:08 UTC
mpich2-1.4.1p1-4.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.