Description of problem: The links involved in the alternatives mechanism for openmpi are all broken. Version-Release number of selected component (if applicable): 1.3.3-6.fc11.x86_64 How reproducible: The files /usr/bin/mpi* no longer exist. The links in /etc/alternatives/mpi* are all broken. Steps to Reproduce: 1. Try to compile or run anything with mpi. 2. Find that none of the exectutables are in your path. 3. Submit bug report. Actual results: Expected results: Additional info: Here is a shell session from my machine: |abacus4>mpicc bash: mpicc: command not found |abacus4>ls -l /etc/alternatives/mpi* lrwxrwxrwx. 1 root root 34 2009-05-28 15:27 /etc/alternatives/mpic++ -> /usr/bin/opal_wrapper-1.3.1-gcc-64 lrwxrwxrwx. 1 root root 34 2009-05-28 15:27 /etc/alternatives/mpicc -> /usr/bin/opal_wrapper-1.3.1-gcc-64 lrwxrwxrwx. 1 root root 34 2009-05-28 15:27 /etc/alternatives/mpiCC -> /usr/bin/opal_wrapper-1.3.1-gcc-64 lrwxrwxrwx. 1 root root 34 2009-05-28 15:27 /etc/alternatives/mpicxx -> /usr/bin/opal_wrapper-1.3.1-gcc-64 lrwxrwxrwx. 1 root root 16 2009-05-28 15:41 /etc/alternatives/mpi-exec -> /usr/bin/orterun lrwxrwxrwx. 1 root root 50 2009-05-28 15:41 /etc/alternatives/mpi-exec-man -> /usr/share/openmpi/1.3.1-gcc/man/man1/orterun.1.gz lrwxrwxrwx. 1 root root 34 2009-05-28 15:27 /etc/alternatives/mpif77 -> /usr/bin/opal_wrapper-1.3.1-gcc-64 lrwxrwxrwx. 1 root root 34 2009-05-28 15:27 /etc/alternatives/mpif90 -> /usr/bin/opal_wrapper-1.3.1-gcc-64 lrwxrwxrwx. 1 root root 16 2009-05-28 15:41 /etc/alternatives/mpi-run -> /usr/bin/orterun lrwxrwxrwx. 1 root root 49 2009-05-28 15:41 /etc/alternatives/mpi-run-man -> /usr/share/openmpi/1.3.1-gcc/man/man1/mpirun.1.gz |abacus4>rpm -q openmpi openmpi-1.3.3-6.fc11.x86_64 |abacus4>rpm -q openmpi-devel openmpi-devel-1.3.3-6.fc11.x86_64 |abacus4>ls -l /usr/bin/mpicc ls: cannot access /usr/bin/mpicc: No such file or directory |abacus4>ls -l /etc/alternatives/mpicc lrwxrwxrwx. 1 root root 34 2009-05-28 15:27 /etc/alternatives/mpicc -> /usr/bin/opal_wrapper-1.3.1-gcc-64 |abacus4>sudo yum provides /usr/bin/opal_wrapper-1.3.1-gcc-64 Loaded plugins: refresh-packagekit openmpi-devel-1.3.1-1.fc11.x86_64 : Development files for openmpi Repo : fedora Matched from: Filename : /usr/bin/opal_wrapper-1.3.1-gcc-64 |abacus4>rpm -qf /usr/bin/opal_wrapper-1.3.1-gcc-64 error: file /usr/bin/opal_wrapper-1.3.1-gcc-64: No such file or directory |abacus4>rpm --verify openmpi |abacus4>rpm --verify openmpi-devel |abacus4>echo $? 0 |abacus4>
If there is a bug present, it is merely that the new openmpi package did not remove the alternatives links from the old package properly. However, the links are not intended to be present any longer. The packaging committee and FESCo ratified the new MPI packaging guidelines. Under those new guidelines, mpi users should use the environment-modules support build into the new openmpi packages to select openmpi or whatever mpi implementation they choose to use. The alternatives version of mpi selection is dead and will not be revived.
(In reply to comment #1) > Under those new guidelines, > mpi users should use the environment-modules support build into the new openmpi > packages to select openmpi or whatever mpi implementation they choose to use. I can understand that there might be a good reason to make such a change in Fedora 12. I think it is insane, however, for a daily update to Fedora 11 to suddenly break the work of anyone actually *using* openmpi-devel. I don't even know what this new "environment-modules" support is or where to find out about it. Are there some sort of daily release notes for the update repository? How is an end-user supposed to cope with this sort of "update?"
The fedora 11 openmpi package had breakage issues from day 1 (it was a last minute update slipped in by someone other than the package maintainer and it has taken this long to get everything sorted out and fixed). If it weren't, then I would agree with you about not wanting to break f11 with this update. As for environment-modules usage, man module will get you the full details, but suffice it to say that the following line in your .bash_profile (or equivalent) will get you what you want (assuming you want x86_64 arch): module load openmpi-x86_64
I've also been hit by this bug. After upgrading to Fedora 12 this week, I tried to build an mpi application, only to discover mpicc was not longer on the system, and all the /etc/alternatives links were broken. I tried various combinations of installing and removing both openmpi and mpich2 packages and got nowhere. Personally, I preferred the alternatives mechanism (it's easer to set up a default on the normal paths than to get all the users to put the right magic in their profiles) but there really should have been some sort of notification for this change. It was only by deciding I wasn't an idiot and the package was completely broken that I found this bug and discovered the extra step that's now necessary to have this package work. Disappointing, since it was openmpi working out of the box in f11 that convinced me to switch from ubuntu. :(
I was happy when i found MPI implementation in my fedora's repos, cuz i admin several server in my university, but how could you thing that all endusers will understad how to load/unload modules in days... We are at end of semester. The modules dont work with elicpse, cuz eclipse call directy the mpicc, mpirun, etc.. I cant change all userspace of all my users. Fedora developers must think that this kind of changes must made in big versions changes. I need to recompile from sourcecode openmpi and install it in several servers. And do it again in futures versions of fedora? Moule Packages are used in big servers with several implematation of MPI or compilers... So they dont use repos to updates they packages for the estability problems. As always.. The populace is the most afected. :(
While I appreciate that people think the alternatives system is easier, it's only easier for rudimentary usage of MPI stacks. For real usage, meaning you have a real cluster with lots of users and lots of applications, it is totally worthless. Ralph: Sometimes working out of the box isn't the right choice. Additionally, I can say that if we hadn't made this change, you might be equally upset about the fact that the mpich maintainer wanted mpich to be the default over openmpi and had set the alternatives links in mpich to override openmpi's, so I would argue that it was better that your setup quit working and alert you to your need to do something than to silently change mpis behind your back (well, silently change behind your back assuming mpich was installed at all). In addition, it is *not* that hard to set this up for your end users. Two files in /etc/profile.d, one for bash and one for csh, which does nothing more than module load openmpi for your users will set openmpi as the system wide default. Then, because of how the system works, if a user were to select mpich in their own personal login scripts it would override the system wide default (as long as they set their preference after the system scripts are parsed). Splen: See the above for how to make eclipse and other programs work with openmpi without the users needing to do anything.
Doug: Thanks for the additional information. I see your point about larger installations with heterogeneous application groups. Speaking for myself personally, I think I would have been able to use the alternatives system to switch things back. If fact, when mpi stopped working, I assumed the installation of mpich2, forced by an explicit dependence in the scotch package, was what had broken it. But of course running update-alternatives, or just uninstalling mpich2 wasn't enough to restore working support. But that's based on prior experience with the alternatives mechanism; a new user would have been equally lost on both counts. Should I file a bug against mpich2 for not using the module system then? Right now my choice is between installing mpich2 and having it available to everyone by default, or putting 'module load openmpi-`uname -p`' in the login scripts. I think Fedora as a project should be consistent about how to choose between parallel-installed packages with with same functionality. And I think that if only a single package is installed, its services should be available by default. This is a feature the alternatives mechanism provided, but the openmpi rpm's use of the module infrastructure does not. Instead, it seems to me the mpich2 maintainer got their wish: $ ls -lah /usr/bin/mpiexec lrwxrwxrwx. 1 root root 26 2009-11-21 19:40 /usr/bin/mpiexec -> /etc/alternatives/mpi-exec $ ls -lah /etc/alternatives/mpi-exec lrwxrwxrwx. 1 root root 19 2009-11-21 19:40 /etc/alternatives/mpi-exec -> /usr/bin/mpiexec.py $ rpm -qf /usr/bin/mpiexec.py mpich2-1.1.1p1-1.fc12.x86_64 $ sudo alternatives --config mpi-run (select /usr/bin/orterun) $ ls -lah /usr/bin/mpiexec lrwxrwxrwx. 1 root root 26 2009-11-24 11:17 /usr/bin/mpiexec -> /etc/alternatives/mpi-exec $ ls -lah /etc/alternatives/mpi-exec lrwxrwxrwx. 1 root root 16 2009-11-24 11:17 /etc/alternatives/mpi-exec -> /usr/bin/orterun $ ls -lah /usr/bin/orterun ls: cannot access /usr/bin/orterun: No such file or directory
> I think Fedora as a project should be consistent about how to choose between > parallel-installed packages with with same functionality. Well, Fedora as a project is. It was recently decided by the packaging committee that the environment-modules was the "one true way" to do MPI installations. However, that was a very recent decree and openmpi was modified to drop the usage of alternatives quicker than mpich was. Actually, that's not entirely accurate, alternatives support is still allowed, but it must be in a separate package so you don't have to have it installed. See the guidelines for the complete story: https://fedoraproject.org/wiki/PackagingDrafts/MPI > Should I file a bug against mpich2 for not using the module system then? Yes. The guidelines call out for the mpi usage of the module system to be installed in all versions of Fedora from 11 on. I believe a version of mpich2 that has module support is likely already in the testing repo for F11. The changes to enable it were checked into CVS about a week ago. However, looking at the sources in CVS, the alternatives links and what not that are supposed to be split off into their own package so they can be removed from the system while leaving the base package present are in fact still part of the base package. So even when you update to the latest package you will still have the alternatives versus module system dilemma. I would file a bug about that. > And I think that if only a single package is installed, its services should > be available by default. This is a feature the alternatives mechanism > provided, but the openmpi rpm's use of the module infrastructure does not. I'm not sure that this is really a wise thing to do given part of the MPI packaging guidelines that were ratified. In particular, packages that have the option of being built either against an MPI library or not (such as blacs) must be built without MPI support in their base package, and built with mpi support in a named mpi subpackage (aka, blacs for normal use, blacs-openmpi for openmpi, blacs-mpich2 for mpich2, etc). The mpi version of any binaries produced are to go into MPI_BIN and if there are shared libraries then they are to go into MPI_LIB. The net result of this is if you have an app that links against a shared library that links either with or without an MPI stack, then whether or not you have loaded the mpi module will change which shared library you get and that includes not only the mpi libs, but the mpi linking libs such as blacs. An example consumer would be scalapack. It links against blacs, and if you have the normal blacs libs in the normal %{_libdir} path, and the mpi blacs libs in MPI_LIB, then loading the mpi module makes a material difference in how scalapack is dynamically linked and what capabilities it has. I don't think that's something we should be assuming users want turned on just because there's only one MPI implementation installed. I would much prefer that they enable that behavior themselves even it it's only once by editing a file somewhere and from then on it is enabled by default. > Right now my choice is between installing mpich2 and having it available to > everyone by default, or putting 'module load openmpi-`uname -p`' in the login > scripts. Unfortunately, that does sound about right at the moment. It might be worth considering creating a new package, default-mpi, that when installed would have a couple init scripts in /etc/profile.d and a configuration file in /etc/sysconfig that would allow a default system wide mpi to be selected, but we don't have that at the moment.
Doug Ledford: Thanks, it work. I was looking for documentation of modules with no luck. There are so many pages with the word "module". And very poor documentation in the main page of modules package. However... The packaging committee and FESCo are insane !! This change could be a good desicion for F12, but not for daily update. When a user take the disicion of update OS, is because had enougth time to see all the changes in packes, alternatives, versions, functions, etc, etc, etc...
(In reply to comment #4) > Personally, I preferred the alternatives mechanism (it's easer to set up a > default on the normal paths than to get all the users to put the right magic in > their profiles) but there really should have been some sort of notification for > this change. It was only by deciding I wasn't an idiot and the package was > completely broken that I found this bug and discovered the extra step that's > now necessary to have this package work. Optionally, you can just put module load openmpi-x86_64 in the system-wide profile, which is a lot easier. Environment modules are a lot handier than alternatives, since you can unload a module you don't like on a session basis..
Whoops, that was already mentioned. The packaging committee or FESCo did not decide anything about when to force the guidelines. I was behind the MPI and environment module guidelines, and would have been happy to get the support working in rawhide before introducing it. Now the user experience has changed. However, as Doug mentioned, the old way of doing things was not perfect either. You couldn't have many MPI compilers and runtimes installed at the same time. From a packager's point of view, converting to the new guidelines straight away is a lot easier, since that way there's a lot less work to do maintaining a package in multiple versions of Fedora. RHEL solved the MPI problem in a different way a few years back, which is again a bit annoying, but we'll just have to write a wrapper package in EPEL to deal with it.
Jay: fix this bug by adding a note in the descriptions: By default the Open MPI module is not loaded. To compile or run programs compiled against Open MPI, run $ module load openmpi-%{_arch}
This message is a reminder that Fedora 11 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 11. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '11'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 11's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 11 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Fedora 11 changed to end-of-life (EOL) status on 2010-06-25. Fedora 11 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. Thank you for reporting this bug and we are sorry it could not be fixed.