Description of problem: New guidelines for packaging MPI compilers have been approved recently by FPC and FESCo, please make the package compliant with them: https://fedoraproject.org/wiki/PackagingDrafts/MPI Version-Release number of selected component (if applicable): mpich2-1.1.1p1-1.fc11.x86_64
ping? While you are at it, notice that the placement of the Fortran modules is also incorrect, they MUST be placed in %{_fmoddir}. https://fedoraproject.org/wiki/Packaging:Fortran But since this is an MPI package the MPI guideline override the file placement to %{_fmoddir}/mpich2 as noted in https://fedoraproject.org/wiki/PackagingDrafts/MPI#Packaging_of_MPI_compilers
Ping again.
Fixed in rawhide. I'm not moving the modules location around until the issue with %{_fmoddir} is resolved (Bug 483765). The present location for is just fine for mpich2 and any app that might be using (or may want to use) mpich2.
OK, thanks. Please use make V=1 to make the compile commands visible in the log. Lines such as CC mpid_ext32_datatype.c CC mpid_type_dup.c CC mpir_type_get_contig_blocks.c are not very informative. By the way, you're missing Provides: mpich2-static in -devel.
Can you please let know what other information you want to see in the logs? The devs prefer it as it is now, makes the warning more visible. The main information in the compile commands that is of interest for Fedora packaging is the rpm %optflags, which is well taken care of already.
The build commands, which are directly related to packaging, since I think the spec file is not doing what you intend it to do: %configure \ (clip) CFLAGS="-m%{__isa_bits} -O2 %{?XFLAGS}" \ CXXFLAGS="-m%{__isa_bits} -O2 %{?XFLAGS}" \ F90FLAGS="-m%{__isa_bits} -O2 %{?XFLAGS}" \ FFLAGS="-m%{__isa_bits} -O2 %{?XFLAGS}" \ LDFLAGS='-Wl,-z,noexecstack' \ I can't see %{optflags} anywhere. Besides, shouldn't the environment variables above be defined *before* %configure so that ./configure sees them? The warnings are not as important as the compiler commands for the Fedora packages.
You mis-quoted the spec file, it is not what you posted but; %configure \ (clip) CFLAGS="-m%{__isa_bits} -O2 %{?XFLAGS}" \ CXXFLAGS="-m%{__isa_bits} -O2 %{?XFLAGS}" \ F90FLAGS="-m%{__isa_bits} -O2 %{?XFLAGS}" \ FFLAGS="-m%{__isa_bits} -O2 %{?XFLAGS}" \ LDFLAGS='-Wl,-z,noexecstack' \ MPICH2LIB_CFLAGS="%{?opt_cc_cflags}" \ MPICH2LIB_CXXFLAGS="%{optflags}" \ MPICH2LIB_F90FLAGS="%{?opt_fc_fflags}" \ MPICH2LIB_FFLAGS="%{?opt_f77_fflags}" You have to trust that I know what I'm doing, or do a test build by yourself by using 'make VERBOSE=1', or read up the instructions on building mpich2 (http://trac.mcs.anl.gov/projects/mpich2/browser/mpich2/trunk/README.vin).
- Don't place any files in %{_bindir}, everything MUST go in $MPI_BIN which is loaded into $PATH by the environment module. - Same thing with MAN pages. - Drop alternatives support, or move it to a separate subpackage. "MUST: By default, NO files are placed in /etc/ld.so.conf.d. If the packager wishes to provide alternatives support, it MUST be placed in a subpackage along with the ld.so.conf.d file so that alternatives support does not need to be installed if not wished for." - You are missing the directories (such as MPI_MAN, MPI_PYTHON_SITEARCH and so on) that need to be owned by mpich2, see the list at http://www.fedoraproject.org/wiki/PackagingDrafts/MPI#Packaging_of_MPI_compilers - I get gcc -pthread -shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic build/temp.linux-x86_64-2.6/mpiext.o -L/usr/lib64/mpich2/lib -L/usr/lib64 -lmpich -lopa -lpthread -lrt -lopa' -lpython2.6 -o build/lib.linux-x86_64-2.6/pypar/mpiext.so /usr/bin/ld: cannot find -lopa' when trying to compile pypar with mpich2.
(In reply to comment #8) > - Don't place any files in %{_bindir}, everything MUST go in $MPI_BIN which is > loaded into $PATH by the environment module. > These are MPICH2 specific tools, there is no need to hide them away. > - Same thing with MAN pages. > What's the problem here; mpich2 configures with '--mandir=%{_mandir}/%{name} ' \ > - Drop alternatives support, or move it to a separate subpackage. > "MUST: By default, NO files are placed in /etc/ld.so.conf.d. If the packager > wishes to provide alternatives support, it MUST be placed in a subpackage along > with the ld.so.conf.d file so that alternatives support does not need to be > installed if not wished for." > Current mpich2 in rawhide doesn't install anything under /etc/ld.so.conf.d . I don't see any problem in providing alternatives support to single-user systems without loading modules. > - You are missing the directories (such as MPI_MAN, MPI_PYTHON_SITEARCH and so > on) that need to be owned by mpich2, see the list at > http://www.fedoraproject.org/wiki/PackagingDrafts/MPI#Packaging_of_MPI_compilers > Those are env. variables; check out 'module load mpich2-x86_64 ; printenv |grep MPI' > - I get > gcc -pthread -shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic > build/temp.linux-x86_64-2.6/mpiext.o -L/usr/lib64/mpich2/lib -L/usr/lib64 > -lmpich -lopa -lpthread -lrt -lopa' -lpython2.6 -o > build/lib.linux-x86_64-2.6/pypar/mpiext.so > /usr/bin/ld: cannot find -lopa' > > when trying to compile pypar with mpich2. > Use the mpich2 cc compiler, mpicc.
(In reply to comment #9) > (In reply to comment #8) > > - Don't place any files in %{_bindir}, everything MUST go in $MPI_BIN which is > > loaded into $PATH by the environment module. > > > These are MPICH2 specific tools, there is no need to hide them away. Because MPI compilers and runtimes have to be separated from each other, which was the whole reason for writing the guidelines. > > - Same thing with MAN pages. > > > What's the problem here; mpich2 configures with '--mandir=%{_mandir}/%{name} ' > \ You have /usr/share/man/man1/mpic++.1.gz /usr/share/man/man1/mpicc.1.gz /usr/share/man/man1/mpicxx.1.gz /usr/share/man/man1/mpiexec.1.gz /usr/share/man/man1/mpif77.1.gz /usr/share/man/man1/mpif90.1.gz in mpich2. These shouldn't be there. > > > - Drop alternatives support, or move it to a separate subpackage. > > "MUST: By default, NO files are placed in /etc/ld.so.conf.d. If the packager > > wishes to provide alternatives support, it MUST be placed in a subpackage along > > with the ld.so.conf.d file so that alternatives support does not need to be > > installed if not wished for." > > > Current mpich2 in rawhide doesn't install anything under /etc/ld.so.conf.d . I > don't see any problem in providing alternatives support to single-user systems > without loading modules. Yes. Still, the alternatives support MUST be placed in a separate package. It is not a wanted feature for the MPI commands to be available by default. If you want to make a package that makes mpich2 useable without loading a module, create a package that loads the mpich2 module automatically upon login by placing files in /etc/profile.d/ > > - You are missing the directories (such as MPI_MAN, MPI_PYTHON_SITEARCH and so > > on) that need to be owned by mpich2, see the list at > > http://www.fedoraproject.org/wiki/PackagingDrafts/MPI#Packaging_of_MPI_compilers > > > Those are env. variables; check out 'module load mpich2-x86_64 ; printenv |grep > MPI' Yes. They are environment variables that point to the directories where the files compiled against mpich2 are placed. mpich2 MUST own these directories. "The module file MUST also set some helper variables (primarily for use in spec files): ... As these directories may be used by software using the MPI stack, the MPI runtime package MUST own all of them."
mpicc doesn't work either: $ mpicc -c mpiext.c mpiext.c:30:20: error: Python.h: No such file or directory mpiext.c:33:31: error: numpy/arrayobject.h: No such file or directory mpiext.c:64: error: expected ‘)’ before ‘*’ token mpiext.c:80: error: expected ‘)’ before ‘*’ token mpiext.c: In function ‘op_map’: mpiext.c:167: error: ‘PyExc_ValueError’ undeclared (first use in this function) mpiext.c:167: error: (Each undeclared identifier is reported only once mpiext.c:167: error: for each function it appears in.) mpiext.c:168: error: ‘NULL’ undeclared (first use in this function) mpiext.c: At top level: $ mpicc --showme /usr/lib/gcc/x86_64-redhat-linux/4.4.2/../../../../lib64/crt1.o: In function `_start': (.text+0x20): undefined reference to `main' collect2: ld returned 1 exit status
(In reply to comment #10) > (In reply to comment #9) > > You have > > /usr/share/man/man1/mpic++.1.gz > /usr/share/man/man1/mpicc.1.gz > /usr/share/man/man1/mpicxx.1.gz > /usr/share/man/man1/mpiexec.1.gz > /usr/share/man/man1/mpif77.1.gz > /usr/share/man/man1/mpif90.1.gz > Yes, they are ghost files which are initialized by the alternatives system. > > Yes. Still, the alternatives support MUST be placed in a separate package. It > is not a wanted feature for the MPI commands to be available by default. If you Why place it in a separate package when I would still have to make the main package require it. The use of alternative system is permitted by Fedora packaging principles (e.g. for java, mta, etc), and it does no harm to your system. If YOU don't want it, then ignore it and continue to load your modules. > want to make a package that makes mpich2 useable without loading a module, > create a package that loads the mpich2 module automatically upon login by > placing files in /etc/profile.d/ > How is place a file in /etc/profile.d/ necessarily better than using alternatives for mpich2. Well I may decide to do that in the future, but it's not going to a separate package, that's meaningless. I will have to place it in the main package then. > > Yes. They are environment variables that point to the directories where the > files compiled against mpich2 are placed. mpich2 MUST own these directories. > Again, you're still missing the picture; .. [deji@rhema ~]$ module load mpich2-x86_64 ; printenv |grep MPI_MAN MPI_MAN=/usr/share/man/mpich2 [deji@rhema ~]$ rpm -q --whatprovides /usr/share/man/mpich2 mpich2-1.2-1.fc12.x86_64 .. > "The module file MUST also set some helper variables (primarily for use in spec > files): ... As these directories may be used by software using the MPI stack, > the MPI runtime package MUST own all of them."
(In reply to comment #11) > mpicc doesn't work either: > I'm sorry to say this, but it seems you don't know what you're doing or you're confused. > $ mpicc -c mpiext.c > mpiext.c:30:20: error: Python.h: No such file or directory > mpiext.c:33:31: error: numpy/arrayobject.h: No such file or directory These are by no means mpicc problem but yours. You need to know how to compile your program properly; $ mpicc -c -I/usr/lib64/python2.6/site-packages/numpy/core/include -I/usr/include/python2.6 mpiext.c > > $ mpicc --showme > /usr/lib/gcc/x86_64-redhat-linux/4.4.2/../../../../lib64/crt1.o: In function > `_start': mpich2 doesn't work this way, run 'mpicc -show' instead.
(In reply to comment #12) > > Yes. Still, the alternatives support MUST be placed in a separate package. It > > is not a wanted feature for the MPI commands to be available by default. If > you > > Why place it in a separate package when I would still have to make the main > package require it. The use of alternative system is permitted by Fedora > packaging principles (e.g. for java, mta, etc), and it does no harm to your > system. If YOU don't want it, then ignore it and continue to load your modules. *You do not need* alternatives support: the environment modules system provides everything that is needed for operation. Please understand that there may be many users on a system, each of which need to use a different MPI runtime. This is why everything is done on user-level, NOT system-level. If you want to provide alternatives support for old times' sake, do it so in a subpackage. Then you need also the config file in /etc/ld.so.conf.d to make MPI programs work. > > want to make a package that makes mpich2 useable without loading a module, > > create a package that loads the mpich2 module automatically upon login by > > placing files in /etc/profile.d/ > > > How is place a file in /etc/profile.d/ necessarily better than using > alternatives for mpich2. Well I may decide to do that in the future, but it's > not going to a separate package, that's meaningless. I will have to place it in > the main package then. Because environment modules are session based and you can unload the module, but you can't "unload" alternatives. The alternatives commands will be available to you even when you don't want them to be present. > > Yes. They are environment variables that point to the directories where the > > files compiled against mpich2 are placed. mpich2 MUST own these directories. > > > Again, you're still missing the picture; > .. > [deji@rhema ~]$ module load mpich2-x86_64 ; printenv |grep MPI_MAN > MPI_MAN=/usr/share/man/mpich2 > [deji@rhema ~]$ rpm -q --whatprovides /usr/share/man/mpich2 > mpich2-1.2-1.fc12.x86_64 > .. You're missing the rest of the man directories, i.e. /usr/share/man/mpich2-%{_arch}man{0p,1p,1x,2,2x,3,3p,3x,4,4x,5,5x,6,6x,7,7x,8,8x,9,l,n} These need to be created and owned by mpich2 also. Note that the directory MUST be suffixed with the architecture to avoid clashes in multilib systems. - You have declared MPI_PYTHON_SITEARCH=%{_libdir}/mpich2/bin This is incorrect, it should be %{python_sitearch}/mpich2/. - MPI_SUFFIX should be declared as '_mpich2', not 'mpich2'. - MPI_FORTRAN_MOD_DIR should be declared as %{_fmoddir}/mpich2/ not /usr/include/mpich2-%{_arch} - The module file MUST prepend LD_LIBRARY_PATH with %{_libdir}/mpich2/lib, and PYTHONPATH with %{python_sitearch}/mpich2/. "The module file MUST prepend the MPI bindir %{_bindir}/%{name}-%{_arch}%{?_opt_cc_suffix} into the users PATH, set LD_LIBRARY_PATH to %{_libdir}/%{name}%{?_opt_cc_suffix}/ and PYTHONPATH to %{python_sitearch}/%{name}%{?_cc_name_suffix}/. " Please don't close the bug until all of the issues have been *really* fixed.
(In reply to comment #13) > > $ mpicc --showme > > /usr/lib/gcc/x86_64-redhat-linux/4.4.2/../../../../lib64/crt1.o: In function > > `_start': > > mpich2 doesn't work this way, run 'mpicc -show' instead. Still, I don't think it should do that, this is clearly a bug.
On x86_64 mpich2 uses: $ mpicc -compile_info -link_info gcc -m64 -O2 -fPIC -Wl,-z,noexecstack -I/usr/include/mpich2-x86_64 -L/usr/lib64/mpich2/lib -L/usr/lib64/mpich2/lib -Wl,-rpath,/usr/lib64/mpich2/lib -lmpich -lopa -lpthread -lrt so you need to drop the rpath part because it is forbidden in Fedora. http://fedoraproject.org/wiki/Packaging/Guidelines#Beware_of_Rpath
Then the module file is also in the wrong place, it MUST be placed in /etc/modulefiles, not in /usr/share/Modules/modulefiles/. See http://fedoraproject.org/wiki/PackagingDrafts/EnvironmentModules
The module file is currently: # Define prefix so PATH and MANPATH can be updated. setenv prefix /usr setenv exec_prefix /usr setenv MPI_BIN /usr/lib64/mpich/bin setenv MPI_SYSCONFIG /etc/mpich2-x86_64 setenv MPI_FORTRAN_MOD_DIR /usr/include/mpich2-x86_64 setenv MPI_INCLUDE /usr/include/mpich2-x86_64 setenv MPI_LIB /usr/lib64/mpich2/lib setenv MPI_MAN /usr/share/man/mpich2 setenv MPI_PYTHON_SITEARCH /usr/lib64/mpich2/bin setenv MPI_COMPILER mpich-x86_64 setenv MPI_SUFFIX mpich setenv MPI_HOME /usr/lib64/mpich prepend-path PATH /usr/lib64/mpich2/bin prepend-path MANPATH /usr/share/man/mpich2 # Undefine prefix and exec_prefix which are too generic environment variables. unsetenv prefix unsetenv exec_prefix Why do you define and undefine prefix and exec_prefix if you don't even use them anywhere?
This bug appears to have been reported against 'rawhide' during the Fedora 12 development cycle. Changing version to '12'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Also, please fix the variables in the module file: use mpich2 instead of mpich in MPI_BIN, MPI_COMPILER, MPI_SUFFIX (remember to add the missing underscore) and MPI_HOME.
(In reply to comment #20) > Also, please fix the variables in the module file: use mpich2 instead of mpich > in MPI_BIN, MPI_COMPILER, MPI_SUFFIX (remember to add the missing underscore) > and MPI_HOME. Already fixed days ago. AFAIC I've fixed this bug, please don't re-open. If you have specific issues with mpich2 packaging, then do open fresh bug reports for them.
Very well, it is cleaner that way. I've opened up bugs 542745, 542749 and 542750, which should be no-brainers to fix.