Bug 1238428 - RPM MPI Requires Provides
Summary: RPM MPI Requires Provides
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: Changes Tracking
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jan Kurik
QA Contact:
URL:
Whiteboard: ChangeAcceptedF23 SelfContainedChange
Depends On: 1241282 1241737
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-07-01 20:25 UTC by Jan Kurik
Modified: 2015-08-27 17:58 UTC (History)
3 users (show)

Fixed In Version: 2.3-10.fc23
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-08-27 17:58:52 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Jan Kurik 2015-07-01 20:25:18 UTC
This is a tracking bug for Change: RPM MPI Requires Provides
For more details, see: https://fedoraproject.org//wiki/Changes/RpmMPIReqProv

Add dependency generators to encode the MPI compiler name in the provides string of a binary to distinguish otherwise identical provides between packages $foo, $foo-openmpi and $foo-mpich.

Comment 1 Jan Kurik 2015-07-14 14:02:41 UTC
This message is a reminder that Fedora 23 Change Checkpoint: Completion deadline (testable) is on 2015-07-28 [1].

At this point, all accepted Changes should be substantially complete, and testable. Additionally, if a change is to be enabled by default, it must be so enabled at Change Completion deadline.

This bug should be set at least to the MODIFIED state to indicate that it achieved completeness. Status will be provided to FESCo right after the deadline. If, for any reasons, your Change is not in required state, let me know and we will try to find solution. For Changes you decide to cancel/move to the next release, please use the NEW status and set needinfo on me and it will be acted upon. 

In case of any questions, don't hesitate to ask Wrangler (jkurik). Thank you.

[1] https://fedoraproject.org/wiki/Releases/23/Schedule

Comment 2 Jan Kurik 2015-07-15 13:51:47 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23

Comment 3 Sandro Mani 2015-07-29 16:26:19 UTC
Remark on current status: test-builds done on COPR [1], requested proven packager help to rebuild affected packages in rawhide and F23 [2].

[1] https://copr.fedoraproject.org/coprs/smani/rpm-mpi-hooks-testing2/
[2] https://lists.fedoraproject.org/pipermail/devel/2015-July/212794.html

Comment 4 Jan Kurik 2015-08-06 08:58:30 UTC
Removing from the scope of F23 as we have already passed the deadline when this change needs to be in MODIFIED state.

Comment 5 Mark Wielaard 2015-08-14 20:03:45 UTC
I could use some help with valgrind-openmpi
Currently in rawhide I get:

valgrind has broken dependencies in the rawhide tree:
On x86_64:
        1:valgrind-openmpi-3.10.1-18.fc24.x86_64 requires libmpi.so.1()(64bit)
On i386:
        1:valgrind-openmpi-3.10.1-18.fc24.i686 requires libmpi.so.1
On armhfp:
        1:valgrind-openmpi-3.10.1-18.fc24.armv7hl requires libmpi.so.1

valgrind-openmpi provides the libmpiwrap library to use memcheck with your mpi programs. See:
http://valgrind.org/docs/manual/mc-manual.html#mc-manual.mpiwrap

libmpiwrap is build using openmpi mpicc and gets linked against openmpi libmpi.so.1 (which is where the requires comes from).

What/How should I change to get the shared library to link and require the correct new library provides?

Comment 6 Sandro Mani 2015-08-14 20:16:35 UTC
Do I understand correctly that, while it is built using openmpi, it can be used for any MPI implementation? In that case I think the cleanest solution would just be to filter the requires from the library, since it is safe to assume that if a user wants to debug an mpi application, some mpi implementation will be installed anyways.

Comment 7 Mark Wielaard 2015-08-14 20:37:30 UTC
(In reply to Sandro Mani from comment #6)
> Do I understand correctly that, while it is built using openmpi, it can be
> used for any MPI implementation?

No, I don't think that would work. The libmpiwrap build for valgrind-openmpi is intended to be used with program build with the same openmpi mpicc.

> In that case I think the cleanest solution
> would just be to filter the requires from the library, since it is safe to
> assume that if a user wants to debug an mpi application, some mpi
> implementation will be installed anyways.

Without openmpi installed valgrind-openmpi is indeed useless. So yes, the user most likely has openmpi already installed. I don't mind filtering out the requires from the libraries (although I don't know how that is done). But I am a little surprised that building a library using the openmpi mpicc doesn't generate the proper provides on the openmpi libmpi.so.

Comment 8 Sandro Mani 2015-08-14 21:00:19 UTC
So, the script generating the requires / provides is looking for binaries within known MPI directories, which are:

- Paths below %{_libdir}/$MPI
- %{python_sitearch}/$MPI
- %{_libdir}/gfortran/modules/$MPI

where $MPI could be openmpi or mpich.

The valgrind wrappers are not installed there:

/usr/lib64/valgrind/libmpiwrap-amd64-linux.so
/usr/lib64/valgrind/libmpiwrap-x86-linux.so

so this is why the MPI dependency generator triggers ignore these.


The filtering is described here [1]. I suppose something like

%global __requires_exclude_from ^%{_libdir}/valgrind/libmpiwrap.*\.so$

should work.

[1] https://fedoraproject.org/wiki/Packaging:AutoProvidesAndRequiresFiltering

Comment 9 Mark Wielaard 2015-08-14 22:51:20 UTC
(In reply to Sandro Mani from comment #8)
> So, the script generating the requires / provides is looking for binaries
> within known MPI directories, which are:
> 
> - Paths below %{_libdir}/$MPI
> - %{python_sitearch}/$MPI
> - %{_libdir}/gfortran/modules/$MPI
> 
> where $MPI could be openmpi or mpich.
> 
> The valgrind wrappers are not installed there:
> 
> /usr/lib64/valgrind/libmpiwrap-amd64-linux.so
> /usr/lib64/valgrind/libmpiwrap-x86-linux.so
> 
> so this is why the MPI dependency generator triggers ignore these.

Thanks. So I just changed things so that the libmpiwrap library gets installed under %{_libdir}/openmpi/valgrind with a symlink from the old location for backwards compatibility. And that seems to have triggered the new libmpi requires.
See valgrind-3.10.1-19.fc24

Comment 10 Sandro Mani 2015-08-14 23:30:02 UTC
Ok cool, nice solution.

Comment 11 Fedora Update System 2015-08-19 15:26:56 UTC
rpm-mpi-hooks-3-1.fc23,mathgl-2.3-10.fc23,elpa-2015.05.001-2.fc23,freefem++-3.31-7.3.fc23,netcdf-4.3.3.1-5.fc23,MUMPS-5.0.1-2.fc23,gpaw-0.11.0.13004-15.fc23,ga-5.3b-18.fc23,valgrind-3.10.1-19.fc23,boost-1.58.0-6.fc23,dl_poly-1.9.20140324-14.fc23,hpl-2.1-13.fc23,netgen-mesher-5.3.1-8.fc23,espresso-3.3.0-7.fc23,scotch-6.0.4-5.fc23,hdf5-1.8.15-6.patch1.fc23,scalapack-2.0.2-10.fc23,gromacs-5.0.6-6.fc23,scorep-1.4.2-2.fc23,sundials-2.6.2-4.fc23,Ray-2.3.1-11.fc23,orsa-0.7.0-33.fc23,elk-3.1.12-14.fc23,mpich-3.1.4-5.fc23,openmpi-1.8.8-3.fc23,mpi4py-1.3.1-14.fc23,pypar-2.1.5_108-11.fc23,paraview-4.3.1-11.fc23 has been submitted as an update for Fedora 23.
https://admin.fedoraproject.org/updates/rpm-mpi-hooks-3-1.fc23,mathgl-2.3-10.fc23,elpa-2015.05.001-2.fc23,freefem++-3.31-7.3.fc23,netcdf-4.3.3.1-5.fc23,MUMPS-5.0.1-2.fc23,gpaw-0.11.0.13004-15.fc23,ga-5.3b-18.fc23,valgrind-3.10.1-19.fc23,boost-1.58.0-6.fc23,dl_poly-1.9.20140324-14.fc23,hpl-2.1-13.fc23,netgen-mesher-5.3.1-8.fc23,espresso-3.3.0-7.fc23,scotch-6.0.4-5.fc23,hdf5-1.8.15-6.patch1.fc23,scalapack-2.0.2-10.fc23,gromacs-5.0.6-6.fc23,scorep-1.4.2-2.fc23,sundials-2.6.2-4.fc23,Ray-2.3.1-11.fc23,orsa-0.7.0-33.fc23,elk-3.1.12-14.fc23,mpich-3.1.4-5.fc23,openmpi-1.8.8-3.fc23,mpi4py-1.3.1-14.fc23,pypar-2.1.5_108-11.fc23,paraview-4.3.1-11.fc23

Comment 12 Mark Wielaard 2015-08-19 18:21:38 UTC
BTW. For valgrind you'll need 3.10.1-20 which has one additional fix for s390x (which doesn't have openmpi, so would break with 3.10.1-19 which did things unconditionally). 3.10.1-20 - Don't move libmpiwrap when not building for openmpi (s390x)


I'll do a backport to f23 and build. Should I also do the update?

Comment 13 Mark Wielaard 2015-08-19 19:12:13 UTC
valgrind-3.10.1-20.fc23 completed http://koji.fedoraproject.org/koji/buildinfo?buildID=678500

Let me know if I should push that as an update or if you want to add that to the update group.

Comment 14 Fedora Update System 2015-08-22 16:25:53 UTC
MUMPS-5.0.1-2.fc23, Ray-2.3.1-11.fc23, boost-1.58.0-6.fc23, dl_poly-1.9.20140324-14.fc23, elk-3.1.12-14.fc23, elpa-2015.05.001-2.fc23, espresso-3.3.0-7.fc23, freefem++-3.31-7.3.fc23, ga-5.3b-18.fc23, gpaw-0.11.0.13004-15.fc23, gromacs-5.0.6-6.fc23, hdf5-1.8.15-6.patch1.fc23, hpl-2.1-13.fc23, mathgl-2.3-10.fc23, mpi4py-1.3.1-14.fc23, mpich-3.1.4-5.fc23, netcdf-4.3.3.1-5.fc23, netgen-mesher-5.3.1-8.fc23, openmpi-1.8.8-3.fc23, orsa-0.7.0-33.fc23, paraview-4.3.1-11.fc23, pypar-2.1.5_108-11.fc23, rpm-mpi-hooks-3-2.fc23, scalapack-2.0.2-10.fc23, scorep-1.4.2-2.fc23, scotch-6.0.4-5.fc23, sundials-2.6.2-4.fc23, valgrind-3.10.1-19.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.\nIf you want to test the update, you can install it with \n su -c 'yum --enablerepo=updates-testing update freefem++ mathgl mpich netgen-mesher sundials paraview espresso netcdf elk valgrind gpaw elpa scotch boost orsa hpl Ray MUMPS mpi4py hdf5 pypar gromacs ga dl_poly openmpi rpm-mpi-hooks scorep scalapack'. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/rpm-mpi-hooks-3-2.fc23%2Cmathgl-2.3-10.fc23%2Celpa-2015.05.001-2.fc23%2Cfreefem%2B%2B-3.31-7.3.fc23%2Cnetcdf-4.3.3.1-5.fc23%2CMUMPS-5.0.1-2.fc23%2Cgpaw-0.11.0.13004-15.fc23%2Cga-5.3b-18.fc23%2Cvalgrind-3.10.1-19.fc23%2Cboost-1.58.0-6.fc23%2Cdl_poly-1.9.20140324-14.fc23%2Chpl-2.1-13.fc23%2Cnetgen-mesher-5.3.1-8.fc23%2Cespresso-3.3.0-7.fc23%2Cscotch-6.0.4-5.fc23%2Chdf5-1.8.15-6.patch1.fc23%2Cscalapack-2.0.2-10.fc23%2Cgromacs-5.0.6-6.fc23%2Cscorep-1.4.2-2.fc23%2Csundials-2.6.2-4.fc23%2CRay-2.3.1-11.fc23%2Corsa-0.7.0-33.fc23%2Celk-3.1.12-14.fc23%2Cmpich-3.1.4-5.fc23%2Copenmpi-1.8.8-3.fc23%2Cmpi4py-1.3.1-14.fc23%2Cpypar-2.1.5_108-11.fc23%2Cparaview-4.3.1-11.fc23

Comment 15 Mark Wielaard 2015-08-25 11:55:37 UTC
For the record I submitted an valgrind f23 update myself because the -19 version doesn't build on s390x. https://bodhi.fedoraproject.org/updates/valgrind-3.10.1-20.fc23

Comment 16 Zbigniew Jędrzejewski-Szmek 2015-08-27 15:06:25 UTC
(In reply to Mark Wielaard from comment #15)
> For the record I submitted an valgrind f23 update myself because the -19
> version doesn't build on s390x.
> https://bodhi.fedoraproject.org/updates/valgrind-3.10.1-20.fc23
Sorry, I missed your question. The update looks good.

Comment 17 Fedora Update System 2015-08-27 17:58:51 UTC
MUMPS-5.0.1-2.fc23, Ray-2.3.1-11.fc23, dl_poly-1.9.20140324-14.fc23, elk-3.1.12-14.fc23, elpa-2015.05.001-2.fc23, espresso-3.3.0-7.fc23, freefem++-3.31-7.3.fc23, ga-5.3b-18.fc23, gpaw-0.11.0.13004-15.fc23, gromacs-5.0.6-6.fc23, hdf5-1.8.15-6.patch1.fc23, hpl-2.1-13.fc23, mathgl-2.3-10.fc23, mpi4py-1.3.1-14.fc23, mpich-3.1.4-5.fc23, netcdf-4.3.3.1-5.fc23, netgen-mesher-5.3.1-8.fc23, openmpi-1.8.8-3.fc23, orsa-0.7.0-33.fc23, paraview-4.3.1-11.fc23, pypar-2.1.5_108-11.fc23, rpm-mpi-hooks-3-2.fc23, scalapack-2.0.2-10.fc23, scorep-1.4.2-2.fc23, scotch-6.0.4-5.fc23, sundials-2.6.2-4.fc23, valgrind-3.10.1-19.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.