Bug 2302033

I think all the bugs 2301933 to 2301980 are also affected by this.
Should I add them all?

I believe this is because environment-modules has fallen off the dependency chain somehow and hence rpm-mpi-hooks fails to load the MPI environment and hence to generate the correct provides. rpm-mpi-hooks actually has

Requires:       environment(modules)

which makes me suspect a dnf5 issue? Explicitly Requiring: environment-modules fixes the issue, I've done this in rpm-mpi-hooks-8-10.fc41. As soon as this build is ready, rebuilding openmpi should be sufficient.

I can confirm that the explicit `Requires: environment(modules)` solves the issue. I did a scratch build of openmpi before I knew of the root cause. That build picked up the fixed rpm-mpi-hooks and produced correct metadata.

Logs of the openmpi build with wrong / incomplete metadata: https://koji.fedoraproject.org/koji/buildinfo?buildID=2519082

Build with fixed rpm-mpi-hooks: https://koji.fedoraproject.org/koji/taskinfo?taskID=121299407

Previous release of openmpi with good meta data: https://koji.fedoraproject.org/koji/buildinfo?buildID=2499148

I'm reassigning to dnf5 for now. From there it can trickle down further once it becomes clear what's actually causing the issue (libsolv?).

Extending on comment 0, here is the full difference of provides between openmpi 5.0.5-1 and 5.0.3-3:

$ rpm -q --provides -p openmpi-5.0.5-1.fc41.x86_64.rpm
config(openmpi) = 5.0.5-1.fc41
libmpi.so.40()(64bit)
libmpi_java.so.40()(64bit)
libmpi_mpifh.so.40()(64bit)
libmpi_usempi_ignore_tkr.so.40()(64bit)
libmpi_usempif08.so.40()(64bit)
liboshmem.so.40()(64bit)
mpi
openmpi = 5.0.5-1.fc41
openmpi(x86-64) = 5.0.5-1.fc41

$ rpm -q --provides -p openmpi-5.0.3-3.fc41.x86_64.rpm
config(openmpi) = 5.0.3-3.fc41
libmpi.so.40()(64bit)(openmpi-x86_64)
libmpi_java.so.40()(64bit)(openmpi-x86_64)
libmpi_mpifh.so.40()(64bit)(openmpi-x86_64)
libmpi_usempi_ignore_tkr.so.40()(64bit)(openmpi-x86_64)
libmpi_usempif08.so.40()(64bit)(openmpi-x86_64)
liboshmem.so.40()(64bit)(openmpi-x86_64)
mpi
openmpi = 5.0.3-3.fc41
openmpi(x86-64) = 5.0.3-3.fc41

(In reply to Sandro Mani from comment #2)
> I believe this is because environment-modules has fallen off the dependency
> chain somehow and hence rpm-mpi-hooks fails to load the MPI environment and
> hence to generate the correct provides. rpm-mpi-hooks actually has
> 
> Requires:       environment(modules)
> 
> which makes me suspect a dnf5 issue? Explicitly Requiring:
> environment-modules fixes the issue, I've done this in
> rpm-mpi-hooks-8-10.fc41. As soon as this build is ready, rebuilding openmpi
> should be sufficient.

I checked both 5.0.5-1 and 5.0.3-3 official builds. Neither has any mention of environment-modules nor environment(modules) in build.log or root.log. What am I missing?

I also saw that we have two packages providing environment(modules):

$ fedrq pkgs --resolve 'environment(modules)'
Lmod-8.7.44-2.fc41.x86_64
environment-modules-5.4.0-2.fc41.x86_64

In https://kojipkgs.fedoraproject.org//packages/openmpi/5.0.5/2.fc41/data/logs/x86_64/root.log you can see that environment-modules is being installed.

Yes. But that's the rebuild after you changed rpm-mpi-hooks from "Requires: environment(modules)" to "Requires: environment-modules". I was curious if it got installed in the 5.0.3-3 build, which has the correct meta data.

The logs for https://koji.fedoraproject.org/koji/buildinfo?buildID=2519082 at https://kojipkgs.fedoraproject.org//packages/openmpi/5.0.5/1.fc41/data/logs/x86_64/root.log show that Lmod was installed.  That provides 'environment(modules)' so it is entirely valid for it to be installed when you have 'Requires: environment(modules)' in your package.

If you really do need 'environment-modules' and not 'Lmod' (which it seems you do), then you should use that as the Requires. In other words, keep the change you made but don't look at it as a workaround -- it fixes a bug in the previous Requires, which were ambigious.

The issue is that an update to lua-lpeg 1.1.0 broke lua-json and thus Lmod - see bz#2302036.

(In reply to Orion Poplawski from comment #8)
> The issue is that an update to lua-lpeg 1.1.0 broke lua-json and thus Lmod -
> see bz#2302036.

Right. I saw a failed build for an updated `Lmod`, but didn't look into why it failed. With `Lmod` broken, that makes me think...

(In reply to Todd Zullinger from comment #7)
> If you really do need 'environment-modules' and not 'Lmod' (which it seems
> you do), then you should use that as the Requires. In other words, keep the
> change you made but don't look at it as a workaround -- it fixes a bug in
> the previous Requires, which were ambigious.

Does `rpm-mpi-hooks` really need `environment-modules`? As pointed out in comment 4 neither of the openmpi builds 5.0.3-3 nor 5.0.5-1 had it installed in the buildroot. Yet one produced correct meta data, while the other, with a broken Lmod, failed to do so.

At the same time installing `environment-modules` allowed the 5.0.5-2 build to succeed with correct meta data output. So, it looks like either `Lmod` or `environment-modules` can be used to produce the required correct meta data. If there's a preference for one over the other, I agree, it should be specified explicitly.

As said in comment 7, "environment(modules)" is provided by two packages:

$ dnf provides 'environment(modules)'
Lmod-8.7.44-2.fc41.x86_64 : Environmental Modules System in Lua
Repo        : rawhide
Matched from:
Provide    : environment(modules)

environment-modules-5.4.0-2.fc41.x86_64 : Provides dynamic modification of a user's
                                        : environment
Repo        : rawhide
Matched from:
Provide    : environment(modules)

For the solver both solutions are valid and you cannot rely on which one is selected. Btw, also openmpi-5.0.3-3.fc41 (previous build with good metadata) was build with Lmod installed as "environment(moduled)" provider.

That said, I don't think dnf5 can resolve this issue. Reassigning to Lmod component for further investigation.

I suppose this has turned out to be a duplicate of bug 2302036, where the Lmod issue is discussed. Though, I find the the `Lmod` vs. `environment-modules` discussion interesting.

> Does `rpm-mpi-hooks` really need `environment-modules`?

See https://src.fedoraproject.org/rpms/rpm-mpi-hooks/blob/rawhide/f/mpi.prov, ultimately it needs to be able to do "module load <module>". And I believe Lmod does provide this functionality, even though I was not aware of it's existence before. As I see it explicitly requiring environment-modules in rpm-mpi-hooks should not harm, and I guess this issue can be closed, as #2302036 is tracking the Lmod issue.

All rpm-mpi-hooks needs is a working 'module' implementation - either environment-modules or Lmod.  I'm fine with it being fixed to environment-modules though.