Bug 2079833 - [tracker] Get rid of multiple source path definitions
Summary: [tracker] Get rid of multiple source path definitions
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: cmake
Version: 37
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
Assignee: Björn 'besser82' Esser
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 2045824 2057738 2059134 2059188 2059201 2061754 2113510 2139909
TreeView+ depends on / blocked
 
Reported: 2022-04-28 10:48 UTC by Michal Schorm
Modified: 2023-09-18 04:36 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-08-24 15:35:31 UTC
Type: Bug
Embargoed:
bcotton: fedora_prioritized_bug+


Attachments (Terms of Use)

Description Michal Schorm 2022-04-28 10:48:12 UTC
Hello,

When 'cmake-3.23.0-rc2' was put in the Rawhide, it broke most of the packages using CMake in Fedora.

There are number of different reports hitting the same core issue:
  https://bugzilla.redhat.com/show_bug.cgi?id=2057738
  https://bugzilla.redhat.com/show_bug.cgi?id=2045824
  https://bugzilla.redhat.com/show_bug.cgi?id=2059134
  https://bugzilla.redhat.com/show_bug.cgi?id=2059188
  https://bugzilla.redhat.com/show_bug.cgi?id=2061754
  https://bugzilla.redhat.com/show_bug.cgi?id=2059201
and more people reporting by e-mail:
  https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/YM4WDDZL6PMSODITVYWMPWC4TC4447AY/#RDX4ZIAWIMFITNQYFGRD5GSTKWQSDHX4

---

The issue is that the CMake upstream changed order of precedence when the source path is defined multiple times.
The CMake upstream also stated, that defining the source path multiple times is an unsupported behavior and should be avoided:
|  cmake(1) now warns when multiple source paths are specified, as in cmake -S src1 src2.
|  This has never been officially documented or supported, but older versions accidentally
|  accepted multiple source paths and used the last path specified.
|  Update scripts to avoid passing multiple source path arguments.
https://cmake.org/cmake/help/v3.23/release/3.23.html#deprecated-and-removed-features

---

After the CMake upstream got aware of what problems it caused in Fedora, they opened a merge request to restore the behavior to the old one, but kept the warnings that that is an unsupported and problematic behavior:
  https://gitlab.kitware.com/cmake/cmake/-/issues/23334

The upstream however pointed out, that the fact that the source path is forced in the '%cmake' macro in Fedora is likely unfortunate.
There are currently more than a hundred _source_ packages using the multiple source path definition by defining the source path after the '%cmake' macro.

|  http://src.fedoraproject.org/repo/rpm-specs-latest.tar.xz
|  # grep -i -w -e '%cmake' -r ./ | grep -e " \." | wc -l
|  159

Some results might be missed e.g.: in cases the source path is defined on different line than the '%cmake' macro is used.

---

Even though you get number of reports you did not revert the update.
The CMake upstream released a version 3.23-1, which should contain the fix from
  https://gitlab.kitware.com/cmake/cmake/-/issues/23334
(please double check that), which could at least allow the Fedora packages to be built.

It is more than two weeks from when it was released:
  https://bugzilla.redhat.com/show_bug.cgi?id=2062783
and months since you broke many Fedora packages and blocked (or slowed) release processes in _not only_ Fedora Linux operating system, but also in its derivatives.

---

Looking at the macro:
  https://src.fedoraproject.org/rpms/cmake/blob/rawhide/f/macros.cmake
it seems that the issue was introduced by the out-of-source building change:
  https://fedoraproject.org/wiki/Changes/CMake_to_do_out-of-source_builds
in which you promised to take care of the CMake packages broken by that change, but you did not even back than (at least not to any of my packages) (and that's likely _one_ of the causes why all of those second source path definitions remained).

The issue is that only in some cases, people defined the second source path the same as the one "hardcoded" in the '%cmake' macro.
So while in some cases the fix might be as easy as simply removing the second definiton, in many others it likely won't.

---

Please,
1/ rebase to the CMake version the restores the original behavior
2/ create a self-contained change to clean up the issues you introduced to the Fedora by the 'cmake-3.23.0-rc2' rebase
3/ help the maintainers fix their packages you broke, create PRs with fixes
4/ discuss on the CMake upstream the possibility of updating the '%cmake' macro in Fedora in such a way it won't force the source path definition
  https://gitlab.kitware.com/cmake/cmake/-/issues/23334#note_1179674
5/ document the issue in the Fedora CMake Packaging Guidelines, with suggestions for the maintainers that needs to use a different path to the source, than the "hardcoded" one in the '%cmake' macro
  https://docs.fedoraproject.org/en-US/packaging-guidelines/CMake/

Comment 1 Michal Schorm 2022-04-28 10:51:29 UTC
Remove the second source path definition from the CMake command

The '%cmake' RPM macro in Fedora actually expands to:
| ...
|   /usr/bin/cmake \
|         -S "." \
|         -B "redhat-linux-build" \
| ...

So in this case the source patch was specified twice.
First in the macro with the '-S' option and second time outside of the macro,
in the SPECfile, without the '-S' option.

CMake upstream declares that:
|  This has never been officially documented or supported,
|  but older versions accidentally accepted multiple source paths
|  and used the last path specified. Update scripts to avoid
|  passing multiple source path arguments.
https://cmake.org/cmake/help/v3.23/release/3.23.html#deprecated-and-removed-features

This was discovered as CMake upstream implemented a change to the 3.23.0-rc2 release
that changed this behavior and it broke many Fedora packages that used this
double source path definition.
  See rhbz#2057738 to see how build behaved

After the CMake upstream got aware of what problems it caused in Fedora,
they opened a merge request to restore the behavior to the old one,
but kept the warnings that that is an unsupported and problematic behavior:
  https://gitlab.kitware.com/cmake/cmake/-/issues/23334

---

As for today, this issue is still not yet fully resolved.
- The CMake maintainers in Fedora haven't rebased the package to 3.23-1 release, so it is still broken
- Affected packages in Fedora should find a way to stop using this unsupported behaviour
- The double '-S' argument passing should be marked as problematic too, in the exact same way
  https://gitlab.kitware.com/cmake/cmake/-/issues/23334#note_1159258
- A change to the %cmake Fedora RPM macro might be in play, so it won't force a source path
  https://gitlab.kitware.com/cmake/cmake/-/issues/23334#note_1159258

Comment 2 Miro Hrončok 2022-06-10 11:11:11 UTC
Nominating as a prioritized bug. This still causes too much friction, see e.g. https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/FEKQKSOF5DA6WQ4CD2I6M5VJ7QBTSO6X/

Comment 3 Ben Cotton 2022-06-15 14:58:10 UTC
In today's Prioritized Bugs meeting, we agreed to accept this as a prioritized bug due to the impact on package maintainers. cmake maintainers should update with the upstream package and submit a System-Wide Change proposal for future releases where upstream re-implements this or similar change that require coordination with other packagers (as outlined in the Rawhide updates policy
https://meetbot.fedoraproject.org/fedora-meeting-1/2022-06-15/fedora_prioritized_bugs_and_issues.2022-06-15-14.00.log.html#l-83

Comment 4 Ben Cotton 2022-07-01 18:39:17 UTC
Björn, do you have an update on this? Is there something Matthew or I can do to get you the resources you need to resolve this?

Comment 5 Ben Cotton 2022-08-06 15:59:52 UTC
I see cmake-3.23.3-1.fc37 landed in Rawhide recently. As I understand it, this should fix (in the short-term, at least) the issue reported here. Can someone please confirm this?

Comment 6 Miro Hrončok 2022-08-06 20:08:38 UTC
I've just tested the meshlab package from before this fix: https://src.fedoraproject.org/rpms/meshlab/c/1423fa94218fb25e88262e263db17b84eba4859d?branch=rawhide

It seems to at least get past the point where it initially failed.



I've also tested libarcus before https://src.fedoraproject.org/rpms/libarcus/c/5b4abb1317e4020b5f903679611dc4b2cca498aa?branch=rawhide


It seems to build.


I don't have enough time to verify this fixes everything here, but it sure seems like the situation has improved.

Comment 7 Michal Schorm 2022-08-06 22:49:30 UTC
Here is the test on the package 'galera'

-----

# fedpkg clone galera && cd galera

# grep "%cmake " galera.spec 
    %cmake \

# fedpkg build --scratch --srpm --arches=x86_64
    https://koji.fedoraproject.org/koji/taskinfo?taskID=90548809

# git revert 64680cf3405ede44b0fb4f5f4014884e8d3b997a
    Auto-merging galera.spec
    [rawhide 2b4fa89] Revert "Remove the second source path definition from the CMake command"
     1 file changed, 1 insertion(+), 1 deletion(-)

# grep "%cmake " galera.spec 
    %cmake . \

# fedpkg build --scratch --srpm --arches=x86_64
    https://koji.fedoraproject.org/koji/taskinfo?taskID=90548814

-----

Both variants (single and double source path definition) succeeded, so the workaround (rebase of the 'cmake' package to a release in which the upstream reverted the change leading to our problems) works.
The issue with double source path definitions (where the first one is baked into '%cmake' macro) across the Fedora packages remains.

Comment 8 Ben Cotton 2022-08-08 19:52:03 UTC
Setting this bug to VERIFIED. At the next Prioritized Bugs meeting (Wed 10 Aug), we'll discuss whether to leave it open. My inclination is to close it as the immediate concern has passed and continue leaning on the cmake maintainers to coordinate a long-term fix for when upstream re-removes the behavior.

Comment 9 Ben Cotton 2022-08-09 13:42:45 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 37 development cycle.
Changing version to 37.

Comment 10 Ben Cotton 2022-08-24 15:35:31 UTC
In today's Prioritized Bugs meeting, we agreed this is solved enough for our purposes and we encourage cmake maintainers to use the Changes process for future updates to prevent similar issues: https://meetbot.fedoraproject.org/fedora-meeting-1/2022-08-24/fedora_prioritized_bugs_and_issues.2022-08-24-14.00.log.html#l-113

Setting status to CLOSED

Comment 11 Red Hat Bugzilla 2023-09-18 04:36:20 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.