Bug 2196438 - ceph fails to build with Python 3.12: fatal error: global_legacy_options.h: No such file or directory
Summary: ceph fails to build with Python 3.12: fatal error: global_legacy_options.h: N...
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: ceph
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kaleb KEITHLEY
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.12
TreeView+ depends on / blocked
 
Reported: 2023-05-09 07:13 UTC by Tomáš Hrnčiar
Modified: 2023-05-11 08:34 UTC (History)
9 users (show)

Fixed In Version: ceph-17.2.6-6.fc39
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-10 15:01:38 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 59690 0 None None None 2023-05-09 13:22:11 UTC

Description Tomáš Hrnčiar 2023-05-09 07:13:33 UTC
ceph fails to build with Python 3.12.0a7.

In file included from /builddir/build/BUILD/ceph-17.2.6/src/common/config_values.h:59,
                 from /builddir/build/BUILD/ceph-17.2.6/src/common/config.h:27,
                 from /builddir/build/BUILD/ceph-17.2.6/src/common/config_proxy.h:6,
                 from /builddir/build/BUILD/ceph-17.2.6/src/common/ceph_context.h:41,
                 from /builddir/build/BUILD/ceph-17.2.6/src/common/dout.h:29,
                 from /builddir/build/BUILD/ceph-17.2.6/src/include/Context.h:19,
                 from /builddir/build/BUILD/ceph-17.2.6/src/mon/mon_types.h:20,
                 from /builddir/build/BUILD/ceph-17.2.6/src/mon/MonMap.h:28,
                 from /builddir/build/BUILD/ceph-17.2.6/src/krbd.cc:44:
/builddir/build/BUILD/ceph-17.2.6/src/common/options/legacy_config_opts.h:1:10: fatal error: global_legacy_options.h: No such file or directory
    1 | #include "global_legacy_options.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

I found a similar error fixed in the upstream. I suppose, something like this will be needed to fix it. 
https://github.com/ceph/ceph/commit/3b3da64d741960e2ecb217c1990142f4a88f2fa3

https://docs.python.org/3.12/whatsnew/3.12.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.12/fedora-rawhide-x86_64/05892193-ceph/

For all our attempts to build ceph with Python 3.12, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.12/package/ceph/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.12:
https://copr.fedorainfracloud.org/coprs/g/python/python3.12/

Let us know here if you have any questions.

Python 3.12 is planned to be included in Fedora 39. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.12.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Kaleb KEITHLEY 2023-05-09 13:02:23 UTC
This is unrelated to the python version. The(In reply to Tomáš Hrnčiar from comment #0)
> ceph fails to build with Python 3.12.0a7.
> 
> In file included from
> /builddir/build/BUILD/ceph-17.2.6/src/common/config_values.h:59,
>                  from
> /builddir/build/BUILD/ceph-17.2.6/src/common/config.h:27,
>                  from
> /builddir/build/BUILD/ceph-17.2.6/src/common/config_proxy.h:6,
>                  from
> /builddir/build/BUILD/ceph-17.2.6/src/common/ceph_context.h:41,
>                  from /builddir/build/BUILD/ceph-17.2.6/src/common/dout.h:29,
>                  from
> /builddir/build/BUILD/ceph-17.2.6/src/include/Context.h:19,
>                  from
> /builddir/build/BUILD/ceph-17.2.6/src/mon/mon_types.h:20,
>                  from /builddir/build/BUILD/ceph-17.2.6/src/mon/MonMap.h:28,
>                  from /builddir/build/BUILD/ceph-17.2.6/src/krbd.cc:44:
> /builddir/build/BUILD/ceph-17.2.6/src/common/options/legacy_config_opts.h:1:
> 10: fatal error: global_legacy_options.h: No such file or directory
>     1 | #include "global_legacy_options.h"
>       |          ^~~~~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> ninja: build stopped: subcommand failed.

This is an issue that some people see occasionally. It's usually "fixed" by deleting the CMakeCache.txt and the cmake build directory and rerunning cmake.(And yes, I know that's not something you can easily do in mock and copr.) It has nothing to do with python or the version of python. It happens on machines with python-3.11 and 3.10 too.

> 
> I found a similar error fixed in the upstream. I suppose, something like
> this will be needed to fix it. 
> https://github.com/ceph/ceph/commit/3b3da64d741960e2ecb217c1990142f4a88f2fa3

That change is already included in ceph-17.2.6.

> 
> https://docs.python.org/3.12/whatsnew/3.12.html
> 
> For the build logs, see:
> https://copr-be.cloud.fedoraproject.org/results/@python/python3.12/fedora-
> rawhide-x86_64/05892193-ceph/
> 
> For all our attempts to build ceph with Python 3.12, see:
> https://copr.fedorainfracloud.org/coprs/g/python/python3.12/package/ceph/
> 
> Testing and mass rebuild of packages is happening in copr. You can follow
> these instructions to test locally in mock if your package builds with
> Python 3.12:
> https://copr.fedorainfracloud.org/coprs/g/python/python3.12/
> 
> Let us know here if you have any questions.
> 
> Python 3.12 is planned to be included in Fedora 39. To make that update
> smoother, we're building Fedora packages with all pre-releases of Python
> 3.12.
> A build failure prevents us from testing all dependent packages (transitive
> [Build]Requires), so if this package is required a lot, it's important for
> us to get it fixed soon.
> We'd appreciate help from the people who know this package best, but if you
> don't want to work on this now, let us know so we can try to work around it
> on our side.

Comment 2 Kaleb KEITHLEY 2023-05-10 13:52:23 UTC
I hacked up python3.12 on my rawhide box. I ran two rpmbuilds back to back. The first failed as described above, the second ran successfully to completion. I opened the ceph tracker referenced above to see if it's possible to get a more reliable fix. Maybe there's a race condition where krbd can try to compile before the header file is generated. Seems unrelated to python3.12 since it's been seen on python3.11 and python3.10 too.

Also the build.log at the link you posted (https://copr-be.cloud.fedoraproject.org/results/@python/python3.12/fedora-rawhide-x86_64/05892193-ceph/) shows a completely unrelated error. Kinda looks like the build was canceled.

I'll let this bz sit open for a few days, and then I'm going to close it as WORKSFORME.

Comment 3 Kaleb KEITHLEY 2023-05-10 15:01:38 UTC
1. works for me (99&44/100 percent of the time)
2. not a python or python3.12 issue
3. tentative fix - added a cmake dependency to the krbd compile in ceph-17.2.6-6

closing as WORKSFORME, reopen if you continue to have problems building in copr

Comment 4 Tomáš Hrnčiar 2023-05-11 08:34:12 UTC
(In reply to Kaleb KEITHLEY from comment #2)
> 
> Also the build.log at the link you posted
> (https://copr-be.cloud.fedoraproject.org/results/@python/python3.12/fedora-
> rawhide-x86_64/05892193-ceph/) shows a completely unrelated error. Kinda
> looks like the build was canceled.
> 

(In reply to Tomáš Hrnčiar from comment #0)
> I found a similar error fixed in the upstream. I suppose, something like
> this will be needed to fix it. 
> https://github.com/ceph/ceph/commit/3b3da64d741960e2ecb217c1990142f4a88f2fa3

My assumption was that something like the commit above will be needed so I tried to patch CMakeLists.txt, but it didn't help and it crashed the build.

 add_dependencies(common_prioritycache_obj legacy-option-headers)
+add_dependencies(common_mountcephfs_obj legacy-option-headers)

I forgot to delete this build when I was opening the bugzilla, sorry about that.

Anyways, thank you for confirming that ceph builds with Python 3.12.


Note You need to log in before you can comment on or make changes to this bug.