Bug 2062182 - systemtap fails to build with Python 3.11: error: storage size of _dummy_frame isn't known
Summary: systemtap fails to build with Python 3.11: error: storage size of _dummy_fram...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: systemtap
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Stan Cox
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 2099178 (view as bug list)
Depends On:
Blocks: PYTHON3.11 F37FTBFS F37FailsToInstall
TreeView+ depends on / blocked
 
Reported: 2022-03-09 10:38 UTC by Tomáš Hrnčiar
Modified: 2022-07-14 20:52 UTC (History)
9 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2022-07-14 20:52:27 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tomáš Hrnčiar 2022-03-09 10:38:11 UTC
systemtap fails to build with Python 3.11.0a5.

HelperSDT/_HelperSDT.c:17:15: error: storage size of ‘_dummy_frame’ isn’t known
   17 | PyFrameObject _dummy_frame;
      |               ^~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1

The PyFrameObject structure member has been moved to the internal C API headers.

While the documentation notes that the PyFrameObject fields are subject to change at any time, they have been stable for a long time and were used in several popular extensions.

In Python 3.11, the frame struct was reorganized to allow performance optimizations. Some fields were removed entirely, as they were details of the old implementation.

PyFrameObject fields:
    f_back: use PyFrame_GetBack().
    f_blockstack: removed.
    f_builtins: use PyObject_GetAttrString((PyObject*)frame, "f_builtins").
    f_code: use PyFrame_GetCode().
    f_gen: removed.
    f_globals: use PyObject_GetAttrString((PyObject*)frame, "f_globals").
    f_iblock: removed.
    f_lasti: use PyObject_GetAttrString((PyObject*)frame, "f_lasti"). Code using f_lasti with PyCode_Addr2Line() must use PyFrame_GetLineNumber() instead.
    f_lineno: use PyFrame_GetLineNumber()
    f_locals: use PyObject_GetAttrString((PyObject*)frame, "f_locals").
    f_stackdepth: removed.
    f_state: no public API (renamed to f_frame.f_state).
    f_trace: no public API.
    f_trace_lines: use PyObject_GetAttrString((PyObject*)frame, "f_trace_lines") (it also be modified).
    f_trace_opcodes: use PyObject_GetAttrString((PyObject*)frame, "f_trace_opcodes") (it also be modified).
    f_localsplus: no public API (renamed to f_frame.localsplus).
    f_valuestack: removed.

The Python frame object is now created lazily. A side effect is that the f_back member must not be accessed directly, since its value is now also computed lazily. The PyFrame_GetBack() function must be called instead.

https://docs.python.org/3.11/whatsnew/3.11.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.11/fedora-rawhide-x86_64/03640728-systemtap/

For all our attempts to build systemtap with Python 3.11, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/package/systemtap/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.11:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/

Let us know here if you have any questions.

Python 3.11 is planned to be included in Fedora 37. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.11.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Frank Ch. Eigler 2022-03-10 16:24:11 UTC
Thanks for the advanced heads-up.  We can probably make this work fine via the new API functions on new enough python.

Comment 2 Miro Hrončok 2022-06-20 10:20:53 UTC
*** Bug 2099178 has been marked as a duplicate of this bug. ***

Comment 3 Stan Cox 2022-07-09 02:27:55 UTC
Systemtap mostly uses the C code to grab the PyFrameObject and then it uses a systemtap tapset to do the actual manipulation.  That can be changed to use the access functions where available:

 systemtap uses	      	  switch to:
f_back	   		  PyFrame_GetBack().
f_code			  PyFrame_GetCode()
f_globals		  PyFrame_GetGlobals()
f_lasti			  PyFrame_GetLasti()
f_lineno		  PyFrame_GetLineNumber()

That leaves
f_localsplus		  (can f_frame.localsplus be accessed?)
			  (used to get local variable names)
f_trace
			  (used to get lineno, is this check needed?)
    if (@Py3FrameObject(frame)->f_trace)
        return @Py3FrameObject(frame)->f_lineno

Comment 4 Stan Cox 2022-07-11 21:58:30 UTC
Peripherally related: builds okay with python3-3.10.5-2.fc36.x86_64 but a field reference in the tapset fails:
 unable to find member 'co_lnotab' for struct PyCodeObject

Comment 5 Frank Ch. Eigler 2022-07-14 20:52:27 UTC
systemtap-4.8~pre16578235g069e109c-1.fc37 fixes the buildability problems; fixes for remaining tapset problems are underway


Note You need to log in before you can comment on or make changes to this bug.