Bug 1923658

Created attachment 1753915 [details]
build.log

file build.log too big, will only attach last 32768 bytes

Created attachment 1753916 [details]
root.log

file root.log too big, will only attach last 32768 bytes

Created attachment 1753917 [details]
state.log

This a segfault described in https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/ADKCB6HXGZHJPYX5LSJ7422FITYSOAL3/

This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Dear Maintainer,

your package has an open Fails To Build From Source bug for Fedora 34.
Action is required from you.

If you can fix your package to build, perform a build in koji, and either create
an update in bodhi, or close this bug without creating an update, if updating is
not appropriate [1]. If you are working on a fix, set the status to ASSIGNED to
acknowledge this. If you have already fixed this issue, please close this Bugzilla report.

Following the policy for such packages [2], your package will be orphaned if
this bug remains in NEW state more than 8 weeks (not sooner than 2021-03-29).

A week before the mass branching of Fedora 35 according to the schedule [3],
any packages not successfully rebuilt at least on Fedora 33 will be
retired regardless of the status of this bug.

[1] https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/
[2] https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/
[3] https://fedorapeople.org/groups/schedule/f-35/f-35-key-tasks.html

Dear Maintainer,

your package has an open Fails To Build From Source bug for Fedora 34.
Action is required from you.

If you can fix your package to build, perform a build in koji, and either create
an update in bodhi, or close this bug without creating an update, if updating is
not appropriate [1]. If you are working on a fix, set the status to ASSIGNED to
acknowledge this. If you have already fixed this issue, please close this Bugzilla report.

Following the policy for such packages [2], your package will be orphaned if
this bug remains in NEW state more than 8 weeks (not sooner than 2021-03-29).

A week before the mass branching of Fedora 35 according to the schedule [3],
any packages not successfully rebuilt at least on Fedora 33 will be
retired regardless of the status of this bug.

[1] https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/
[2] https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/
[3] https://fedorapeople.org/groups/schedule/f-35/f-35-key-tasks.html

Dear Maintainer,

your package has an open Fails To Build From Source bug for Fedora 34.
Action is required from you.

If you can fix your package to build, perform a build in koji, and either create
an update in bodhi, or close this bug without creating an update, if updating is
not appropriate [1]. If you are working on a fix, set the status to ASSIGNED to
acknowledge this. If you have already fixed this issue, please close this Bugzilla report.

Following the policy for such packages [2], your package will be orphaned if
this bug remains in NEW state more than 8 weeks (not sooner than 2021-03-29).

A week before the mass branching of Fedora 35 according to the schedule [3],
any packages not successfully rebuilt at least on Fedora 33 will be
retired regardless of the status of this bug.

[1] https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/
[2] https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/
[3] https://fedorapeople.org/groups/schedule/f-35/f-35-key-tasks.html

The previous comment was me, tying to debug why the email went twice before.

Created attachment 1757667 [details]
Reproducer script (reduced Lib/ctypes/test/test_as_parameter.py)

I'm stumped; after a day I can't figure this out.
I couldn't reproduce with mockbuild.

I used koji's save-failed-tree plugin to get the tree from a failed official build of python3.5, here: https://kojipkgs.fedoraproject.org//work/tasks/3640/62173640/broot-25777485.tar.gz

Using build/optimized/python from that, I could reproduce the segfault. I've reduced the segfaulting test case to a minimal reproducer.

Thanks Petr!

Just to clarify things:

If you build the package in mock, it won't crash during the tests? It won't crash when running the reproducer from the installed package either?

If you build the package in Koji with tests disabled, does it crash locally (in mock or podman) when running the reproducer from the installed package? If it does, we should not disable the crashing test.

Now I reprocuced it locally. Weird. But I got a bit further.

As far as I can tell, it's an alignment issue! The pointer from GenericPyCData_new is not aligned.

Notes mainly for myself follow.

The relevant code in PyCFuncPtr_new is:

    self = (PyCFuncPtrObject *)GenericPyCData_new(type, args, kwds);
    if (self == NULL) {
        Py_DECREF(thunk);
        return NULL;
    }

    Py_INCREF(callable);
    self->callable = callable;

    self->thunk = thunk;
    *(void **)self->b_ptr = (void *)thunk->pcl_exec;

which disassembles to:

# call GenericPyCData_new
   0x7fffe9b413c5 <PyCFuncPtr_new+229>     call   0x7fffe9b41760 <GenericPyCData_new>
# put `self` in %rax
   0x7fffe9b413ca <PyCFuncPtr_new+234>     mov    %rax,%r15
# do the `(self == NULL)` check
   0x7fffe9b413cd <PyCFuncPtr_new+237>     test   %rax,%rax
   0x7fffe9b413d0 <PyCFuncPtr_new+240>     je     0x7fffe9b3db2e <PyCFuncPtr_new-14258>
# start the INCREF (`callable` is in %rsi)
   0x7fffe9b413d6 <PyCFuncPtr_new+246>     mov    0x60(%rsp),%rsi
# meanwhile, put `thunk` in %xmm0 (from %r14)
   0x7fffe9b413db <PyCFuncPtr_new+251>     movq   %r14,%xmm0
# put `self->b_ptr` into %rdi
   0x7fffe9b413e0 <PyCFuncPtr_new+256>     mov    0x10(%rax),%rdi
# put `thunk` in %rdx
   0x7fffe9b413e4 <PyCFuncPtr_new+260>     mov    %r14,%rdx
# put `callable` in %xmm1
   0x7fffe9b413e7 <PyCFuncPtr_new+263>     movq   %rsi,%xmm1
# finish the INCREF (add 1 to callable's reference count)
   0x7fffe9b413ec <PyCFuncPtr_new+268>     addq   $0x1,(%rsi)
# ??? zero %esi, why?
   0x7fffe9b413f0 <PyCFuncPtr_new+272>     xor    %esi,%esi
# put `thunk` into lower half of %xmm0 (and `callable` in the upper half)
   0x7fffe9b413f2 <PyCFuncPtr_new+274>     punpcklqdq %xmm1,%xmm0
# put `thunk` into `self->thunk` (i.e. self+0x60)
   0x7fffe9b413f6 <PyCFuncPtr_new+278>     movaps %xmm0,0x60(%rax)

For movaps, the 0x60(%rax) must be aligned, but it's not: I get 0x7fffe9b614f8 for `self`.

> For movaps, the 0x60(%rax) must be aligned, but it's not: I get 0x7fffe9b614f8 for `self`.

Oh, great analysis Petr ;-)

This issue reminds me https://bugs.python.org/issue36618

The x86-64 ABI requires that memory allocated on the heap is aligned to 16 bytes, but Python 3.6 pymalloc memory allocator only provides alignment on 8 bytes. This has been fixed in Python 3.7 in newer by https://bugs.python.org/issue27987

See the Python 3.7 fix:
https://github.com/python/cpython/commit/8766cb74e186d3820db0a855ccd780d6d84461f7

Rawhide is built, older Fedoras in progress.

FEDORA-2021-3e8064da6e has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-3e8064da6e

FEDORA-2021-3e8064da6e has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-3e8064da6e`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-3e8064da6e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2021-3e8064da6e has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.