Bug 2046865

Created attachment 1856221 [details]
build.log

file build.log too big, will only attach last 32768 bytes

Created attachment 1856222 [details]
root.log

file root.log too big, will only attach last 32768 bytes

Created attachment 1856223 [details]
state.log

It seems the tests don't like https://fedoraproject.org/wiki/Changes/SetBuildFlagsBuildCheck

Will try to opt-out.

https://src.fedoraproject.org/rpms/python-cffi/pull-request/13

https://src.fedoraproject.org/rpms/python-cffi/pull-request/13 fixes the main issue, but the build still fails on ppc64le:

=================================== FAILURES ===================================
__________________________ test_longdouble_precision ___________________________
    def test_longdouble_precision():
        # Test that we don't loose any precision of 'long double' when
        # passing through Python and CFFI.
        ffi = FFI()
        ffi.cdef("long double step1(long double x);")
        SAME_SIZE = ffi.sizeof("long double") == ffi.sizeof("double")
        lib = ffi.verify("""
            long double step1(long double x)
            {
                return 4*x-x*x;
            }
        """)
        def do(cast_to_double):
            x = 0.9789
            for i in range(10000):
                x = lib.step1(x)
                if cast_to_double:
                    x = float(x)
            return float(x)
    
        more_precise = do(False)
        less_precise = do(True)
        if SAME_SIZE:
            assert more_precise == less_precise
        else:
>           assert abs(more_precise - less_precise) > 0.1
E           assert 0.0 > 0.1
E            +  where 0.0 = abs((0.0 - 0.0))
testing/cffi0/test_verify.py:199: AssertionError
__________________________ test_longdouble_precision ___________________________
    def test_longdouble_precision():
        # Test that we don't loose any precision of 'long double' when
        # passing through Python and CFFI.
        ffi = FFI()
        ffi.cdef("long double step1(long double x);")
        SAME_SIZE = ffi.sizeof("long double") == ffi.sizeof("double")
        lib = ffi.verify("""
            long double step1(long double x)
            {
                return 4*x-x*x;
            }
        """)
        def do(cast_to_double):
            x = 0.9789
            for i in range(10000):
                x = lib.step1(x)
                if cast_to_double:
                    x = float(x)
            return float(x)
    
        more_precise = do(False)
        less_precise = do(True)
        if SAME_SIZE:
            assert more_precise == less_precise
        else:
>           assert abs(more_precise - less_precise) > 0.1
E           assert 0.0 > 0.1
E            +  where 0.0 = abs((0.0 - 0.0))
testing/cffi0/test_verify.py:199: AssertionError

does python-cffi interact with libffi or is it a standalone module? libffi will need updates for the "long double" change, it is FTBFS too

It has:

BuildRequires:  libffi-devel


And python3-cffi requires (on x86_64):

libffi.so.8()(64bit)
libffi.so.8(LIBFFI_BASE_8.0)(64bit)
libffi.so.8(LIBFFI_CLOSURE_8.0)(64bit)

Retrying in https://src.fedoraproject.org/rpms/python-cffi/pull-request/13 with libffi-3.4.2-8.fc36

It still fails :(

I wonder if it needs python to be rebuilt too, it uses libffi-devel in its buildroot too ...

Trying a Python rebuild at ppc64le-test.fedorainfracloud.org

Bah, still the same problem.

python3.10 got rebuilt for another reason in rawhide. Trying https://src.fedoraproject.org/rpms/python-cffi/pull-request/13 again in case my local testing was not bulletproof.

Same failure.

yeah, I think we have exhausted the simple tricks, now is hard work needed ...

This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

This poses a significant risk for timely Python 3.11 delivery in Fedora 37 - a month has passed and there is no clear way forward. Hence, rising severity.

Still happening with gcc-12.0.1-0.12.fc37.ppc64le.

Narrowed it down to `force_generic_engine`, which is apparently used in PyPy and tests.
FWIW, the skips ("don't know the very exact precision of 'long double'") should be fine, that's an extra part of the test explicitly for some Intel-like arches.
Will look further.

Smaller reproducer:

from cffi import FFI

ffi = FFI()
ffi.cdef("long double step1(long double x);")

lib = ffi.verify(r"""
    #include <stdio.h>
        long double step1(long double x)
        {
            printf("C got: %Lf\n", x);
            long double result =  4*x-x*x;
            printf("C returns: %Lf\n", x);
            return result;
        }
    """,
    force_generic_engine=True,
)

x = lib.step1(0.9789)
print(f"Python: {x = }, {float(x) = }")


----
C got: 0.978900
C returns: 0.978900
Python: x = <cdata 'long double' 5.130162E-4937>, float(x) = 0.0

(That should be `result`, and `C returns: 2.957355`)

Looks like an alignment issue, goes away with the following PoC patch for generating cif_descr.exchange_offset_arg (offsets to return value & arguments in the buffer used when calling a ffi function).
Hardcoding long double alignment isn't the right solution, I'll dig further for a real fix.

diff -rU3 cffi-1.15.0-orig/c/_cffi_backend.c cffi-1.15.0/c/_cffi_backend.c
--- cffi-1.15.0-orig/c/_cffi_backend.c	2021-10-13 01:59:28.000000000 +0200
+++ cffi-1.15.0/c/_cffi_backend.c	2022-03-18 16:33:38.086767827 +0100
@@ -5665,7 +5665,7 @@
     }
 }
 
-#define ALIGN_ARG(n)  ((n) + 7) & ~7
+#define ALIGN_ARG(n)  ((n) + 15) & ~15
 
 static int fb_build(struct funcbuilder_s *fb, PyObject *fargs,
                     CTypeDescrObject *fresult)

I opened an upstream issue: https://foss.heptapod.net/pypy/pypy/-/issues/3708

https://foss.heptapod.net/pypy/cffi/-/commit/4d0cc66daf88f477f3a11262d0e2e093f9397e2c

https://src.fedoraproject.org/rpms/python-cffi/pull-request/16

thanks guys, it makes sense about the alignment, the new "long double" is 16B long and might require a "natural" alignment

FEDORA-2022-6042335ce9 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-6042335ce9

FEDORA-2022-a41682259c has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2022-a41682259c

FEDORA-2022-a41682259c has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.

Follow-up on the CPython side:

- I checked that PyObject_Malloc() currently aligns to 16B (on 64-bit arches).
- I filed https://bugs.python.org/issue47179 to use correct alignment, using C11's `max_align_t`.

FEDORA-2022-6042335ce9 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-6042335ce9`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-6042335ce9

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2022-6042335ce9 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.