Bug 2343969
| Summary: | protobuf doesn't build on s390x and i686, blocking the Python 3.14 rebuild; also raises "TypeError: Metaclasses with custom tp_new are not supported" in dependent packages | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Karolina Surma <ksurma> |
| Component: | protobuf | Assignee: | Michel Lind <michel> |
| Status: | CLOSED WORKSFORME | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rawhide | CC: | adrian, code, fkolwa, fti-bugs, igor.raits, jakub, jonathan, ksurma, mhroncok, michel, mizdebsk, msaulnier, nolear, orion, python-packagers-sig, sander |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | protobuf-3.19.6-14.fc43 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2025-06-17 21:58:09 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 2322407, 2339432, 2339435, 2365821, 2371687, 2371708, 2371760, 2371785, 2371884, 2371917, 2372025, 2372120, 2372186, 2365822, 2366200, 2366202, 2366486, 2366510, 2368591, 2371673, 2371691, 2371710, 2371721, 2371728, 2371735, 2371741, 2371751, 2371753, 2371764, 2371925, 2371926, 2371927, 2371928, 2371931, 2372027, 2372044, 2372125, 2372136 | ||
I see this has been reported upstream years ago and likely mitigated in the newer code, but Fedora's protobuf has not been updated since: https://github.com/protocolbuffers/protobuf/issues/12186 *** Bug 2356165 has been marked as a duplicate of this bug. *** https://copr.fedorainfracloud.org/coprs/g/python/python3.14/package/protobuf succeeded but failed in Koji on s390x. *** Bug 2371775 has been marked as a duplicate of this bug. *** failed in Koji on i686 as well. i686:
In function 'TagDispatch',
inlined from 'ToTagDispatch.isra' at ./google/protobuf/generated_message_tctable_impl.h:137:41:
./google/protobuf/generated_message_tctable_impl.h:124:48: error: cannot tail-call: callee required more stack slots than the caller
124 | PROTOBUF_MUSTTAIL return fast_entry->target(PROTOBUF_TC_PARAM_PASS);
| ^
make[3]: *** [/tmp/cccMkbnw.mk:8: /tmp/ccIoel2x.ltrans3.ltrans.o] Error 1
make[3]: *** Waiting for unfinished jobs....
lto-wrapper: fatal error: make returned 2 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:2746: libprotobuf-lite.la] Error 1
s390x:
In function 'google::protobuf::internal::TcParser::TagDispatch(google::protobuf::MessageLite*, char const*, google::protobuf::internal::ParseContext*, google::protobuf::internal::TcParseTableBase const*, unsigned long, google::protobuf::internal::TcFieldData)',
inlined from 'google::protobuf::internal::TcParser::ToTagDispatch(google::protobuf::MessageLite*, char const*, google::protobuf::internal::ParseContext*, google::protobuf::internal::TcParseTableBase const*, unsigned long, google::protobuf::internal::TcFieldData)' at ./google/protobuf/generated_message_tctable_impl.h:137:41,
inlined from 'google::protobuf::internal::TcParser::SingularVarint<bool, unsigned short, (google::protobuf::internal::TcParser::VarintDecode)0>(google::protobuf::MessageLite*, char const*, google::protobuf::internal::ParseContext*, google::protobuf::internal::TcParseTableBase const*, unsigned long, google::protobuf::internal::TcFieldData)char const*' at google/protobuf/generated_message_tctable_lite.cc:313:41:
./google/protobuf/generated_message_tctable_impl.h:124:48: error: cannot tail-call: target is not able to optimize the call into a sibling call
124 | PROTOBUF_MUSTTAIL return fast_entry->target(PROTOBUF_TC_PARAM_PASS);
| ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
(repeats several times)
Note that even when built, the "Metaclasses with custom tp_new are not supported" problem in dependent packages will probably surface again. Adding Jakub to CC as he seems to be involved in the GCC upstream issue linked from https://github.com/protocolbuffers/protobuf/issues/21333 Temporarily use GCC 14 to workaround FTBFS on i686 and s390x: https://src.fedoraproject.org/rpms/protobuf/pull-request/30 > Status: NEW → POST
FWIW this only solves the FTBFS, not the TypeError: Metaclasses with custom tp_new are not supported problem. That's why I kept this as NEW.
There's also https://src.fedoraproject.org/rpms/protobuf/pull-request/19, which claims to fix the tp_new thing but Was not tested with 3.14. I'll test that in our copr. Thanks for the PR fixing FTBFS. The was merged, do you think it's safe to build it now and push to rawhide, or you would rather wait until the other issue is fixed too? As I understand, the existing build in rawhide also has the issue with tp_new, so making a new build with GCC 14 won't make things worse. (In reply to Miro Hrončok from comment #10) > > Status: NEW → POST > > FWIW this only solves the FTBFS, not the TypeError: Metaclasses with custom > tp_new are not supported problem. That's why I kept this as NEW. I'm fine with NEW, POST or MODIFIED as a (partial) fix was committed. Please attach preprocessed sources + gcc command line options for the failures you get, it has been a nightmare to get anything from protobuf upstream. If you want a workaround, just disable the clang::musttail attribute on specific architectures. Currently the only known thing are [[clang::musttail]] errors with -fsanitize=address, those have fixed written but waiting for patch review. For s390x, the thing is that some tail calls are simply not possible on the architecture at all, clang has been cheating and silently ignored the attribute. There is an optimization for s390x in some specific cases (passing through unmodified argument, see https://gcc.gnu.org/PR119873 and https://gcc.gnu.org/r16-139 and https://gcc.gnu.org/r15-9590 ), but those should be in Fedora already. As for i686, that arch indeed can't handle tail calls from functions with fewer arguments to functions with more arguments (generally all arches need that beyond a few arguments passed in registers, so it affects primarily arches which pass everything on stack). And I bet we ought to prevent some IPA optimizations changing call argument types and their numbers on functions which have musttail calls, but really will need a reproducer. > The was merged, do you think it's safe to build it now and push to rawhide, or you would rather wait until the other issue is fixed too? I was planning to ship it (as some dependent packages are not affected by the other issue). But now, when I discovered https://src.fedoraproject.org/rpms/protobuf/pull-request/19 I plan to test it in Copr (via https://src.fedoraproject.org/rpms/protobuf/pull-request/31 + builds of python-googleapis-common-protos, criu, python-axolotl, python-aioesphomeapi, pychromecast, onnx, python-carbon). ---- > If you want a workaround, just disable the clang::musttail attribute on specific architectures. I am not sure how to do that. For now, the workaround is to use GCC 14. There are now 2 dirty workarounds in protobuf (uses GCC 14 and pure Python module only). Hopefully, we can get rid of them with https://fedoraproject.org/wiki/Changes/Protobuf_5.x/6.x I will close this bug with protobuf-3.19.6-13.fc43 -- I don't have the energy to track a "proper" fix for this here, considering how old this protobuf version in Fedora is. Eventually, we might be able to attempt a gcc 15 rebuild at least on non-i686 tracked via https://github.com/fedora-eln/eln/issues/256 It is a matter of patching
src/google/protobuf/port_def.inc
Seems current protobuf trunk disables it on both i?86 and s390x already:
#if ABSL_HAVE_CPP_ATTRIBUTE(clang::musttail) && !defined(__arm__) && \
!defined(_ARCH_PPC) && !defined(__wasm__) && \
!(defined(_MSC_VER) && defined(_M_IX86)) && !defined(__i386__) && \
!defined(__s390x__)
// Compilation fails on ARM32: b/195943306
// Compilation fails on powerpc64le: b/187985113
// Compilation fails on X86 Windows:
// https://github.com/llvm/llvm-project/issues/53271
#define PROTOBUF_MUSTTAIL [[clang::musttail]]
#define PROTOBUF_TAILCALL true
#else
#define PROTOBUF_MUSTTAIL
#define PROTOBUF_TAILCALL false
#endif
If there is an older version, just add && !defined(__i386__) and/or && !defined()__s390x__) to it.
Thanks, trying a scratchbuild with: #if __has_cpp_attribute(clang::musttail) && \ - !defined(__arm__) && !defined(_ARCH_PPC) && !defined(__wasm__) + !defined(__arm__) && !defined(__i386__) && !defined(__s390x__) && !defined(_ARCH_PPC) && !defined(__wasm__) Apparently, that works. https://src.fedoraproject.org/rpms/protobuf/pull-request/32 Thanks, Jakub. Hello, Please note that this comment was generated automatically by https://pagure.io/releng/blob/main/f/scripts/ftbfs-fti/follow-policy.py If you feel that this output has mistakes, please open an issue at https://pagure.io/releng/ All subpackages of a package against which this bug was filled are now installable or removed from Fedora 43. Thanks for taking care of it! |
python-googleapis-common-protos fails to build with Python 3.14.0a4. =================================== FAILURES =================================== ______________________________ test_error_reason _______________________________ def test_error_reason(): > from google.api import error_reason_pb2 tests/unit/test_google_api_error_reason.py:18: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../BUILDROOT/usr/lib/python3.14/site-packages/google/api/error_reason_pb2.py:21: in <module> from google.protobuf import descriptor as _descriptor _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ # Protocol Buffers - Google's data interchange format # Copyright 2008 Google Inc. All rights reserved. # https://developers.google.com/protocol-buffers/ # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. """Descriptors essentially contain exactly the information found in a .proto file, in types that make this information accessible in Python. """ __author__ = 'robinson (Will Robinson)' import threading import warnings from google.protobuf.internal import api_implementation _USE_C_DESCRIPTORS = False if api_implementation.Type() == 'cpp': # Used by MakeDescriptor in cpp mode import binascii import os > from google.protobuf.pyext import _message E TypeError: Metaclasses with custom tp_new are not supported. https://docs.python.org/3.14/whatsnew/3.14.html For the build logs, see: https://copr-be.cloud.fedoraproject.org/results/@python/python3.14/fedora-rawhide-x86_64/08584155-python-googleapis-common-protos/ For all our attempts to build python-googleapis-common-protos with Python 3.14, see: https://copr.fedorainfracloud.org/coprs/g/python/python3.14/package/python-googleapis-common-protos/ Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.14: https://copr.fedorainfracloud.org/coprs/g/python/python3.14/ Let us know here if you have any questions. Python 3.14 is planned to be included in Fedora 43. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.14. A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon. We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.