Bug 2043555

Summary: __CHAR_UNSIGNED__, which is a reserved symbol, is defined in /usr/include/python3.10/pyconfig-64.h
Product: [Fedora] Fedora Reporter: Kaleb KEITHLEY <kkeithle>
Component: python3.10Assignee: Tomáš Hrnčiar <thrnciar>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: rawhideCC: cstratak, fberat, jakub, mhroncok, python-maint, python-sig, thrnciar, torsava, vstinner
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python3.10-3.10.2-3.fc36 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-02-02 09:24:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kaleb KEITHLEY 2022-01-21 14:08:06 UTC
Description of problem:

After rawhide updated gcc to gcc-12 the ceph builds have started to fail on aarch64,ppc64le, and s390x, e.g. 

/usr/bin/g++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -I/builddir/build/BUILD/ceph-16.2.7/build/redhat-linux-build/src/include -I/builddir/build/BUILD/ceph-16.2.7/src -isystem /builddir/build/BUILD/ceph-16.2.7/build/redhat-linux-build/include -isystem /builddir/build/BUILD/ceph-16.2.7/src/xxHash -isystem /builddir/build/BUILD/ceph-16.2.7/src/rapidjson/include -isystem /usr/include/python3.10 -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -O2 -g -DNDEBUG -fPIE   -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -std=c++17 -MD -MT src/mgr/CMakeFiles/ceph-mgr.dir/ActivePyModule.cc.o -MF src/mgr/CMakeFiles/ceph-mgr.dir/ActivePyModule.cc.o.d -o src/mgr/CMakeFiles/ceph-mgr.dir/ActivePyModule.cc.o -c /builddir/build/BUILD/ceph-16.2.7/src/mgr/ActivePyModule.cc
FAILED: src/mgr/CMakeFiles/ceph-mgr.dir/ActivePyModule.cc.o 
/usr/bin/g++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -I/builddir/build/BUILD/ceph-16.2.7/build/redhat-linux-build/src/include -I/builddir/build/BUILD/ceph-16.2.7/src -isystem /builddir/build/BUILD/ceph-16.2.7/build/redhat-linux-build/include -isystem /builddir/build/BUILD/ceph-16.2.7/src/xxHash -isystem /builddir/build/BUILD/ceph-16.2.7/src/rapidjson/include -isystem /usr/include/python3.10 -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -O2 -g -DNDEBUG -fPIE   -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -std=c++17 -MD -MT src/mgr/CMakeFiles/ceph-mgr.dir/ActivePyModule.cc.o -MF src/mgr/CMakeFiles/ceph-mgr.dir/ActivePyModule.cc.o.d -o src/mgr/CMakeFiles/ceph-mgr.dir/ActivePyModule.cc.o -c /builddir/build/BUILD/ceph-16.2.7/src/mgr/ActivePyModule.cc
In file included from /usr/include/boost/integer.hpp:20,
                 from /usr/include/boost/integer/integer_mask.hpp:16,
                 from /usr/include/boost/random/mersenne_twister.hpp:26,
                 from /usr/include/boost/uuid/random_generator.hpp:17,
                 from /usr/include/boost/uuid/uuid_generators.hpp:17,
                 from /builddir/build/BUILD/ceph-16.2.7/src/include/uuid.h:16,
                 from /builddir/build/BUILD/ceph-16.2.7/src/include/types.h:21,
                 from /builddir/build/BUILD/ceph-16.2.7/src/msg/msg_types.h:23,
                 from /builddir/build/BUILD/ceph-16.2.7/src/common/ceph_context.h:36,
                 from /builddir/build/BUILD/ceph-16.2.7/src/common/dout.h:29,
                 from /builddir/build/BUILD/ceph-16.2.7/src/common/debug.h:18,
                 from /builddir/build/BUILD/ceph-16.2.7/src/mgr/ActivePyModule.cc:16:
/usr/include/boost/integer_traits.hpp:83:64: error: narrowing conversion of '255' from 'int' to 'char' [-Wnarrowing]
   83 |     public detail::integer_traits_base<char, CHAR_MIN, CHAR_MAX>
      |                                                                ^




(I have not tried to investigate why this is not an issue with gcc-11.)

see https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ZRYPYA46DJCASRCA4OTUNRPNQT7U6MQX/


Version-Release number of selected component (if applicable):

3.10

How reproducible:

build ceph on f36/rawhide with gcc-12

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Miro Hrončok 2022-01-21 15:11:36 UTC
This seems to come from the configure.ac AC_C_CHAR_UNSIGNED macro that has been there since Python 1.2. Previously, it was a configure.in AC_CHAR_UNSIGNED macro since at least Python 1.0.

I am not quite sure what's wrong with that or what should be done to avoid that.

Comment 2 Miro Hrončok 2022-01-21 15:14:02 UTC
https://mail.gnu.org/archive/html/autoconf/2008-06/msg00056.html

I guess we could just drop AC_C_CHAR_UNSIGNED entirely?

Comment 3 Jakub Jelinek 2022-01-21 15:18:49 UTC
For GCC/clang/ICC certainly, e.g. GCC 3.2 certainly predefines it and so does current version of clang.
But even if you keep it, you should filter it out from the pyconfig headers that are installed, because it overrides changes from the command line when using those headers.

Comment 4 Jakub Jelinek 2022-01-21 15:22:10 UTC
Arguably, it might be a good idea if autoconf for it checks whether the specified compiler supports -fsigned-char and -funsigned-char options and whether
__CHAR_UNSIGNED__ is predefined in the latter and not predefined in the former case and whether for the default case it matches the auto-detected case,
and if so, leave the macro undefined because the compiler handles it, so there is no point overriding it explicitly.

Comment 5 Miro Hrončok 2022-01-21 15:41:48 UTC
> For GCC/clang/ICC certainly, e.g. GCC 3.2 certainly predefines it and so does current version of clang.

Well, I guess we can submit a dirty downstream-only modification if this is indeed urgent. But I think this needs to be fixed in autotools.

Comment 6 Victor Stinner 2022-01-21 16:19:17 UTC
On my Fedora 35, /usr/include/python3.10/pyconfig-64.h contains:
---
/* Define to 1 if type `char' is unsigned and you are not using gcc.  */
#ifndef __CHAR_UNSIGNED__
/* # undef __CHAR_UNSIGNED__ */
#endif
---

I see that you build C++ code using -fsigned-char.

I didn't know -fsigned-char and -funsigned-char options of GCC.

I don't know if Python itself can be built with char declared as unsigned (-funsigned-char). I suppose that most code expects 'char' type to be signed.

But the public Python C API should not be affected by these options.

> https://mail.gnu.org/archive/html/autoconf/2008-06/msg00056.html

Q: "What's the point of the AC_C_CHAR_UNSIGNED macro?"
A: "If the C type `char' is unsigned, define `__CHAR_UNSIGNED__', unless the C compiler predefines it."

The __CHAR_UNSIGNED__ macro is used in a single C file in Python, Modules/audioop.c.
---
#if defined(__CHAR_UNSIGNED__)
#if defined(signed)
/* This module currently does not work on systems where only unsigned
   characters are available.  Take it out of Setup.  Sorry. */
#endif
#endif
---

audioop.c prefers "signed char" and "unsigned char" over "char". Maybe it should even use int8_t and uint8_t instead to avoid any risk of mistake.

Since Python doesn't use __CHAR_UNSIGNED__ (audioop.c can be fixed, if needed), IMO "AC_C_CHAR_UNSIGNED" can be removed from configure.ac, and so from pyconfig.h.

Comment 7 Kaleb KEITHLEY 2022-01-21 16:26:00 UTC
(In reply to Victor Stinner from comment #6)
> On my Fedora 35, /usr/include/python3.10/pyconfig-64.h contains:
> ---
> /* Define to 1 if type `char' is unsigned and you are not using gcc.  */
> #ifndef __CHAR_UNSIGNED__
> /* # undef __CHAR_UNSIGNED__ */
> #endif
> ---

On x86_64.

On aarch64, ppc64le, and s390x it's:
...
/* Define to 1 if type `char' is unsigned and your compiler does not
   predefine this macro.  */
#ifndef __CHAR_UNSIGNED__
# define __CHAR_UNSIGNED__ 1
#endif
...

Comment 8 Frédéric Bérat 2022-01-24 10:04:08 UTC
(In reply to Jakub Jelinek from comment #4)
> Arguably, it might be a good idea if autoconf for it checks whether the
> specified compiler supports -fsigned-char and -funsigned-char options and
> whether
> __CHAR_UNSIGNED__ is predefined in the latter and not predefined in the
> former case and whether for the default case it matches the auto-detected
> case,
> and if so, leave the macro undefined because the compiler handles it, so
> there is no point overriding it explicitly.

That looks fairly complicated for a macro that is documented as unnecessary by the autotools doc.
They considered obsoleting it 14 years ago, but didn't do so for various reasons, and preferred marking it as "basically, don't use it".

I'll see if I can do something about this, but I'd recommend to get rid of it on your side if you can do so.

Comment 10 Miro Hrončok 2022-01-24 13:47:08 UTC
https://github.com/python/cpython/pull/30851

Comment 11 Jakub Jelinek 2022-01-24 13:53:16 UTC
I'll note that the whole idea of pyconfig*.h as installed header is problematic.
The thing is, various packages that use autoconf will have various similar macros in their own config.h and when they also include pyconfig*.h, that can cause a lot of issues.
You should look what e.g. libstdc++ does; it uses autoconf, but the autoconf generated header is then adjusted using sed, so that all the autoconf emitted
macros are in a namespace specific to the library:
/usr/include/c++/*/*/bits/c++config.h
It would be best if python did the same, picked some prefix and used that consistently.

Comment 12 Victor Stinner 2022-01-25 18:38:20 UTC
The issue is going to be fixed upstream: https://bugs.python.org/issue46513

I don't see the need to track this issue downstream in Fedora, so I close the issue.

Comment 13 Miro Hrončok 2022-01-25 20:13:54 UTC
My assumption was that this blocks the reporter, as it was opened with "Severity: urgent", and we need to backport the fix as soon as available.

Comment 14 Victor Stinner 2022-01-26 13:58:33 UTC
Miro:
> My assumption was that this blocks the reporter, as it was opened with "Severity: urgent", and we need to backport the fix as soon as available.

Oh sorry, I didn't notice. It makes sense to keep this issue open in this case ;-)

Comment 15 Tomáš Hrnčiar 2022-02-01 13:31:24 UTC
PR: https://src.fedoraproject.org/rpms/python3.10/pull-request/95

Comment 16 Fedora Update System 2022-02-02 09:22:40 UTC
FEDORA-2022-077731e655 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-077731e655

Comment 17 Fedora Update System 2022-02-02 09:24:32 UTC
FEDORA-2022-077731e655 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.