Bug 1900529

Summary: python-pystemd fails to build with Python 3.10: pystemd/dbuslib.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send (uses pregenerated Cython code)
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: python-pystemdAssignee: Neal Gompa <ngompa13>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: davide, mhroncok, michel, ngompa13, thrnciar
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-pystemd-0.9.0-1.fc35 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-07 20:48:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1890881, 1927309, 1927313    

Description Miro Hrončok 2020-11-23 09:51:56 UTC
python-pystemd fails to build with Python 3.10.0a2.

ImportError: Failed to import test module: test_base
Traceback (most recent call last):
  File "/usr/lib64/python3.10/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/lib64/python3.10/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/builddir/build/BUILD/pystemd-0.8.0/tests/test_base.py", line 14, in <module>
    from pystemd.base import SDObject
  File "/builddir/build/BUILDROOT/python-pystemd-0.8.0-1.fc34.x86_64/usr/lib64/python3.10/site-packages/pystemd/__init__.py", line 102, in <module>
    from pystemd import DBus, __version__, machine1, systemd1
  File "/builddir/build/BUILDROOT/python-pystemd-0.8.0-1.fc34.x86_64/usr/lib64/python3.10/site-packages/pystemd/DBus/__init__.py", line 9, in <module>
    from pystemd.base import SDObject
  File "/builddir/build/BUILDROOT/python-pystemd-0.8.0-1.fc34.x86_64/usr/lib64/python3.10/site-packages/pystemd/base.py", line 13, in <module>
    from pystemd.dbuslib import DBus, apply_signature
ImportError: /builddir/build/BUILDROOT/python-pystemd-0.8.0-1.fc34.x86_64/usr/lib64/python3.10/site-packages/pystemd/dbuslib.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send


Is it possible that the extension module is built from pre-generated Cython sources? That is usually the cause:

https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_packages_using_cython


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/01775038-python-pystemd/

For all our attempts to build python-pystemd with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/python-pystemd/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2020-11-23 10:49:36 UTC
$ rg Generated | grep Cython
pystemd-0.8.0/pystemd/dbusexc.c:/* Generated by Cython 0.29.21 */
pystemd-0.8.0/pystemd/journal.c:/* Generated by Cython 0.29.21 */
pystemd-0.8.0/pystemd/daemon.c:/* Generated by Cython 0.29.21 */
pystemd-0.8.0/pystemd/dbuslib.c:/* Generated by Cython 0.29.21 */


However, when I delete them in %prep, I get:

+ /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s'

Error compiling Cython file:
------------------------------------------------------------
...
# This source code is licensed under the license found in the LICENSE file in
# the root directory of this source tree.
#


cimport pystemd.dbusc as dbusc
       ^
------------------------------------------------------------

pystemd/daemon.pyx:11:8: 'pystemd/dbusc.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

from libc.stdint cimport uint64_t
from pystemd.utils import x2char_star


LISTEN_FDS_START = dbusc.SD_LISTEN_FDS_START
                       ^
------------------------------------------------------------

pystemd/daemon.pyx:17:24: cimported module has no attribute 'SD_LISTEN_FDS_START'

Error compiling Cython file:
------------------------------------------------------------
...
class PystemdDaemonError(Exception):
  "generic daemon error"


def listen_fds(int unset_environment=0):
  return dbusc.sd_listen_fds(unset_environment)
             ^
------------------------------------------------------------

pystemd/daemon.pyx:24:14: cimported module has no attribute 'sd_listen_fds'

Error compiling Cython file:
------------------------------------------------------------
...
  pystates.extend(
    b'='.join([x2char_star(k.upper()), x2char_star(v, convert_all=True)])
    for k, v in kwstates.items()
  )
  state = b'\n'.join(pystates)
  return dbusc.sd_notify(unset_environment, state)
             ^
------------------------------------------------------------

pystemd/daemon.pyx:58:14: cimported module has no attribute 'sd_notify'

Error compiling Cython file:
------------------------------------------------------------
...
  return dbusc.sd_notify(unset_environment, state)


def booted():
  "Returns True if system was booted with systemd"
  cdef int status = dbusc.sd_booted()
                        ^
------------------------------------------------------------

pystemd/daemon.pyx:63:25: cimported module has no attribute 'sd_booted'

Error compiling Cython file:
------------------------------------------------------------
...

  cdef:
    int result
    uint64_t usec

  result = dbusc.sd_watchdog_enabled(unset_environment, &usec)
               ^
------------------------------------------------------------

pystemd/daemon.pyx:80:16: cimported module has no attribute 'sd_watchdog_enabled'
Compiling pystemd/daemon.pyx because it changed.
Compiling pystemd/dbusexc.pyx because it changed.
Compiling pystemd/dbuslib.pyx because it changed.
Compiling pystemd/journal.pyx because it changed.
[1/4] Cythonizing pystemd/daemon.pyx
Traceback (most recent call last):
  File "/builddir/build/BUILD/pystemd-0.8.0/setup.py", line 71, in <module>
    external_modules = cythonize(
  File "/usr/lib64/python3.9/site-packages/Cython/Build/Dependencies.py", line 1102, in cythonize
    cythonize_one(*args)
  File "/usr/lib64/python3.9/site-packages/Cython/Build/Dependencies.py", line 1225, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: pystemd/daemon.pyx

Comment 2 Ben Cotton 2021-02-09 15:27:09 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Comment 3 Miro Hrončok 2021-06-04 20:14:43 UTC
This is a mass-posted update. Sorry if it is not 100% accurate to this bugzilla.


The Python 3.10 rebuild is in progress in a Koji side tag. If you manage to fix the problem, please commit the fix in the rawhide branch, but don't build the package in regular rawhide.

You can either build the package in the side tag, with:

    $ fedpkg build --target=f35-python

Or you can the build and we will eventually build it for you.

Note that the rebuild is still in progress, so not all (build) dependencies of this package might be available right away.

Thanks.

See also https://fedoraproject.org/wiki/Changes/Python3.10

If you have general questions about the rebuild, please use this mailing list thread: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/G47SGOYIQLRDTWGOSLSWERZSSHXDEDH5/

Comment 4 Miro Hrončok 2021-06-07 22:59:18 UTC
The f35-python side tag has been merged to Rawhide. From now on, build as you would normally build.

Comment 5 Miro Hrončok 2021-06-08 11:31:20 UTC
*** Bug 1969108 has been marked as a duplicate of this bug. ***

Comment 6 Miro Hrončok 2021-06-15 20:23:17 UTC
Hello,

This is the first reminder (step 3 from https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/#_package_removal_for_long_standing_ftbfs_and_fti_bugs).

If you know about this problem and are planning on fixing it, please acknowledge so by setting the bug status to ASSIGNED. If you don't have time to maintain this package, consider orphaning it, so maintainers of dependent packages realize the problem.

Comment 7 Miro Hrončok 2021-07-07 11:18:33 UTC
Hello,

This is the second reminder (step 4 from https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/#_package_removal_for_long_standing_ftbfs_and_fti_bugs).

If you know about this problem and are planning on fixing it, please acknowledge so by setting the bug status to ASSIGNED. If you don't have time to maintain this package, consider orphaning it, so maintainers of dependent packages realize the problem.

Comment 8 Neal Gompa 2021-07-07 11:28:35 UTC
Davide, Michel, do either of you know if this has been addressed upstream yet?

Comment 9 Davide Cavalca 2021-07-07 17:39:09 UTC
We're fixing this upstream, the release tarball doesn't include the necessary bits to regenerate the bindings, and the pregenerated bindings won't work as they are with 3.10.

Comment 10 Davide Cavalca 2021-07-07 20:27:33 UTC
https://src.fedoraproject.org/rpms/python-pystemd/pull-request/1 should do the trick here

Comment 11 Fedora Update System 2021-07-07 20:47:33 UTC
FEDORA-2021-b5bc0af106 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-b5bc0af106

Comment 12 Fedora Update System 2021-07-07 20:48:55 UTC
FEDORA-2021-b5bc0af106 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.