Bug 2255342 - python-twisted fails to build with Python 3.13: ModuleNotFoundError: No module named 'cgi' & many other failures
Summary: python-twisted fails to build with Python 3.13: ModuleNotFoundError: No modul...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-twisted
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jonathan Steffan
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 2316869
Blocks: PYTHON3.13
TreeView+ depends on / blocked
 
Reported: 2023-12-20 08:50 UTC by Karolina Surma
Modified: 2025-01-17 16:42 UTC (History)
6 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2025-01-17 16:42:13 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github twisted twisted issues 11848 0 None open DeprecationWarning: "'cgi' is deprecated and slated for removal in Python 3.13" 2023-12-20 08:50:35 UTC
Github twisted twisted issues 12052 0 None closed twisted tests failing on main on github actions with Python 3.12.1 2023-12-20 09:36:27 UTC
Github twisted twisted issues 12098 0 None open Python 3.13: test_flatten fails because of the missing traceback poiniting characters 2024-01-30 14:28:26 UTC
Github twisted twisted issues 12099 0 None open Python 3.13: test_findFailureInGenerator and _test_urlParameters_1 fail 2024-01-30 14:28:26 UTC

Description Karolina Surma 2023-12-20 08:50:21 UTC
python-twisted fails to build with Python 3.13.0a2.

Test suite result:
FAILED (skips=640, failures=14, errors=51, successes=9085)

See the build logs for the full results:

https://copr-be.cloud.fedoraproject.org/results/@python/python3.13/fedora-rawhide-x86_64/06770350-python-twisted/

According to https://docs.python.org/3.13/whatsnew/3.13.html module cgi was removed:

PEP 594: Remove the cgi and cgitb modules, deprecated in Python 3.11.

cgi.FieldStorage can typically be replaced with urllib.parse.parse_qsl() for GET and HEAD requests, and the email.message module or multipart PyPI project for POST and PUT.

cgi.parse() can be replaced by calling urllib.parse.parse_qs() directly on the desired query string, except for multipart/form-data input, which can be handled as described for cgi.parse_multipart().

cgi.parse_multipart() can be replaced with the functionality in the email package (e.g. email.message.EmailMessage and email.message.Message) which implements the same MIME RFCs, or with the multipart PyPI project.

cgi.parse_header() can be replaced with the functionality in the email package, which implements the same MIME RFCs. For example, with email.message.EmailMessage:

from email.message import EmailMessage
msg = EmailMessage()
msg['content-type'] = 'application/json; charset="utf8"'
main, params = msg.get_content_type(), msg['content-type'].params

(Contributed by Victor Stinner in gh-104773.)


https://docs.python.org/3.13/whatsnew/3.13.html

For all our attempts to build python-twisted with Python 3.13, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.13/package/python-twisted/

Testing and mass rebuild of packages is happening in copr.
You can follow these instructions to test locally in mock if your package builds with Python 3.13:
https://copr.fedorainfracloud.org/coprs/g/python/python3.13/

Let us know here if you have any questions.

Python 3.13 is planned to be included in Fedora 41.
To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.13.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires),
so if this package is required a lot, it's important for us to get it fixed soon.

We'd appreciate help from the people who know this package best,
but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2023-12-20 09:36:28 UTC
This is also affected by https://github.com/twisted/twisted/issues/12052 which also impacts builds with Python 3.12.1

https://koschei.fedoraproject.org/package/python-twisted shows
FAILED (skips=689, failures=2, errors=8, successes=10369)

Comment 2 Miro Hrončok 2023-12-20 12:03:48 UTC
(In reply to Miro Hrončok from comment #1)
> This is also affected by https://github.com/twisted/twisted/issues/12052
> which also impacts builds with Python 3.12.1
> 
> https://koschei.fedoraproject.org/package/python-twisted shows
> FAILED (skips=689, failures=2, errors=8, successes=10369)

This part should be fixed via https://src.fedoraproject.org/rpms/python-twisted/pull-request/22

Comment 3 Miro Hrončok 2023-12-20 12:13:57 UTC
With https://src.fedoraproject.org/rpms/python-twisted/pull-request/22 I get this with Python 3.13:

FAILED (skips=640, failures=12, errors=49, successes=9087)

29 of this is builtins.ModuleNotFoundError: No module named 'cgi' -> should be fixed by https://github.com/twisted/twisted/pull/12058

Comment 4 Karolina Surma 2024-01-16 10:16:21 UTC
With cgi fixes backported in https://src.fedoraproject.org/rpms/python-twisted/pull-request/23 it's better, but not completely:

FAILED (skips=709, failures=13, errors=20, successes=10324)

Errors come from:

[ERROR]
Traceback (most recent call last):
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/web/test/test_html.py", line 32, in test_deprecation
    assertDeprecationWarningOf("PRE")
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/web/test/test_html.py", line 22, in assertDeprecationWarningOf
    warningsShown = self.flushWarnings([self.test_deprecation])
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/trial/_synctest.py", line 1196, in flushWarnings
    if not (min(lineNumbers) <= aWarning.lineno <= max(lineNumbers)):
builtins.TypeError: '<' not supported between instances of 'int' and 'NoneType'

twisted.web.test.test_html.WebHtmlTests.test_deprecation

Failures are more diverse, but 5 of them comes from the stripped whitespace characters.

Comment 5 Karolina Surma 2024-01-16 13:58:50 UTC
With another added backport in the above linked PR, we're down to 10 failures. The non-whitespace-related ones are below:

===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/test/test_inlinecb.py", line 192, in test_forwardLotsOfTracebacks
    self.assertIn("in erroring", tb)
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/trial/_synctest.py", line 509, in assertIn
    raise self.failureException(msg or f"{containee!r} not in {container!r}")
twisted.trial.unittest.FailTest: 'in erroring' not in 'Traceback (most recent call last):\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/python/util.py", line 958, in runWithWarningsSuppressed\n    return f(*args, **kwargs)\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/test/test_inlinecb.py", line 189, in test_forwardLotsOfTracebacks\n    d = calling()\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/defer.py", line 2256, in unwindGenerator\n    return _cancellableInlineCallbacks(gen)\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/defer.py", line 2168, in _cancellableInlineCallbacks\n    _inlineCallbacks(None, gen, status, _copy_context())\n--- <exception caught here> ---\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/defer.py", line 1996, in _inlineCallbacks\n    result = context.run(\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/python/failure.py", line 519, in throwExceptionIntoGenerator\n    return g.throw(self.value.with_traceback(self.tb))\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/test/test_inlinecb.py", line 187, in calling\n    yield calling2()\nbuiltins.Exception: Error Marker\n'

twisted.internet.test.test_inlinecb.ForwardTraceBackTests.test_forwardLotsOfTracebacks
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/test/test_inlinecb.py", line 149, in test_forwardTracebacks
    self.assertIn("in erroring", tb)
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/trial/_synctest.py", line 509, in assertIn
    raise self.failureException(msg or f"{containee!r} not in {container!r}")
twisted.trial.unittest.FailTest: 'in erroring' not in 'Traceback (most recent call last):\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/python/util.py", line 958, in runWithWarningsSuppressed\n    return f(*args, **kwargs)\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/test/test_inlinecb.py", line 146, in test_forwardTracebacks\n    d = calling()\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/defer.py", line 2256, in unwindGenerator\n    return _cancellableInlineCallbacks(gen)\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/defer.py", line 2168, in _cancellableInlineCallbacks\n    _inlineCallbacks(None, gen, status, _copy_context())\n--- <exception caught here> ---\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/defer.py", line 1996, in _inlineCallbacks\n    result = context.run(\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/python/failure.py", line 519, in throwExceptionIntoGenerator\n    return g.throw(self.value.with_traceback(self.tb))\n  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/internet/test/test_inlinecb.py", line 144, in calling\n    yield erroring()\nbuiltins.Exception: Error Marker\n'

twisted.internet.test.test_inlinecb.ForwardTraceBackTests.test_forwardTracebacks
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/python/test/test_shellcomp.py", line 362, in test_poorlyDescribedOptMethod
    self.assertEqual(descr, "silly")
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/trial/_synctest.py", line 444, in assertEqual
    super().assertEqual(first, second, msg)
  File "/usr/lib64/python3.13/unittest/case.py", line 887, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib64/python3.13/unittest/case.py", line 1253, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))
twisted.trial.unittest.FailTest: '' != 'silly'
+ silly


twisted.python.test.test_shellcomp.ZshTests.test_poorlyDescribedOptMethod
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/test/test_failure.py", line 967, in test_findFailureInGenerator
    self.assertEqual(foundFailures, [f])
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/trial/_synctest.py", line 444, in assertEqual
    super().assertEqual(first, second, msg)
  File "/usr/lib64/python3.13/unittest/case.py", line 887, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib64/python3.13/unittest/case.py", line 1093, in assertListEqual
    self.assertSequenceEqual(list1, list2, msg, seq_type=list)
  File "/usr/lib64/python3.13/unittest/case.py", line 1075, in assertSequenceEqual
    self.fail(msg)
twisted.trial.unittest.FailTest: Lists differ: [None] != [<twisted.python.failure.Failure builtins.ZeroDivisionError: division by zero>]

First differing element 0:
None
<twisted.python.failure.Failure builtins.ZeroDivisionError: division by zero>

- [None]
+ [<twisted.python.failure.Failure builtins.ZeroDivisionError: division by zero>]

twisted.test.test_failure.ExtendedGeneratorTests.test_findFailureInGenerator
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/web/test/test_cgi.py", line 389, in _test_urlParameters_1
    self.assertEqual(res, expected)
  File "/builddir/build/BUILD/twisted-twisted-23.10.0/src/twisted/trial/_synctest.py", line 444, in assertEqual
    super().assertEqual(first, second, msg)
  File "/usr/lib64/python3.13/unittest/case.py", line 887, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib64/python3.13/unittest/case.py", line 880, in _baseAssertEqual
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: b'\n<html>\n  <head><title>500 - CGI Script[125 chars]l>\n' != b'1234\n'

twisted.web.test.test_cgi.CGIScriptTests.test_urlParameters

Comment 6 Karolina Surma 2024-01-30 14:28:27 UTC
With some more upstream fixes included into the package, three tests fail: twisted.test.test_failure.ExtendedGeneratorTests.test_findFailureInGenerator, twisted.web.test.test_cgi.CGIScriptTests.test_urlParameters and test_flatten (related to the changed traceback output). Reported upstream.

Comment 7 Aoife Moloney 2024-02-15 23:08:11 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 40 development cycle.
Changing version to 40.


Note You need to log in before you can comment on or make changes to this bug.