Bug 2331375 - python-asttokens fails to build with Python 3.14: AttributeError: module 'ast' has no attribute 'Str'
Summary: python-asttokens fails to build with Python 3.14: AttributeError: module 'ast...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-asttokens
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Zbigniew Jędrzejewski-Szmek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.14
TreeView+ depends on / blocked
 
Reported: 2024-12-10 14:03 UTC by Karolina Surma
Modified: 2025-01-07 15:22 UTC (History)
5 users (show)

Fixed In Version: python-asttokens-2.4.1-8.fc42
Clone Of:
Environment:
Last Closed: 2025-01-07 15:22:56 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github gristlabs asttokens pull 157 0 None open Replace ast.Str usages with ast.Constant for compatibility with Python 3.14 2025-01-07 09:43:11 UTC

Description Karolina Surma 2024-12-10 14:03:51 UTC
python-asttokens fails to build with Python 3.14.0a2.

______________________ TestASTTokens.test_unicode_offsets ______________________

self = <tests.test_asttokens.TestASTTokens testMethod=test_unicode_offsets>

    def test_unicode_offsets(self):
      # ast modules provides utf8 offsets, while tokenize uses unicode offsets. Make sure we
      # translate correctly.
      source = "foo('фыва',a,b)\n"
      atok = asttokens.ASTTokens(source)
      self.assertEqual([six.text_type(t) for t in atok.tokens], [
        "NAME:'foo'",
        "OP:'('",
        'STRING:"%s"' % repr('фыва').lstrip('u'),
        "OP:','",
        "NAME:'a'",
        "OP:','",
        "NAME:'b'",
        "OP:')'",
        "NEWLINE:'\\n'",
        "ENDMARKER:''"
      ])
      self.assertEqual(atok.tokens[2].startpos, 4)
      self.assertEqual(atok.tokens[2].endpos, 10)      # Counting characters, not bytes
      self.assertEqual(atok.tokens[4].startpos, 11)
      self.assertEqual(atok.tokens[4].endpos, 12)
      self.assertEqual(atok.tokens[6].startpos, 13)
      self.assertEqual(atok.tokens[6].endpos, 14)
    
      root = ast.parse(source)
    
      # Verify that ast parser produces offsets as we expect. This is just to inform the
      # implementation.
>     string_node = next(n for n in ast.walk(root) if isinstance(n, ast.Str))

tests/test_asttokens.py:130: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

.0 = <generator object walk at 0x7f39ea817970>

>   string_node = next(n for n in ast.walk(root) if isinstance(n, ast.Str))
E   AttributeError: module 'ast' has no attribute 'Str'

tests/test_asttokens.py:130: AttributeError
____________________________ TestUtil.test_replace _____________________________

self = <tests.test_util.TestUtil testMethod=test_replace>

    def test_replace(self):
      self.assertEqual(asttokens.util.replace("this is a test", [(0, 4, "X"), (8, 9, "THE")]),
                       "X is THE test")
      self.assertEqual(asttokens.util.replace("this is a test", []), "this is a test")
      self.assertEqual(asttokens.util.replace("this is a test", [(7,7," NOT")]), "this is NOT a test")
    
      source = "foo(bar(1 + 2), 'hello' + ', ' + 'world')"
      atok = asttokens.ASTTokens(source, parse=True)
      names = [n for n in asttokens.util.walk(atok.tree) if isinstance(n, ast.Name)]
>     strings = [n for n in asttokens.util.walk(atok.tree) if isinstance(n, ast.Str)]
E     AttributeError: module 'ast' has no attribute 'Str'

tests/test_util.py:104: AttributeError
=========================== short test summary info ============================
FAILED tests/test_asttokens.py::TestASTTokens::test_unicode_offsets - Attribu...
FAILED tests/test_util.py::TestUtil::test_replace - AttributeError: module 'a...

According to: https://docs.python.org/dev/whatsnew/3.14.html#id2

Remove the following classes. They were all deprecated since Python 3.8, and have emitted deprecation warnings since Python 3.12:
ast.Bytes
ast.Ellipsis
ast.NameConstant
ast.Num
ast.Str

Use ast.Constant instead. As a consequence of these removals, user-defined visit_Num, visit_Str, visit_Bytes, visit_NameConstant and visit_Ellipsis methods on custom ast.NodeVisitor subclasses will no longer be called when the NodeVisitor subclass is visiting an AST. Define a visit_Constant method instead.

Also, remove the following deprecated properties on ast.Constant, which were present for compatibility with the now-removed AST classes:
ast.Constant.n
ast.Constant.s

Use ast.Constant.value instead.
(Contributed by Alex Waygood in gh-119562.)


https://docs.python.org/3.14/whatsnew/3.14.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.14/fedora-rawhide-x86_64/08372621-python-asttokens/

For all our attempts to build python-asttokens with Python 3.14, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.14/package/python-asttokens/

Testing and mass rebuild of packages is happening in copr.
You can follow these instructions to test locally in mock if your package builds with Python 3.14:
https://copr.fedorainfracloud.org/coprs/g/python/python3.14/

Let us know here if you have any questions.

Python 3.14 is planned to be included in Fedora 43.
To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.14.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires),
so if this package is required a lot, it's important for us to get it fixed soon.

We'd appreciate help from the people who know this package best,
but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Fedora Update System 2025-01-07 15:19:46 UTC
FEDORA-2025-be3a212c93 (python-asttokens-2.4.1-8.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-be3a212c93

Comment 2 Fedora Update System 2025-01-07 15:22:56 UTC
FEDORA-2025-be3a212c93 (python-asttokens-2.4.1-8.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.