Bug 2019843 - python-pexpect fails to build with Python 3.11: AttributeError: module 'asyncio' has no attribute 'coroutine'
Summary: python-pexpect fails to build with Python 3.11: AttributeError: module 'async...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pexpect
Version: 36
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Andrew McNabb
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F36FTBFS PYTHON3.11 F37FTBFS 2068014
TreeView+ depends on / blocked
 
Reported: 2021-11-03 12:57 UTC by Tomáš Hrnčiar
Modified: 2022-03-25 14:01 UTC (History)
10 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2022-03-24 23:52:43 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github pexpect pexpect issues 677 0 None open Replace deprecated `@asyncio.coroutine` with `async`/`await` 2021-11-03 18:43:31 UTC
Github pexpect pexpect pull 715 0 None open Convert @asyncio.coroutine to async def 2022-03-24 14:18:04 UTC

Description Tomáš Hrnčiar 2021-11-03 12:57:57 UTC
python-pexpect fails to build with Python 3.11.0a1.

=================================== FAILURES ===================================
_________________________ AsyncTests.test_async_and_gc _________________________

self = <tests.test_async.AsyncTests testMethod=test_async_and_gc>

    def test_async_and_gc(self):
        p = pexpect.spawn('%s sleep_for.py 1' % self.PYTHONBIN, encoding='utf8')
>       assert run(p.expect_exact(u'READY', async_=True)) == 0

tests/test_async.py:63: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/spawnbase.py:418: in expect_exact
    from ._async import expect_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_async_and_gc 
________________________ AsyncTests.test_async_and_sync ________________________

self = <tests.test_async.AsyncTests testMethod=test_async_and_sync>

    def test_async_and_sync(self):
        p = pexpect.spawn('echo 1234', encoding='utf8', maxread=1)
>       assert run(p.expect_exact(u'1', async_=True)) == 0

tests/test_async.py:69: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/spawnbase.py:418: in expect_exact
    from ._async import expect_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_async_and_sync 
________________________ AsyncTests.test_async_replwrap ________________________

self = <tests.test_async.AsyncTests testMethod=test_async_replwrap>

    def test_async_replwrap(self):
        bash = replwrap.bash()
>       coro = bash.run_command("time", async_=True)

tests/test_async.py:75: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/replwrap.py:92: in run_command
    from ._async import repl_run_command_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_async_replwrap 
___________________ AsyncTests.test_async_replwrap_multiline ___________________

self = <tests.test_async.AsyncTests testMethod=test_async_replwrap_multiline>

    def test_async_replwrap_multiline(self):
        bash = replwrap.bash()
>       coro = bash.run_command("echo '1 2\n3 4'", async_=True)

tests/test_async.py:81: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/replwrap.py:92: in run_command
    from ._async import repl_run_command_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_async_replwrap_multiline 
__________________________ AsyncTests.test_async_utf8 __________________________

self = <tests.test_async.AsyncTests testMethod=test_async_utf8>

    def test_async_utf8(self):
        p = pexpect.spawn('%s list100.py' % self.PYTHONBIN, encoding='utf8')
>       assert run(p.expect_exact(u'5', async_=True)) == 0

tests/test_async.py:57: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/spawnbase.py:418: in expect_exact
    from ._async import expect_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_async_utf8 
_____________________________ AsyncTests.test_eof ______________________________

self = <tests.test_async.AsyncTests testMethod=test_eof>

    def test_eof(self):
        p = pexpect.spawn('cat')
        p.sendline('Hi')
>       coro = p.expect(pexpect.EOF, async_=True)

tests/test_async.py:39: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/spawnbase.py:343: in expect
    return self.expect_list(compiled_pattern_list,
pexpect/spawnbase.py:369: in expect_list
    from ._async import expect_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_eof 
_________________________ AsyncTests.test_expect_exact _________________________

self = <tests.test_async.AsyncTests testMethod=test_expect_exact>

    def test_expect_exact(self):
        p = pexpect.spawn('%s list100.py' % self.PYTHONBIN)
>       assert run(p.expect_exact(b'5', async_=True)) == 0

tests/test_async.py:51: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/spawnbase.py:418: in expect_exact
    from ._async import expect_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_expect_exact 
________________________ AsyncTests.test_simple_expect _________________________

self = <tests.test_async.AsyncTests testMethod=test_simple_expect>

    def test_simple_expect(self):
        p = pexpect.spawn('cat')
        p.sendline('Hello asyncio')
>       coro = p.expect(['Hello', pexpect.EOF] , async_=True)

tests/test_async.py:22: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/spawnbase.py:343: in expect
    return self.expect_list(compiled_pattern_list,
pexpect/spawnbase.py:369: in expect_list
    from ._async import expect_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_simple_expect 
___________________________ AsyncTests.test_timeout ____________________________

self = <tests.test_async.AsyncTests testMethod=test_timeout>

    def test_timeout(self):
        p = pexpect.spawn('cat')
>       coro = p.expect('foo', timeout=1, async_=True)

tests/test_async.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pexpect/spawnbase.py:343: in expect
    return self.expect_list(compiled_pattern_list,
pexpect/spawnbase.py:369: in expect_list
    from ._async import expect_async
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import errno
    import signal
    
    from pexpect import EOF
    
>   @asyncio.coroutine
E   AttributeError: module 'asyncio' has no attribute 'coroutine'

pexpect/_async.py:7: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_async.AsyncTests.test_timeout 
_________________________ ExpectTestCase.test_bad_arg __________________________

self = <tests.test_expect.ExpectTestCase testMethod=test_bad_arg>

    def test_bad_arg(self):
        p = pexpect.spawn('cat')
>       with self.assertRaisesRegexp(TypeError, '.*must be one of'):
E       AttributeError: 'ExpectTestCase' object has no attribute 'assertRaisesRegexp'

tests/test_expect.py:572: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_expect.ExpectTestCase.test_bad_arg 
_______________ TestCaseMisc.test_bad_arguments_suggest_fdpsawn ________________

self = <tests.test_misc.TestCaseMisc testMethod=test_bad_arguments_suggest_fdpsawn>

    def test_bad_arguments_suggest_fdpsawn(self):
        " assert custom exception for spawn(int). "
        expect_errmsg = "maybe you want to use fdpexpect.fdspawn"
>       with self.assertRaisesRegexp(pexpect.ExceptionPexpect,
                                     ".*" + expect_errmsg):
E                                    AttributeError: 'TestCaseMisc' object has no attribute 'assertRaisesRegexp'

tests/test_misc.py:227: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_misc.TestCaseMisc.test_bad_arguments_suggest_fdpsawn 
_______________________ TestCaseMisc.test_bad_child_pid ________________________

self = <tests.test_misc.TestCaseMisc testMethod=test_bad_child_pid>

    def test_bad_child_pid(self):
        " assert bad condition error in isalive(). "
        expect_errmsg = re.escape("isalive() encountered condition where ")
        child = pexpect.spawn('cat')
        child.terminate(force=1)
        # Force an invalid state to test isalive
        child.ptyproc.terminated = 0
        try:
>           with self.assertRaisesRegexp(pexpect.ExceptionPexpect,
                                         ".*" + expect_errmsg):
E                                        AttributeError: 'TestCaseMisc' object has no attribute 'assertRaisesRegexp'

tests/test_misc.py:217: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_misc.TestCaseMisc.test_bad_child_pid 
_________________________ ExpectTestCase.test_bad_arg __________________________

self = <tests.test_popen_spawn.ExpectTestCase testMethod=test_bad_arg>

    def test_bad_arg(self):
        p = PopenSpawn('cat')
>       with self.assertRaisesRegexp(TypeError, '.*must be one of'):
E       AttributeError: 'ExpectTestCase' object has no attribute 'assertRaisesRegexp'

tests/test_popen_spawn.py:113: AttributeError
----------------------------- Captured stdout call -----------------------------

 tests.test_popen_spawn.ExpectTestCase.test_bad_arg 

Removed many old deprecated unittest features:
    TestCase method aliases failUnlessEqual, failIfEqual, failUnless, failIf,
    failUnlessRaises, failUnlessAlmostEqual, failIfAlmostEqual (deprecated in
    Python 3.1), assertEquals, assertNotEquals, assert_, assertAlmostEquals,
    assertNotAlmostEquals, assertRegexpMatches, assertRaisesRegexp (deprecated in
    Python 3.2), and assertNotRegexpMatches (deprecated in Python 3.5).

The @asyncio.coroutine decorator enabling legacy generator-based coroutines to be compatible with async/await code. The function has been deprecated since Python 3.8 and the removal was initially scheduled for Python 3.10. Use async def instead. (Contributed by Illia Volochii in bpo-43216.)

https://bugs.python.org/issue45162
https://docs.python.org/3.11/whatsnew/3.11.html


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.11/fedora-rawhide-x86_64/02911231-python-pexpect/

For all our attempts to build python-pexpect with Python 3.11, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/package/python-pexpect/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.11:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/

Let us know here if you have any questions.

Python 3.11 is planned to be included in Fedora 37. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.11.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Ben Cotton 2022-02-08 21:34:43 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 2 Miro Hrončok 2022-02-27 00:12:54 UTC
Many packages need this, hence adjusting the severity.

$ repoquery --repo=rawhide{,-source} --whatrequires python3-pexpect 
ansible-core-0:2.12.2-1.fc36.src
catfish-0:4.16.3-2.fc36.1.noarch
catfish-0:4.16.3-2.fc36.1.src
chess_db-0:0.2-3.20210517giteb41ddf.fc36.src
cinnamon-0:5.2.7-2.fc36.x86_64
coq-0:8.14.1-4.fc36.src
duplicity-0:0.8.21-5.fc36.src
duplicity-0:0.8.21-5.fc36.x86_64
fence-agents-0:4.11.0-3.fc36.src
fence-agents-common-0:4.11.0-3.fc36.noarch
fence-agents-vmware-0:4.11.0-3.fc36.noarch
fence-agents-xenapi-0:4.11.0-3.fc36.noarch
fish-0:3.3.1^1075ge0bc944d5c5-1.fc37.src
fleet-commander-admin-0:0.15.1-5.fc36.noarch
fleet-commander-admin-0:0.15.1-5.fc36.src
hatch-0:0.23.1-3.20220204git804d5fd.fc36.noarch
hatch-0:0.23.1-3.20220204git804d5fd.fc36.src
ipython-0:8.0.1-1.fc36.src
libvirt-test-API-0:1.1-6.fc36.noarch
libvirt-test-API-0:1.1-6.fc36.src
mycli-0:1.24.3-1.fc36.src
pgcli-0:3.1.0-4.fc35.src
poetry-0:1.1.12-2.fc36.src
pychess-0:1.0.3-1.fc37.src
pytest-0:6.2.5-2.fc36.src
python-ansible-runner-0:2.1.1-2.fc36.src
python-argcomplete-0:2.0.0-2.fc36.src
python-bash-kernel-0:0.7.2-3.fc36.src
python-hypothesis-0:6.36.0-1.fc36.src
python-jupyter-polymake-0:0.16-18.20180129.7049940.fc36.src
python-metakernel-0:0.28.2-2.fc36.src
python-pytest-timeout-0:2.1.0-1.fc36.src
python3-ansible-runner-0:2.1.1-2.fc36.noarch
python3-bash-kernel-0:0.7.2-3.fc36.noarch
python3-ipatests-0:4.9.8-3.fc36.noarch
python3-ipython+notebook-0:8.0.1-1.fc36.noarch
python3-ipython-0:8.0.1-1.fc36.noarch
python3-jupyter-polymake-0:0.16-18.20180129.7049940.fc36.noarch
python3-metakernel-0:0.28.2-2.fc36.noarch
python3-poetry-0:1.1.12-2.fc36.noarch
python3-pygatt-0:4.0.5-8.fc36.noarch
python3-spyder-0:5.2.1-2.fc36.noarch
reptyr-0:0.8.0-4.fc36.src
rr-0:5.5.0-3.fc36.src
rr-testsuite-0:5.5.0-3.fc36.x86_64
sagemath-0:9.4-3.fc36.src
sagemath-core-0:9.4-3.fc36.x86_64
scoutfish-0:1.1-3.20210517gitb619262.fc36.src
sigul-0:1.1-4.fc35.src
sigul-server-0:1.1-4.fc35.x86_64

Andrew, will you be able to fix this?

Comment 3 Scott Talbert 2022-03-23 23:33:31 UTC
Miro, I think @amcnabb may be unresponsive.  I don't see any commits in 8+ years.

Comment 4 Miro Hrončok 2022-03-24 10:37:53 UTC
(In reply to Scott Talbert from comment #3)
> Miro, I think @amcnabb may be unresponsive.  I don't see any commits in 8+
> years.

I've started bz2068014.

In the meantime, I see you are a co-maintainer, would you be able to fix this, or do you need some pointers?

Comment 5 Scott Talbert 2022-03-24 13:00:05 UTC
(In reply to Miro Hrončok from comment #4)
> (In reply to Scott Talbert from comment #3)
> > Miro, I think @amcnabb may be unresponsive.  I don't see any commits in 8+
> > years.
> 
> I've started bz2068014.
> 
> In the meantime, I see you are a co-maintainer, would you be able to fix
> this, or do you need some pointers?

I have looked at it previously, but it wasn't clear to me how to replace @asyncio with async/await.  It doesn't seem like they are exact one-for-one replacements, and I didn't want to introduce any new bugs due to my lack of understanding.  I was hoping upstream would fix this, but it seems like upstream is somewhat stalled.

Comment 6 Miro Hrončok 2022-03-24 13:51:39 UTC
As far as I know, it should be as easy as:

Before:

  @asyncio.coroutine
  def expect_async(expecter, timeout=None):

After:

  async def expect_async(expecter, timeout=None):




They should be exact one-for-one replacements, but I am not 100% sure.

Comment 7 Miro Hrončok 2022-03-24 14:03:22 UTC
No, not that simple :)


SyntaxError: 'return' with value in async generator

Comment 8 Miro Hrončok 2022-03-24 14:14:33 UTC
yield from -> await does it

testing mockbuilds, will submit an upstream PR you can take

Comment 9 Miro Hrončok 2022-03-24 14:18:05 UTC
https://github.com/pexpect/pexpect/pull/715

Comment 10 Scott Talbert 2022-03-24 23:52:43 UTC
Thanks, Miro - I didn't realize that was all it needed.  Applied your patch in Rawhide.

Comment 11 Miro Hrončok 2022-03-25 14:01:28 UTC
Thanks, Scott.

PS After talking to @amcnabb, I've made you the main maintainer of the package.


Note You need to log in before you can comment on or make changes to this bug.