Bug 1728063 - pew fails to build with Python 3.8 (too old python-virtualenv-clone)
Summary: pew fails to build with Python 3.8 (too old python-virtualenv-clone)
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-virtualenv-clone
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Fedora Infrastructure SIG
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON38
TreeView+ depends on / blocked
 
Reported: 2019-07-08 23:20 UTC by Miro Hrončok
Modified: 2019-08-14 08:51 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2019-08-14 08:51:44 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Miro Hrončok 2019-07-08 23:20:26 UTC
pew fails to build with Python 3.8.0b1.

_________________________ test_source_does_not_exists __________________________

workon_home = PosixPath('/tmp/WORKON_HOME')

    def test_source_does_not_exists(workon_home):
        err = invoke('cp', 'virtualenvthatdoesntexist', 'foo').err.strip()
>       assert 'Please provide a valid virtualenv to copy' == err
E       AssertionError: assert 'Please provi...alenv to copy' == '/usr/lib/pyth...alenv to copy'
E         + /usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \.
E         +   pybinre = re.compile('pythonw?([0-9]+(\.[0-9]+(\.[0-9]+)?)?)?$')
E           Please provide a valid virtualenv to copy

tests/test_cp.py:50: AssertionError
_______________________________ test_getproject ________________________________

env1 = None

    def test_getproject(env1):
        """Check that ``getproject`` prints an environment's project directory."""
        with temp_environ():
            os.environ.pop('VIRTUAL_ENV', None)
            with TemporaryDirectory() as tmpdir:
                invoke('setproject', 'env1', tmpdir)
                res = invoke('getproject', 'env1')
>               assert not res.err
E               assert not "/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')"
E                +  where "/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')" = Result(returncode=0, out='/tmp/tmpnp0j16_7', err="/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')").err

tests/test_getproject.py:16: AssertionError
________________________ test_project_directory_not_set ________________________

env1 = None

    def test_project_directory_not_set(env1):
        """Check the error message if no project directory was set.
    
        If no project directory has been configured for an environment,
        ``getproject`` should quit with an error message.
        """
        name = 'env1'
        with temp_environ():
            os.environ.pop('VIRTUAL_ENV', None)
            with TemporaryDirectory() as tmpdir:
                res = invoke('getproject', name)
                assert not res.out
>               assert res.err == (
                    "ERROR: no project directory set for Environment '{0}'"
                    .format(name)
                )
E               assert "/usr/lib/pyt...onment 'env1'" == "ERROR: no pro...onment 'env1'"
E                 - /usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \.
E                 -   pybinre = re.compile('pythonw?([0-9]+(\.[0-9]+(\.[0-9]+)?)?)?$')
E                   ERROR: no project directory set for Environment 'env1'

tests/test_getproject.py:32: AssertionError
___________________________ test_unknown_environment ___________________________

    def test_unknown_environment():
        """Check the error message if passed an unknown environment name.
    
        If ``getproject`` is invoked with the name of an environment that
        does not exist, the call should fail with an appropriate error
        message.
        """
        name = 'bogus-environment-that-/hopefully/-does-not-exist'
        res = invoke('getproject', name)
        assert not res.out
>       assert res.err == "ERROR: Environment '{0}' does not exist.".format(name)
E       AssertionError: assert '/usr/lib/pyt...es not exist.' == 'ERROR: Enviro...es not exist.'
E         - /usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \.
E         -   pybinre = re.compile('pythonw?([0-9]+(\.[0-9]+(\.[0-9]+)?)?)?$')
E           ERROR: Environment 'bogus-environment-that-/hopefully/-does-not-exist' does not exist.

tests/test_getproject.py:48: AssertionError
__________________ test_call_without_args_outside_active_venv __________________

    def test_call_without_args_outside_active_venv():
        """Check the error message if called without args outside a virtualenv.
    
        If ``getproject`` is called without additional arguments outside of
        an active virtualenv, it should print an error message.
        """
        os.environ.pop('VIRTUAL_ENV', None)
        res = invoke('getproject')
        assert not res.out
>       assert res.err == "ERROR: no virtualenv active"
E       AssertionError: assert '/usr/lib/pyt...ualenv active' == 'ERROR: no virtualenv active'
E         - /usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \.
E         -   pybinre = re.compile('pythonw?([0-9]+(\.[0-9]+(\.[0-9]+)?)?)?$')
E           ERROR: no virtualenv active

tests/test_getproject.py:60: AssertionError
___________________________________ test_ls ____________________________________

workon_home = PosixPath('/tmp/WORKON_HOME')

    def test_ls(workon_home):
        r = invoke('ls')
>       assert not r.out and not r.err
E       assert (not '' and not "/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')")
E        +  where '' = Result(returncode=0, out='', err="/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')").out
E        +  and   "/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')" = Result(returncode=0, out='', err="/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')").err

tests/test_ls.py:8: AssertionError
____________________________ test_implicit_project _____________________________

workon_home = PosixPath('/tmp/WORKON_HOME'), env1 = None

    def test_implicit_project(workon_home, env1):
        "use the cwd as project directory"
        with temp_environ():
            os.environ.pop('VIRTUAL_ENV', None)
            with TemporaryDirectory() as tmpdir:
                res = invoke('setproject', 'env1', cwd=tmpdir)
>               assert not res.err
E               assert not "/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')"
E                +  where "/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')" = Result(returncode=0, out='Setting project for env1 to /tmp/tmp1vgxs4qt', err="/usr/lib/python3.8/site-packages/clonevi...119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')").err

tests/test_setproject.py:35: AssertionError
_______________________________ test_setproject ________________________________

workon_home = PosixPath('/tmp/WORKON_HOME'), env1 = None

    def test_setproject(workon_home, env1):
        with temp_environ():
            os.environ.pop('VIRTUAL_ENV', None)
            with TemporaryDirectory() as tmpdir:
                res = invoke('setproject', 'env1', tmpdir)
>               assert not res.err
E               assert not "/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')"
E                +  where "/usr/lib/python3.8/site-packages/clonevirtualenv.py:119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')" = Result(returncode=0, out='Setting project for env1 to /tmp/tmp1x1ntsz9', err="/usr/lib/python3.8/site-packages/clonevi...119: SyntaxWarning: invalid escape sequence \\.\n  pybinre = re.compile('pythonw?([0-9]+(\\.[0-9]+(\\.[0-9]+)?)?)?$')").err

tests/test_setproject.py:44: AssertionError


A backslash-character pair that is not a valid escape sequence generates a DeprecationWarning since Python 3.6. In Python 3.8 it generates a SyntaxWarning instead.


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.8/fedora-rawhide-x86_64/00964804-pew/

For all our attempts to build pew with Python 3.8, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.8/package/pew/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.8:
https://copr.fedorainfracloud.org/coprs/g/python/python3.8/

Let us know here if you have any questions.

Comment 1 Miro Hrončok 2019-07-30 15:03:41 UTC
Tadej? Could you please have a look?

Comment 2 Tadej Janež 2019-07-31 10:23:25 UTC
(In reply to Miro Hrončok from comment #1)
> Tadej? Could you please have a look?

Yes, I'll take a look.

Comment 3 Tadej Janež 2019-08-01 11:47:15 UTC
(In reply to Tadej Janež from comment #2)
> 
> Yes, I'll take a look.

So, Pew itself should support Python 3.8 without a change.

I've tested it locally and also submitted an upstream pull request to start testing it on Python 3.8:
https://github.com/berdario/pew/pull/214.

The issue is that one of its dependencies, python-virtualenv-clone, has an outdated version in Fedora. Namely, version 0.2.6 from June 2015:
https://src.fedoraproject.org/rpms/python-virtualenv-clone/blob/5d1c44488598abe298989e9f2525fae6c5a83a28/f/python-virtualenv-clone.spec#_7
https://pypi.org/project/virtualenv-clone/#history

I've submitted a PR to update python-virtualenv-clone to version 0.5.3:
https://src.fedoraproject.org/rpms/python-virtualenv-clone/pull-request/3

Miro, please take a look and review/merge.

This version of python-virtualenv-clone contains fixes for the above issues regarding invalid escape sequences.

Ideally, python-virtualenv-clone package should run its test suite so that this issue would be detected at its source.

Comment 4 Miro Hrončok 2019-08-01 11:59:02 UTC
Nice analysis!

> Ideally, python-virtualenv-clone package should run its test suite so that this issue would be detected at its source.

I agree. Ideally, all packages should run their tests. Unfortunately python-virtualenv-clone is not maintained much.

Comment 5 Tadej Janež 2019-08-01 12:12:25 UTC
> Nice analysis!

Thanks!

> Unfortunately python-virtualenv-clone is not maintained much.

I can help maintain it since it is a vital dependency of Pew.

So, someone from the Infra-sig team can add me as a co-maintainer through Pagure?

Comment 6 Miro Hrončok 2019-08-01 12:58:17 UTC
I suppose that technically, Ralph would need to do that as Infra SIG doesn't have admin Rights. However I guess that at least some Infra SIG members have Pagure admin rights and can do that (or I can, if they agree).

Comment 7 Ben Cotton 2019-08-13 16:50:59 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 8 Ben Cotton 2019-08-13 18:37:32 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Comment 9 Tadej Janež 2019-08-14 08:45:49 UTC
Miro,

can this one be closed since updated python-virtualenv-clone has been pushed and Pew was built successfully with it (https://copr.fedorainfracloud.org/coprs/g/python/python3.8/build/993410/)?

Comment 10 Miro Hrončok 2019-08-14 08:51:44 UTC
Sure thing!


Note You need to log in before you can comment on or make changes to this bug.