Bug 1708004 - python-sqlalchemy fails to build with Python 3.8
Summary: python-sqlalchemy fails to build with Python 3.8
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: python-sqlalchemy
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Nils Philippsen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON38
TreeView+ depends on / blocked
 
Reported: 2019-05-08 23:02 UTC by Miro Hrončok
Modified: 2019-05-17 21:08 UTC (History)
6 users (show)

Fixed In Version: python3-3.8.0~a4-2.fc31
Clone Of:
Environment:
Last Closed: 2019-05-17 09:27:30 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Full log from Copr (1.30 MB, text/plain)
2019-05-08 23:02 UTC, Miro Hrončok
no flags Details

Description Miro Hrončok 2019-05-08 23:02:29 UTC
Created attachment 1565837 [details]
Full log from Copr

python-sqlalchemy-1.3.3-1.fc31 fails to build with Python 3.8.0a4 due to a DeprecationWarning being treated as error:

+ /usr/bin/python3 -m pytest test
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 207, in wrap_session
INTERNALERROR>     config.hook.pytest_sessionstart(session=session)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 59, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/fixtures.py", line 61, in pytest_sessionstart
INTERNALERROR>     session._fixturemanager = FixtureManager(session)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/fixtures.py", line 1119, in __init__
INTERNALERROR>     session.config.pluginmanager.register(self, "funcmanage")
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 316, in register
INTERNALERROR>     ret = super(PytestPluginManager, self).register(plugin, name)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 95, in register
INTERNALERROR>     hookimpl = HookImpl(plugin, plugin_name, method, hookimpl_opts)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 344, in __init__
INTERNALERROR>     self.argnames, self.kwargnames = varnames(self.function)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 167, in varnames
INTERNALERROR>     spec = _getargspec(func)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 130, in _getargspec
INTERNALERROR>     return inspect.getfullargspec(func)
INTERNALERROR>   File "/usr/lib64/python3.8/inspect.py", line 1114, in getfullargspec
INTERNALERROR>     warnings.warn("Use inspect.signature() instead of inspect.getfullargspec()",
INTERNALERROR> DeprecationWarning: Use inspect.signature() instead of inspect.getfullargspec()

Full log attached.

Comment 1 Miro Hrončok 2019-05-09 11:38:48 UTC
See also https://github.com/pytest-dev/pluggy/issues/209

Comment 2 Michael Bayer 2019-05-09 13:56:02 UTC
as the link above implies that seems to be originating from py.test, but SQLAlchemy uses getfullargspec() also internally, not sure if the warning would just move.  getfullargspec() is already a Py3k-only method I had to move from getargspec(), I wish they'd stop changing this.

Comment 3 Miro Hrončok 2019-05-09 14:12:32 UTC
What bothers me is that a DeprecationWarning renders the entire thing failed :(

Comment 4 Michael Bayer 2019-05-09 17:53:17 UTC
so this should be tunable via the warnings filter and possibly a py.test option in setup.cfg. OTOH if pluggy is forcing this to behave this way then they'd have to fix it or fedora needs to patch it.

Comment 5 Miro Hrončok 2019-05-09 18:10:25 UTC
Let's wait a bit for pluggy upstream response I guess. It's in pytest-dev GitHub org, so if pytest is forcing it, they should know as well.

Comment 6 Miro Hrončok 2019-05-13 09:15:57 UTC
(In reply to Michael Bayer from comment #2)
> as the link above implies that seems to be originating from py.test, but
> SQLAlchemy uses getfullargspec() also internally, not sure if the warning
> would just move.  getfullargspec() is already a Py3k-only method I had to
> move from getargspec(), I wish they'd stop changing this.

Apparently, you are not alone. If I get this right, this will not be deprecated in 3.8:

https://bugs.python.org/issue36751
https://github.com/python/cpython/pull/13245

Comment 7 Miro Hrončok 2019-05-15 12:14:41 UTC
Pluggy 0.11.1 will include a fix for this.

Comment 8 Miro Hrončok 2019-05-16 16:38:26 UTC
Heh, but sqlalchemy has that as well:

Traceback (most recent call last):
  File "/usr/lib64/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.8/site-packages/pytest.py", line 89, in <module>
    raise SystemExit(pytest.main())
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 79, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 296, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 68, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 59, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 242, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 234, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 296, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 68, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 59, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 656, in pytest_sessionfinish
    outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/plugin/pytestplugin.py", line 98, in pytest_sessionfinish
    plugin_base.final_process_cleanup()
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/plugin/plugin_base.py", line 565, in final_process_cleanup
    engines.testing_reaper._stop_test_ctx_aggressive()
  File "<string>", line 2, in _stop_test_ctx_aggressive
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/assertions.py", line 124, in decorate
    return fn(*args, **kw)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/engines.py", line 97, in _stop_test_ctx_aggressive
    rec.dispose()
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/engine/base.py", line 2014, in dispose
    self.pool = self.pool.recreate()
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/impl.py", line 298, in recreate
    return self.__class__(
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/impl.py", line 290, in __init__
    Pool.__init__(self, creator, **kw)
  File "<string>", line 2, in __init__
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/util/deprecations.py", line 130, in warned
    return fn(*args, **kwargs)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/base.py", line 202, in __init__
    self._creator = creator
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/base.py", line 238, in _creator
    self._invoke_creator = self._should_wrap_creator(creator)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/base.py", line 247, in _should_wrap_creator
    argspec = util.get_callable_argspec(self._creator, no_self=True)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/util/langhelpers.py", line 351, in get_callable_argspec
    return compat.inspect_getfullargspec(fn)
  File "/usr/lib64/python3.8/inspect.py", line 1114, in getfullargspec
    warnings.warn("Use inspect.signature() instead of inspect.getfullargspec()",
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/assertions.py", line 152, in our_warn
    return real_warn(msg, *arg, **kw)
DeprecationWarning: Use inspect.signature() instead of inspect.getfullargspec()

Comment 9 Michael Bayer 2019-05-16 17:17:20 UTC
yes see comment #2 as well as https://github.com/sqlalchemy/sqlalchemy/issues/4674

Comment 10 Michael Bayer 2019-05-16 17:53:22 UTC
see also https://github.com/sqlalchemy/alembic/issues/563 as people that use SQLAlchemy usually use Alembic as well as https://github.com/sqlalchemy/mako/issues/295 which Alembic uses.

Comment 11 Miro Hrončok 2019-05-16 19:39:54 UTC
See https://src.fedoraproject.org/rpms/python-tornado/pull-request/5 - that is an example of a probably simplest course of action for a downstream maintainer.

Comment 12 Miro Hrončok 2019-05-16 19:42:34 UTC
lib/sqlalchemy/testing/warnings.py has:

def setup_filters():
    """Set global warning behavior for the test suite."""

    warnings.filterwarnings(
        "ignore", category=sa_exc.SAPendingDeprecationWarning
    )
    warnings.filterwarnings("error", category=sa_exc.SADeprecationWarning)
    warnings.filterwarnings("error", category=sa_exc.SAWarning)

    # some selected deprecations...
    warnings.filterwarnings("error", category=DeprecationWarning)
    warnings.filterwarnings(
        "ignore", category=DeprecationWarning, message=".*StopIteration"
    )
    warnings.filterwarnings(
        "ignore", category=DeprecationWarning, message=".*inspect.getargspec"
    )

I suppose we can sed s/getargspec/*/ and call it a day.

Comment 13 Miro Hrončok 2019-05-16 19:49:06 UTC
Testing a build with:

%prep
...
# https://bugzilla.redhat.com/show_bug.cgi?id=1708004
sed -i 's/getargspec//' lib/sqlalchemy/testing/warnings.py


It is enough to make the tests at least run.

Comment 14 Michael Bayer 2019-05-16 19:54:17 UTC
yes if you can do that on the package side here, that is your best course right now.

Comment 15 Miro Hrončok 2019-05-16 19:57:00 UTC
A test failure:

=================================== FAILURES ===================================
_ ComponentReflectionTest_sqlite+pysqlite_3_28_0.test_deprecated_get_primary_keys _
Traceback (most recent call last):
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/suite/test_reflection.py", line 594, in test_deprecated_get_primary_keys
    users.name,
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/assertions.py", line 309, in assert_raises_message
    assert False, "Callable did not raise an exception"
AssertionError: Callable did not raise an exception
=========================== short test summary info ============================


It speaks of deprecation, will check the test code.

Comment 16 Miro Hrončok 2019-05-16 20:10:19 UTC
It fails even with: sed -i 's/getargspec/get(full)?argspec/' lib/sqlalchemy/testing/warnings.py

Comment 17 Miro Hrončok 2019-05-16 20:47:05 UTC
For now, upstream has removed the DeprecationWarning. I will do the same and later upload an up to date build.log if it keeps failing.

Comment 18 Michael Bayer 2019-05-16 21:51:19 UTC
that test is looking for:

 sa_exc.SADeprecationWarning,
            r".*get_primary_keys\(\) method is deprecated",

Comment 19 Miro Hrončok 2019-05-17 09:27:30 UTC
> For now, upstream has removed the DeprecationWarning. I will do the same and later upload an up to date build.log if it keeps failing.

All good now. Thank you, Michael.

Comment 20 Michael Bayer 2019-05-17 21:08:28 UTC
what a drama.  I'm going to pursue changing getfullargspec() in any case since the one in Python 3 is very slow.


Note You need to log in before you can comment on or make changes to this bug.