Bug 1435135 - ImportError: /usr/lib64/python3.6/site-packages/rpm/_rpm.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_AdjustIndices
ImportError: /usr/lib64/python3.6/site-packages/rpm/_rpm.cpython-36m-x86_64-l...
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: python3 (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Charalampos Stratakis
Fedora Extras Quality Assurance
: Reopened
: 1435136 1447934 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-03-23 05:00 EDT by Lukas Slebodnik
Modified: 2017-05-04 05:35 EDT (History)
16 users (show)

See Also:
Fixed In Version: python3-3.6.1-4.fc26
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-04-28 10:35:41 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Python 27867 None None None 2017-04-04 05:23 EDT
Python 29943 None None None 2017-03-30 02:47 EDT

  None (edit)
Description Lukas Slebodnik 2017-03-23 05:00:42 EDT
Description of problem:
The latest upgrade of rpm broke python bindings and dnf+yum is unusable

sh# dnf clean
Traceback (most recent call last):
  File "/usr/bin/dnf", line 57, in <module>
    from dnf.cli import main
  File "/usr/lib/python3.6/site-packages/dnf/__init__.py", line 31, in <module>
    import dnf.base
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 29, in <module>
    from dnf.yum import history
  File "/usr/lib/python3.6/site-packages/dnf/yum/history.py", line 31, in <module>
    import dnf.rpm.miscutils
  File "/usr/lib/python3.6/site-packages/dnf/rpm/__init__.py", line 22, in <module>
    from . import transaction
  File "/usr/lib/python3.6/site-packages/dnf/rpm/transaction.py", line 14, in <module>
    import rpm
  File "/usr/lib64/python3.6/site-packages/rpm/__init__.py", line 38, in <module>
    from rpm._rpm import *
ImportError: /usr/lib64/python3.6/site-packages/rpm/_rpm.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_AdjustIndices

Version-Release number of selected component (if applicable):
rpm -q rpm python3-rpm
rpm-4.13.0.1-3.fc26.x86_64
python3-rpm-4.13.0.1-3.fc26.x86_64

How reproducible:
deterministic

Steps to Reproduce:
1. //run python shell
   python3
2. //try to import rpm
   import rpm

Actual results:

sh# python3
Python 3.6.0 (default, Feb 28 2017, 13:40:35) 
[GCC 7.0.1 20170225 (Red Hat 7.0.1-0.10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/site-packages/rpm/__init__.py", line 38, in <module>
    from rpm._rpm import *
ImportError: /usr/lib64/python3.6/site-packages/rpm/_rpm.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_AdjustIndices

Expected results:
No import errors
Comment 1 Igor Gnatenko 2017-03-23 05:02:34 EDT
*** Bug 1435136 has been marked as a duplicate of this bug. ***
Comment 2 Lukas Slebodnik 2017-03-23 05:05:33 EDT
I cannot see a problem with python2 bindings

sh$ rpm -q python2
python2-rpm-4.13.0.1-14.fc27.x86_64

Python 2.7.13 (default, Feb 21 2017, 12:00:39) 
[GCC 7.0.1 20170219 (Red Hat 7.0.1-0.9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpm
>>>
Comment 3 Igor Gnatenko 2017-03-23 05:13:01 EDT
$ python3 -c "import rpm"; echo $?; rpm -q python3-rpm
0
python3-rpm-4.13.0.1-14.fc27.x86_64
Comment 4 Lukas Slebodnik 2017-03-23 05:27:23 EDT
sh$ rpm -q python3
python3-3.6.0-20.fc26.x86_64
Comment 5 Igor Gnatenko 2017-03-23 05:28:08 EDT
(In reply to Lukas Slebodnik from comment #4)
> sh$ rpm -q python3
> python3-3.6.0-20.fc26.x86_64

some steps to reproduce?
Comment 6 Panu Matilainen 2017-03-23 05:30:30 EDT
> Version-Release number of selected component (if applicable):
> rpm -q rpm python3-rpm
> rpm-4.13.0.1-3.fc26.x86_64
> python3-rpm-4.13.0.1-3.fc26.x86_64

vs

> sh$ rpm -q python2
> python2-rpm-4.13.0.1-14.fc27.x86_64

This doesn't quite add up. Querying for 'python2' cannot return 'python2-rpm', so this is a made-up copy-paste which makes me suspicious about the other versions.

The rpm bindings don't call PySlice_AdjustIndices() directly, that is a new function in Python 3.6.1. So sure enough it doesn't work when you have:

> sh# python3
> Python 3.6.0 (default, Feb 28 2017, 13:40:35) 

It seems that you have an rpm which was built with Python 3.6.1 installed on a system which has Python 3.6.0. Ie rpm from rawhide (as the python2-rpm version above suggests) on F26.
Comment 7 Lukas Slebodnik 2017-03-23 05:36:14 EDT
(In reply to Igor Gnatenko from comment #5)
> (In reply to Lukas Slebodnik from comment #4)
> > sh$ rpm -q python3
> > python3-3.6.0-20.fc26.x86_64
> 
> some steps to reproduce?

sh# docker pull docker.io/fedora:rawhide
Trying to pull repository docker.io/library/fedora ... 
sha256:bc6f39f0ddc11f7ef1c61c5825f2398f2b60f1142c088fa1f070472e7aca0885: Pulling from docker.io/library/fedora
Digest: sha256:bc6f39f0ddc11f7ef1c61c5825f2398f2b60f1142c088fa1f070472e7aca0885
Status: Image is up to date for docker.io/fedora:rawhide

sh# docker run -ti --rm docker.io/fedora:rawhide bash
[root@1c95e9479893 /]# rpm -q rpm
rpm-4.13.0.1-7.fc27.x86_64
[root@1c95e9479893 /]# python3
Python 3.6.0 (default, Feb 28 2017, 13:40:35) 
[GCC 7.0.1 20170225 (Red Hat 7.0.1-0.10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpm
>>> 
[root@1c95e9479893 /]# dnf update -y -d 0 rpm
[root@1c95e9479893 /]# rpm -q rpm
rpm-4.13.0.1-14.fc27.x86_64
[root@1c95e9479893 /]# python3
Python 3.6.0 (default, Feb 28 2017, 13:40:35) 
[GCC 7.0.1 20170225 (Red Hat 7.0.1-0.10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/site-packages/rpm/__init__.py", line 38, in <module>
    from rpm._rpm import *
ImportError: /usr/lib64/python3.6/site-packages/rpm/_rpm.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_AdjustIndices
Comment 8 Lukas Slebodnik 2017-03-23 05:39:21 EDT
(In reply to Panu Matilainen from comment #6)
> > Version-Release number of selected component (if applicable):
> > rpm -q rpm python3-rpm
> > rpm-4.13.0.1-3.fc26.x86_64
> > python3-rpm-4.13.0.1-3.fc26.x86_64
>
Sorry it was a copy&paste error. As you can see version is from f26 and not from f27.

@see more details in Comment 7
Comment 9 Panu Matilainen 2017-03-23 05:44:10 EDT
So yes, you have an rpm built with python 3.6.1 installed in a system with python 3.6.0. On a rawhide system, not F26. Such details tend are rather relevant.

Library versioning is supposed to prevent these kind of things but that depends on upstream cares, and is careful about it. There's nothing rpm can do about it, reassigning to rawhide python3:

There seems to be a missing library version increment in moving to 3.6.1, assuming python does that in the first place. If not, then this is NOTABUG.
Comment 10 Lukas Slebodnik 2017-03-23 05:58:29 EDT
I do not think that bug is in python3.
/lib64/libpython3.6m.so.1.0 does not contain version script. and they added just few functions there.

+PySlice_AdjustIndices
+PySlice_Unpack
+_PyDict_DelItemIf
+_PyDict_Pop_KnownHash

And adding new functions is not a backward incompatible change; therefore soname should not be bumped.
Comment 11 Panu Matilainen 2017-03-23 06:01:26 EDT
Nobody said anything about bumping soname, but library minor/micro version needs to be incremented when new functions are added. Otherwise this is what happens.
Comment 12 Panu Matilainen 2017-03-23 06:28:08 EDT
The worst part of this is that Python 3.6.1 uses this new function internally in a manner that makes code built against 3.6.1 incompatible with 3.6.0 even though said code does not use the new function(s) at all, AND does not indicate this incompatibility in library versioning in any way.

IF rpm (or whatever other code) actually used PySlice_AdjustIndices() directly then it'd be clearly up to rpm to require Python >= 3.6.1 because of it, but that's not the case.
Comment 13 Iryna Shcherbina 2017-03-23 10:53:56 EDT
Python does not increment the version of shared library [0] so it is NOTABUG. If you want ABI compatibility you should use the subset of the API that guarantees a stable ABI [1].

[0] https://github.com/python/cpython/blame/master/configure.ac#L129
[1] https://docs.python.org/3/c-api/stable.html
Comment 14 Lukas Slebodnik 2017-03-23 10:56:13 EDT
(In reply to Iryna Shcherbina from comment #13)
> Python does not increment the version of shared library [0] so it is
> NOTABUG.

It is a bug. @see description + few more comments.
Comment 15 Charalampos Stratakis 2017-03-23 11:46:03 EDT
(In reply to Lukas Slebodnik from comment #14)
> (In reply to Iryna Shcherbina from comment #13)
> > Python does not increment the version of shared library [0] so it is
> > NOTABUG.
> 
> It is a bug. @see description + few more comments.

You try to import a module which is compiled (not byte compiled, so it is basically a python c extension) with python 3.6.1 to 3.6.0. Why do you consider it a bug? This is the expected behaviour as pointed out by [0].

'Unfortunately, the API compatibility does not extend to binary compatibility (the ABI). The reason is primarily the evolution of struct definitions, where addition of a new field, or changing the type of a field, might not break the API, but can break the ABI. As a consequence, extension modules need to be recompiled for every Python release'

[0] https://docs.python.org/3/c-api/stable.html
Comment 16 Miro Hrončok 2017-03-23 12:00:24 EDT
We could add this to the python3-rpm package:

Requires: system-python3-libs >= %(%{__python3} --version | cut -f2 -d" ")

This should make sure the python3-rpm package requires al least the version of python that built it, in x.y.x form, and would prevent this.



The real question is: is rpm the only affected module?

If not, should Python also provide "python(abi) = x.y.x" in addition to "python(abi) = x.y" and packages require "python(abi) >= x.y.x" in addition to "python(abi) = x.y"?
Comment 17 Igor Gnatenko 2017-03-23 12:01:15 EDT
(In reply to Miro Hrončok from comment #16)
> We could add this to the python3-rpm package:
> 
> Requires: system-python3-libs >= %(%{__python3} --version | cut -f2 -d" ")
> 
> This should make sure the python3-rpm package requires al least the version
> of python that built it, in x.y.x form, and would prevent this.
> 
> 
> 
> The real question is: is rpm the only affected module?
Definitely not.
> 
> If not, should Python also provide "python(abi) = x.y.x" in addition to
> "python(abi) = x.y" and packages require "python(abi) >= x.y.x" in addition
> to "python(abi) = x.y"?
Comment 18 Miro Hrončok 2017-03-23 12:04:41 EDT
(In reply to Charalampos Stratakis from comment #15)
> You try to import a module which is compiled (not byte compiled, so it is
> basically a python c extension) with python 3.6.1 to 3.6.0. Why do you
> consider it a bug?

That's a package dependency bug. That package clearly should require an interpreter that is at least 3.6.1 (as it cannot work with 3.6.0), but it only requires an interpreter that is 3.6.
Comment 19 Miro Hrončok 2017-03-23 12:08:06 EDT
(In reply to Igor Gnatenko from comment #17)
> > The real question is: is rpm the only affected module?
> Definitely not.

I think so. So a proposal for the rpm automatic require/provide generator for python(abi):

Also provide the x.y.z version. For Python packages containing .so files, also require >= x.y.x.

Note that the system-python(abi) dance in the specfiles of dnf and others would need to be updated as well.
Comment 20 Miro Hrončok 2017-03-23 12:17:47 EDT
(In reply to Miro Hrončok from comment #19)
> Note that the system-python(abi) dance in the specfiles of dnf and others
> would need to be updated as well.

Probably just the %{?system_python_abi} macro.
Comment 21 Panu Matilainen 2017-03-24 03:27:43 EDT
(In reply to Miro Hrončok from comment #18)
> (In reply to Charalampos Stratakis from comment #15)
> > You try to import a module which is compiled (not byte compiled, so it is
> > basically a python c extension) with python 3.6.1 to 3.6.0. Why do you
> > consider it a bug?
> 
> That's a package dependency bug. That package clearly should require an
> interpreter that is at least 3.6.1 (as it cannot work with 3.6.0), but it
> only requires an interpreter that is 3.6.

And shared library dependency generation relies on upstreams doing the right thing with library versions. Yes it can probably be worked around on rpm dependency genereration level by using the full python version as a dependency.

But really, there's something not right in this whole picture. If python really doesn't give a damn about their library version compatibility, how come this doesn't happen more often? I mean, its the first time I see such a thing in 15+ years of being involved, and there are LOTS of compiled python extensions out there. Or wait, maybe its one of the reasons behind all those unreproducable python-involving crashes in the bugzilla... 

Also, it's not that people necessarily expect API/ABI *stability* but they do expect an explicit break when things change, silent untrackable breakage is just EVIL. This case is almost lucky because Python ends up inserting a call to a new symbol so the breakage is rather explicit (but still one step too late).

Of course it's not the Fedora maintainers fault, but I do find this rather strange.
Comment 22 Miro Hrončok 2017-03-24 05:42:44 EDT
Well from [0], it's hard to tell whether "Traditionally, the C API of Python will change with every release." also means patch level releases like this one. If it does, then this never happening before may be a set of coincidences (such that nobody ever did a dnf update on a single package that happens to be rebuilt after a patch Python update where this happens came to Fedora). If the statement is meant about minor releases only, then this is clearly a bug in Python itself.

I suppose we wait for Petr, who has deeper understanding of CPython, to give some advice. If he cannot help, we might ask upstream.

In the meantime, shall we "hotfix" python3-rpm?

[0] https://docs.python.org/3/c-api/stable.html
Comment 23 Igor Gnatenko 2017-03-24 05:44:04 EDT
(In reply to Miro Hrončok from comment #22)
> Well from [0], it's hard to tell whether "Traditionally, the C API of Python
> will change with every release." also means patch level releases like this
> one. If it does, then this never happening before may be a set of
> coincidences (such that nobody ever did a dnf update on a single package
> that happens to be rebuilt after a patch Python update where this happens
> came to Fedora). If the statement is meant about minor releases only, then
> this is clearly a bug in Python itself.
> 
> I suppose we wait for Petr, who has deeper understanding of CPython, to give
> some advice. If he cannot help, we might ask upstream.
> 
> In the meantime, shall we "hotfix" python3-rpm?
Can we fix %{?system_python_abi} to include full version of python, not only major.minor ?
Comment 25 Igor Gnatenko 2017-03-24 05:45:29 EDT
(In reply to Igor Gnatenko from comment #23)
> (In reply to Miro Hrončok from comment #22)
> > Well from [0], it's hard to tell whether "Traditionally, the C API of Python
> > will change with every release." also means patch level releases like this
> > one. If it does, then this never happening before may be a set of
> > coincidences (such that nobody ever did a dnf update on a single package
> > that happens to be rebuilt after a patch Python update where this happens
> > came to Fedora). If the statement is meant about minor releases only, then
> > this is clearly a bug in Python itself.
> > 
> > I suppose we wait for Petr, who has deeper understanding of CPython, to give
> > some advice. If he cannot help, we might ask upstream.
> > 
> > In the meantime, shall we "hotfix" python3-rpm?
> Can we fix %{?system_python_abi} to include full version of python, not only
> major.minor ?
Hmm, but this will not help for non-system-python.. Weird stuff.
Comment 26 Miro Hrončok 2017-03-24 05:46:30 EDT
(In reply to Igor Gnatenko from comment #23)
> (In reply to Miro Hrončok from comment #22)
> > In the meantime, shall we "hotfix" python3-rpm?
> Can we fix %{?system_python_abi} to include full version of python, not only
> major.minor ?

"system-python(abi) = x.y.z" is not provided. So that would ahv to be system-python-libs.

This would add an unnecessary strict dependency on all system-python-enabled packages including the pure-python ones. Also, it would not solve this problem for non-system-python packages using the C API.
Comment 27 Petr Viktorin 2017-03-27 05:36:38 EDT
(In reply to Miro Hrončok from comment #22)
> Well from [0], it's hard to tell whether "Traditionally, the C API of Python
> will change with every release." also means patch level releases like this
> one. If it does, then this never happening before may be a set of
> coincidences (such that nobody ever did a dnf update on a single package
> that happens to be rebuilt after a patch Python update where this happens
> came to Fedora). If the statement is meant about minor releases only, then
> this is clearly a bug in Python itself.
> 
> I suppose we wait for Petr, who has deeper understanding of CPython, to give
> some advice. If he cannot help, we might ask upstream.

Please ask upstream.

I expect the reply will be that ABI is forward compatible, so you can use modules built for 3.6.0 on 3.6.1, but not the other way around. But it would be good to have them explicitly agree on guarantees they're providing.
Comment 28 Panu Matilainen 2017-03-27 06:28:54 EDT
I dont think the question here is so much whether X is compatible with Y or not, but about *tracking* of the incompatibilities. Silent breaks in compatibility is nasty.

FWIW, I've added a manual dependency on the full python version to rawhide rpm for now. It's one thing to have random user program temporarily break because of this, system updater breaking is slightly worse...
Comment 29 Charalampos Stratakis 2017-03-27 09:21:15 EDT
(In reply to Panu Matilainen from comment #28)
> I dont think the question here is so much whether X is compatible with Y or
> not, but about *tracking* of the incompatibilities. Silent breaks in
> compatibility is nasty.
> 
> FWIW, I've added a manual dependency on the full python version to rawhide
> rpm for now. It's one thing to have random user program temporarily break
> because of this, system updater breaking is slightly worse...

Could you create an issue at http://bugs.python.org/ or maybe start a thread at https://mail.python.org/mailman/listinfo/python-dev ?

It would be really interesting to see the outcome of these issues you raise, although I suspect it might have been brought up before.
Comment 30 Panu Matilainen 2017-03-28 00:56:03 EDT
I'm not participating in python upstream in any way and not interested in starting now, got plenty enough time sinks as it is thank you. I'd expect the Fedora Python maintainers to have channels open to that direction however.
Comment 31 Miro Hrončok 2017-03-28 07:08:36 EDT
I'm going to ask on python-dev mailing list.
Comment 32 Charalampos Stratakis 2017-04-18 14:02:16 EDT
python3 will be updated to 3.6.1 for F26 which should solve the issue as the code is forward compatible and the issues have been addressed upstream for the 2.7 and 3.5 branches.
Comment 33 Fedora Update System 2017-04-18 14:03:09 EDT
python3-3.6.1-3.fc26 python3-docs-3.6.1-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-1918e9452a
Comment 34 Miro Hrončok 2017-04-18 16:37:19 EDT
(In reply to Charalampos Stratakis from comment #32)
> python3 will be updated to 3.6.1 for F26 which should solve the issue as the
> code is forward compatible and the issues have been addressed upstream for
> the 2.7 and 3.5 branches.

Well doesn't that mean we'd need to make sure all the newly built python3 packages depend on >= 3.6.1?
Comment 35 Fedora Update System 2017-04-19 08:51:28 EDT
python3-3.6.1-3.fc26, python3-docs-3.6.1-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-1918e9452a
Comment 36 Fedora Update System 2017-04-19 12:46:52 EDT
python3-3.6.1-3.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-1918e9452a
Comment 37 Fedora Update System 2017-04-19 12:53:06 EDT
python3-3.6.1-4.fc26 python3-docs-3.6.1-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-e1e5e717c6
Comment 38 Fedora Update System 2017-04-19 12:53:28 EDT
python3-3.6.1-4.fc26 python3-docs-3.6.1-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-e1e5e717c6
Comment 39 Fedora Update System 2017-04-19 22:22:46 EDT
python3-3.6.1-4.fc26, python3-docs-3.6.1-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-e1e5e717c6
Comment 40 Fedora Update System 2017-04-28 10:35:41 EDT
python3-3.6.1-4.fc26, python3-docs-3.6.1-1.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
Comment 41 Christian Heimes 2017-05-04 05:35:23 EDT
*** Bug 1447934 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.