Bug 1447934

Summary: custodia does not seem to pull correct dependencies
Product: [Fedora] Fedora Reporter: Jan Pazdziora <jpazdziora>
Component: custodiaAssignee: Simo Sorce <ssorce>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: cheimes, jpazdziora, mhroncok, ssorce
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-04 09:35:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Pazdziora 2017-05-04 09:23:04 UTC
Description of problem:

After building custodia in container, the custodia server cannot be started.

Version-Release number of selected component (if applicable):

custodia-0.3.1-2.fc27

How reproducible:

Deterministic.

Steps to Reproduce:
1. Have Dockerfile

FROM registry.fedoraproject.org/fedora:27
RUN dnf -y install custodia && dnf clean all
ENTRYPOINT /usr/sbin/custodia /etc/custodia/custodia.conf

2. Build with docker build -t custodia-fedora-27 .
3. Run with docker run --rm -ti custodia-fedora-27

Actual results:

$ docker run --rm -ti custodia-fedora-27 
2017-05-04 09:15:04 - custodia                         - Custodia debug logger enabled
2017-05-04 09:15:04 - custodia                         - Custodia audit log: /var/log/custodia/audit.log
2017-05-04 09:15:04 - custodia                         - Config file <_io.TextIOWrapper name='/etc/custodia/custodia.conf' mode='r' encoding='ANSI_X3.4-1968'> loaded
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/custodia/server/__init__.py", line 84, in _create_plugin
    handler = _load_plugin_class(menu, handler_name)
  File "/usr/lib/python3.6/site-packages/custodia/server/__init__.py", line 65, in _load_plugin_class
    return ep.resolve()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2308, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.6/site-packages/custodia/store/encgen.py", line 6, in <module>
    from jwcrypto.jwe import JWE
  File "/usr/lib/python3.6/site-packages/jwcrypto/jwe.py", line 10, in <module>
    from cryptography.hazmat.primitives import constant_time, hashes, hmac
  File "/usr/lib64/python3.6/site-packages/cryptography/hazmat/primitives/constant_time.py", line 9, in <module>
    from cryptography.hazmat.bindings._constant_time import lib
ImportError: /usr/lib64/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_AdjustIndices

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/custodia/server/__init__.py", line 189, in _load_plugins
    config[menu][name] = _create_plugin(parser, s, menu)
  File "/usr/lib/python3.6/site-packages/custodia/server/__init__.py", line 89, in _create_plugin
    '[%r]: %s' % (e, handler_name))
ValueError: Invalid format for "handler" option [ImportError('/usr/lib64/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_AdjustIndices',)]: EncryptedOverlay

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/sbin/custodia", line 11, in <module>
    load_entry_point('custodia==0.3.1', 'console_scripts', 'custodia')()
  File "/usr/lib/python3.6/site-packages/custodia/server/__init__.py", line 211, in main
    _load_plugins(config, cfgparser)
  File "/usr/lib/python3.6/site-packages/custodia/server/__init__.py", line 191, in _load_plugins
    raise RuntimeError(menu, name, e)
RuntimeError: ('stores', 'encrypted_sqlite', ValueError('Invalid format for "handler" option [ImportError(\'/usr/lib64/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_AdjustIndices\',)]: EncryptedOverlay',))

Expected results:

Python detected LC_CTYPE=C: LC_ALL & LANG coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).
2017-05-04 09:19:40 - custodia                         - Custodia debug logger enabled
2017-05-04 09:19:40 - custodia                         - Custodia audit log: /var/log/custodia/audit.log
2017-05-04 09:19:40 - custodia                         - Config file <_io.TextIOWrapper name='/etc/custodia/custodia.conf' mode='r' encoding='UTF-8'> loaded
2017-05-04 09:19:40 - server                           - Serving on Unix socket /var/run/custodia/custodia.sock

Additional info:

Comment 1 Jan Pazdziora 2017-05-04 09:24:23 UTC
Putting

RUN dnf -y upgrade && dnf clean all

to the Dockerfile before the custodia installation makes the error go away. But if custodia needs some additional libraries / packages / packages in particular version, it should pull them via versioned dependencies.

Comment 2 Christian Heimes 2017-05-04 09:35:23 UTC
That's not a bug in Custodia but some incompatibility between python3-cffi and python3's ABI. python3-cffi is a dependency of python3-cryptography. The bug is caused by an ABI mismatch between Python 3.6.0 and packages that have been compiled for Python 3.6.1 without Py_LIMITED_API, see #1435135.

*** This bug has been marked as a duplicate of bug 1435135 ***

Comment 3 Jan Pazdziora 2017-05-04 09:43:15 UTC
Well, trying the reproducer,

$ docker run --rm -ti registry.fedoraproject.org/fedora:27 python3 -c "import rpm"
$ echo $?
0

passes.

So the image itself does not seem to have the problem.

Do we need this bugzilla reset to python3-cffi or some other component that needs to be rebuilt?

Comment 4 Christian Heimes 2017-05-04 10:04:32 UTC
Here is a reproducer:

$ docker run custodia-fedora-27 python3 -c "import _cffi_backend"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: /usr/lib64/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_AdjustIndices

The Fedora rawhide image has Python 3.6.0, you need 3.6.1:

$ docker run custodia-fedora-27 /usr/bin/rpm -qa python3
python3-3.6.0-20.fc26.x86_64

Comment 5 Miro HronĨok 2017-05-04 11:42:48 UTC
Imagine this as a bug in the python3 package that has been fixed. All you have to do is update python3.