Red Hat Bugzilla – Bug 1553520
Cannot create listener with TLS termination
Last modified: 2018-06-27 09:35:58 EDT
Description of problem: Octavia requires python2-cryptography!=2.0,>=1.9 [1] and is synced with global-requirement.txt [2]. RHEL/CentOS7 provides python2-cryptography-1.7.2-1.el7 which is not good enough and throws exceptions on load balancer create in Octavia: 2018-03-08 23:45:46.453 24634 ERROR octavia.controller.worker.controller_worker File "/usr/lib/python2.7/site-packages/octavia/certificates/common/pkcs12.py", line 35, in get_certificate 2018-03-08 23:45:46.453 24634 ERROR octavia.controller.worker.controller_worker return self.certificate.to_cryptography().public_bytes( 2018-03-08 23:45:46.453 24634 ERROR octavia.controller.worker.controller_worker AttributeError: 'X509' object has no attribute 'to_cryptography' Version-Release number of selected component (if applicable): openstack-octavia-api-2.0.0-1.el7.noarch openstack-octavia-common-2.0.0-1.el7.noarch openstack-octavia-health-manager-2.0.0-1.el7.noarch openstack-octavia-housekeeping-2.0.0-1.el7.noarch openstack-octavia-worker-2.0.0-1.el7.noarch python2-octaviaclient-1.4.0-1.el7.noarch python-octavia-2.0.0-1.el7.noarch How reproducible: 100% Steps to Reproduce: 1. openstack loadbalancer create lb2 2. openstack loadbalancer listener create --protocol-port 443 --protocol TERMINATED_HTTPS --name listener2 --default-tls-container=http://<ommitted>:9311/v1/secrets/50a1b6e0-b53c-4b33-a06d-0544eaaf02f0 lb2 Actual results: 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/controller/queue/endpoint.py", line 68, in create_listener 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server self.worker.create_listener(listener_id) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/controller/worker/controller_worker.py", line 206, in create_listener 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server create_listener_tf.run() 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server failures[0].reraise() 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/controller/worker/tasks/amphora_driver_tasks.py", line 56, in execute 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server self.amphora_driver.update(listener, loadbalancer.vip) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 67, in update 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server certs = self._process_tls_certificates(listener) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 175, in _process_tls_certificates 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server self.cert_manager, listener) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/common/tls_utils/cert_parser.py", line 350, in load_certificates_data 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server check_only=True)) 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/common/tls_utils/cert_parser.py", line 363, in _map_cert_tls_container 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server primary_cn=get_primary_cn(cert), 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/common/tls_utils/cert_parser.py", line 373, in get_primary_cn 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server return get_host_names(tls_cert.get_certificate())['cn'] 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/certificates/common/pkcs12.py", line 35, in get_certificate 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server return self.certificate.to_cryptography().public_bytes( 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server AttributeError: 'X509' object has no attribute 'to_cryptography' 2018-03-08 23:45:46.482 24634 ERROR oslo_messaging.rpc.server Thanks to Iain MacDonnell for finding and help reporting the bug!
To clarify, the root cause of this bug is two-fold: 1) Wrong minimum python2-cryptography set in octavia.spec (set to >= 1.7.2 [1]) while upstream it's set correctly [2]. 2) Minimum required version of python-pyOpenSSL was not reflected in octavia/requirements.txt. A bump of minimum version should had been proposed to requirements/global-requirements.txt by octavia folks but that slipped. So, to fix this bug we need: - python-cryptography >= 1.9 - python-pyOpenSSL >= 17.1.0 [1] https://github.com/rdo-packages/octavia-distgit/blob/queens-rdo/openstack-octavia.spec#L115-L116 [2] https://github.com/openstack/octavia/blob/stable/queens/requirements.txt#L47
On puddle 2018-03-20.2 - There's no "python-cryptography" nor "python2-cryptography" packages at all, only "cryptography" version 1.7.2. ---- (overcloud) [stack@undercloud-0 ~]$ cat /etc/yum.repos.d/latest-installed 13 -p 2018-03-20.2 (overcloud) [stack@undercloud-0 ~]$ pip list | egrep *cryptography cryptography 1.7.2 (overcloud) [stack@undercloud-0 ~]$ pip list | egrep *OpenSSL* pyOpenSSL 17.3.0 ----
Use yum, not pip: rpm -qa | grep *cryptography*
(In reply to Carlos Goncalves from comment #4) > Use yum, not pip: rpm -qa | grep *cryptography* Nothing either. (overcloud) [stack@undercloud-0 ~]$ rpm -qa | grep *cryptography* (overcloud) [stack@undercloud-0 ~]$
You're running it on the undercloud, sorry I missed that from before. You have to run it in the octavia-worker docker container which runs on the controller nodes.
Indeed, for example I got: [heat-admin@controller-0 ~]$ sudo docker exec -ti octavia_worker bash ()[octavia@controller-0 /]$ rpm -qi python2-cryptography Name : python2-cryptography Version : 2.1.4 Release : 1.el7ost [...] (latest puddle)
Right, but octavia's .spec file is not yet enforcing latest minimum required dependency versions. It's pending import from RDO.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:2086