Bug 1303966 - Can't re-launch horizon after a controller reboot: ImportError: cannot import name base
Summary: Can't re-launch horizon after a controller reboot: ImportError: cannot import...
Keywords:
Status: CLOSED EOL
Alias: None
Product: RDO
Classification: Community
Component: python-cinderclient
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: Kilo
Assignee: Eric Harney
QA Contact: nlevinki
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-02-02 14:42 UTC by Udi Kalifon
Modified: 2016-05-19 15:55 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-05-19 15:55:20 UTC


Attachments (Terms of Use)

Description Udi Kalifon 2016-02-02 14:42:51 UTC
Description of problem:
Installed Mitaka in a virtualized environment according to https://www.rdoproject.org/testday/mitaka/milestone2/ and https://www.rdoproject.org/rdo-manager/. After rebooting the controller, when trying to access horizon I see: 

503 Service Unavailable
No server is available to handle this request. 


There are exceptions in /var/log/httpd/horizon_error.log:

mod_wsgi (pid=7703): Exception occurred processing WSGI script '/usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi'.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 170, in __call__
    self.load_middleware()
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 52, in load_middleware
    mw_instance = mw_class()
  File "/usr/lib/python2.7/site-packages/django/middleware/locale.py", line 24, in __init__
    for url_pattern in get_resolver(None).url_patterns:
  File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 401, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 395, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/urls.py", line 35, in <module>
    url(r'^api/', include('openstack_dashboard.api.rest.urls')),
  File "/usr/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 33, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/__init__.py", line 36, in <module>
    from openstack_dashboard.api import cinder
  File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/cinder.py", line 30, in <module>
    from cinderclient.v2.contrib import list_extensions as cinder_list_extensions 
  File "/usr/lib/python2.7/site-packages/cinderclient/v2/__init__.py", line 17, in <module>
    from cinderclient.v2.client import Client    # noqa
  File "/usr/lib/python2.7/site-packages/cinderclient/v2/client.py", line 16, in <module>
    from cinderclient import client
  File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 52, in <module>
    from eventlet import sleep
  File "/usr/lib/python2.7/site-packages/eventlet/__init__.py", line 10, in <module> 
    from eventlet import convenience
  File "/usr/lib/python2.7/site-packages/eventlet/convenience.py", line 3, in <module>
    from eventlet import greenio
  File "/usr/lib/python2.7/site-packages/eventlet/greenio/__init__.py", line 3, in <module>
    from eventlet.greenio.base import *  # noqa
  File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 441, in <module>
    from OpenSSL import SSL
  File "/usr/lib/python2.7/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import rand, crypto, SSL
  File "/usr/lib/python2.7/site-packages/OpenSSL/rand.py", line 11, in <module> 
    from OpenSSL._util import (
  File "/usr/lib/python2.7/site-packages/OpenSSL/_util.py", line 7, in <module> 
    binding = Binding()
  File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 63, in __init__
    self._ensure_ffi_initialized()
  File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 95, in _ensure_ffi_initialized
    cls._register_osrandom_engine()
  File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 70, in _register_osrandom_engine
    raise RuntimeError("osrandom engine already registered")
RuntimeError: osrandom engine already registered
mod_wsgi (pid=7703): Exception occurred processing WSGI script '/usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi'.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 170, in __call__
    self.load_middleware()
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 52, in load_middleware
    mw_instance = mw_class()
  File "/usr/lib/python2.7/site-packages/django/middleware/locale.py", line 24, in __init__
    for url_pattern in get_resolver(None).url_patterns:
  File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 401, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 395, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/urls.py", line 35, in <module>
    url(r'^api/', include('openstack_dashboard.api.rest.urls')),
  File "/usr/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 33, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/__init__.py", line 34, in <module>
    from openstack_dashboard.api import base
ImportError: cannot import name base


Version-Release number of selected component (if applicable):
python-django-horizon-9.0.0.0b3-dev67.el7.centos.noarch


How reproducible:
100%


Steps to Reproduce:
1. Install Mitaka according to https://www.rdoproject.org/testday/mitaka/milestone2/ and https://www.rdoproject.org/rdo-manager/
2. Reboot the overcloud (I rebooted the host).
3. Wait for the services to come back up and access horizon


Actual results:
503 Service Unavailable
No server is available to handle this request.

Comment 1 Matthias Runge 2016-02-02 15:38:01 UTC
This looks suspicious here, after Horizon, there is cinderclient, then eventlet and OpenSSL involved.


I could not reproduce this in tests, thus it might be timing related, which makes a bit of sense, after working fine before reboot.

The error is thrown in python-cryptography itself.

Comment 2 Matthias Runge 2016-02-03 09:27:37 UTC
Udi, would it be possible to reboot again?

Comment 3 Udi Kalifon 2016-02-07 10:34:05 UTC
I rebooted again and horizon came up right this time.

Comment 4 Matthias Runge 2016-02-09 07:39:48 UTC
There seems to be a timing issue in the whole stack. Not really sure, who to blame in the long queue.

  File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 95, in _ensure_ffi_initialized
    cls._register_osrandom_engine()
  File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 70, in _register_osrandom_engine
    raise RuntimeError("osrandom engine already registered")
RuntimeError: osrandom engine already registered

states, python-cryptography had the issue, re-adjusting to python-cinderclient, as that was the last RDO component causing this issue.

Comment 5 Chandan Kumar 2016-05-19 15:55:20 UTC
This bug is against a Version which has reached End of Life.
If it's still present in supported release (http://releases.openstack.org), please update Version and reopen.


Note You need to log in before you can comment on or make changes to this bug.