RDO tickets are now tracked in Jira https://issues.redhat.com/projects/RDO/issues/
Bug 1148988 - nova-api fails to start after reboot
Summary: nova-api fails to start after reboot
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RDO
Classification: Community
Component: openstack-nova
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: Juno
Assignee: Russell Bryant
QA Contact: Ami Jeain
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-10-02 20:09 UTC by Rob Crittenden
Modified: 2015-03-30 01:12 UTC (History)
12 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2015-03-30 01:12:02 UTC
Embargoed:


Attachments (Terms of Use)
snipped of nova-api.log (115.91 KB, text/plain)
2014-10-02 20:09 UTC, Rob Crittenden
no flags Details

Description Rob Crittenden 2014-10-02 20:09:51 UTC
Created attachment 943548 [details]
snipped of nova-api.log

Description of problem:

Installed using packstack --allinone

Did some basic verification that system was working:

- upload image to glance
- booted nova instance as demo user using that image

I realized that the VM I was using for the RDO test day had only 1 GB of RAM and I was in swap city. I shut down, bumped it up to 2GB and rebooted.

Upon reboot nova-api fails to start:

The crux of the error seems to be:

2014-10-02 16:06:03.351 7085 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
2014-10-02 16:06:03.353 7085 CRITICAL nova [-] OSError: [Errno 38] Function not implemented
2014-10-02 16:06:03.353 7085 TRACE nova Traceback (most recent call last):
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/bin/nova-api", line 10, in <module>
2014-10-02 16:06:03.353 7085 TRACE nova     sys.exit(main())
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/cmd/api.py", line 55, in main
2014-10-02 16:06:03.353 7085 TRACE nova     server = service.WSGIService(api, use_ssl=should_use_ssl)
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/service.py", line 331, in __init__
2014-10-02 16:06:03.353 7085 TRACE nova     self.manager = self._get_manager()
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/service.py", line 383, in _get_manager
2014-10-02 16:06:03.353 7085 TRACE nova     return manager_class()
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/api/manager.py", line 30, in __init__
2014-10-02 16:06:03.353 7085 TRACE nova     self.network_driver.metadata_accept()
       2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/network/linux_net.py", line 666, in metadata_accept
2014-10-02 16:06:03.353 7085 TRACE nova     iptables_manager.apply()
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/network/linux_net.py", line 434, in apply
2014-10-02 16:06:03.353 7085 TRACE nova     self._apply()
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line 322, in inner
2014-10-02 16:06:03.353 7085 TRACE nova     with lock(name, lock_file_prefix, external, lock_path):
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2014-10-02 16:06:03.353 7085 TRACE nova     return self.gen.next()
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line 287, in lock
2014-10-02 16:06:03.353 7085 TRACE nova     with ext_lock:
2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line 171, in __enter__
2014-10-02 16:06:03.353 7085 TRACE nova     self.acquire()
          2014-10-02 16:06:03.353 7085 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line 166, in acquire
2014-10-02 16:06:03.353 7085 TRACE nova     initial_value=1)
2014-10-02 16:06:03.353 7085 TRACE nova OSError: [Errno 38] Function not implemented
                                                       
Version-Release number of selected component (if applicable):

openstack-nova-api-2014.2-0.4.b3.el7.centos.noarch

I tested on a RHEL-7 VM

Comment 2 Russell Bryant 2014-10-13 15:06:36 UTC
Now that Juno RCs are out, please try again with a newer package and see if you still have a problem.  Thanks!

Comment 3 Pádraig Brady 2014-10-13 15:56:04 UTC
posix_ipc.Semaphore() is failing with ENOSYS
That's surprising. I tried the following on Centos7 for example without issue:

    s=posix_ipc.Semaphore('/2jmj7l5rSw0yVb_vlWAYkK_YBwk=',flags=posix_ipc.O_CREAT, initial_value=1)


I wonder might /dev/shm not be mounted on your system?

Note this posix_ipc is going away anyway in the next Juno release.
That should be testable using the version of nova at:

http://copr-be.cloud.fedoraproject.org/results/jruzicka/rdo-juno-epel-7/epel-7-x86_64/openstack-nova-2014.2-0.5rc1.el7/

Comment 4 Lars Kellogg-Stedman 2015-03-30 01:12:02 UTC
I've tested this with RDO Juno on RHEL7 (openstack-nova-api-2014.1.3-2.el7.centos.noarch), and everything comes up correctly after a reboot.


Note You need to log in before you can comment on or make changes to this bug.