DescriptionMarian Krcmarik
2016-11-21 12:17:46 UTC
Description of problem:
If an AppManager.close call is started and all AppManager.services are stopped, AppManager.run_apps starts another close() call, resulting in a KeyError exception in close() (*1). Prevent that using a semaphore.
The upstream patch:
https://github.com/osrg/ryu/commit/b0ab4f16028c452374c5a0f22bd970038194f142
I hit this problem when stopping neutron-openvswitch-agent service on Openstack computes/controllers in order to setup up Instance Ha (where the service is being stopped firstly and then put under pacemaker management for computes and left under systemd management for controllers).
The traceack on which openvswitch failed was following:
2016-11-20 07:30:59.464 102285 ERROR neutron.agent.linux.async_process [-] Error received from [ovsdb-client monitor Interface name,ofport,external_ids --format=json]: None
2016-11-20 07:30:59.465 102285 ERROR neutron.agent.linux.async_process [-] Process [ovsdb-client monitor Interface name,ofport,external_ids --format=json] dies due to the error: None
2016-11-20 07:30:59.465 102285 DEBUG neutron.agent.linux.async_process [-] Output received from [ovsdb-client monitor Interface name,ofport,external_ids --format=json]: None _read_stdout /usr/lib/python2.7/site-packages/neutron/agent/linux/async_process.py:237
2016-11-20 07:30:59.466 102285 ERROR ryu.lib.hub [-] hub: uncaught exception: Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ryu/lib/hub.py", line 54, in _launch
return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/ryu/base/app_manager.py", line 545, in close
self.uninstantiate(app_name)
File "/usr/lib/python2.7/site-packages/ryu/base/app_manager.py", line 528, in uninstantiate
app = self.applications.pop(name)
KeyError: 'ofctl_service'
Version-Release number of selected component (if applicable):
python-ryu-4.3-2.el7ost.noarch.rpm
How reproducible:
Sometimes
Steps to Reproduce:
1. Stop neutron-openvswitch-agent service
Actual results:
service fails, not sure about other consequences - maybe some binded ports remain active
Expected results:
Sane service stop
Additional info: