Description of problem: If ovsdbapp throws errors/exceptions, they are logged in journalctl/syslog. There should be an option configure ovsdbapp logging to ovirt-provider-ovn.log, so we won't miss important errors/exceptions when looking at this log. Version-Release number of selected component (if applicable): RHV 4.2.2.2-0.1.el7
Can you share an example of a syslogged exception?
An example would be a failed ovs db transaction (as in the case of the unicode bug). Ovsdbapp retrieves the error from ovsdb and wraps it in its own exception, but without ovsdbapp logging enabled we loose some of the original information. By default we will only log the error messages.
Sure. This is an example taken from the journalctl of ovirt-provider-ovn service: # systemctl status ovirt-provider-ovn Jan 22 13:12:42 network-ge-2.scl.lab.tlv.redhat.com python[25021]: self.__target(*self.__args, **self.__kwargs) Jan 22 13:12:42 network-ge-2.scl.lab.tlv.redhat.com python[25021]: File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 96, in run Jan 22 13:12:42 network-ge-2.scl.lab.tlv.redhat.com python[25021]: self.idl.run() Jan 22 13:12:42 network-ge-2.scl.lab.tlv.redhat.com python[25021]: File "/usr/lib/python2.7/site-packages/ovs/db/idl.py", line 173, in run Jan 22 13:12:42 network-ge-2.scl.lab.tlv.redhat.com python[25021]: assert not self.txn Jan 22 13:12:42 network-ge-2.scl.lab.tlv.redhat.com python[25021]: AssertionError
Available in provider 1.2.9
Checked on: ovirt-provider-ovn-1.2.9-1.el7ev.noarch RHV 4.2.2-6 Provider restarts its service when detecting exception on ovsdbapp: 2018-03-18 09:24:11,011 requests.packages.urllib3.connectionpool Starting new HTTPS connection (1): network-ge-1.scl.lab.tlv.redhat.com 2018-03-18 09:24:11,286 root Uncaught exception Traceback (most recent call last): File "/usr/share/ovirt-provider-ovn/ovirt_provider_ovn.py", line 62, in run_with_except_hook run_original(*args2, **kwargs2) File "/usr/lib64/python2.7/threading.py", line 765, in run self.__target(*self.__args, **self.__kwargs) File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 96, in run assert False ## DELETE AssertionError 2018-03-18 09:24:11,288 root Irrecoverable error. Exiting! 2018-03-18 09:24:13,247 root Starting server 2018-03-18 09:24:13,248 root Version: 1.2.9-1 2018-03-18 09:24:13,248 root Build date: 20180313095726 2018-03-18 09:24:13,248 root Githash: 135935e In logger.conf we supply 'qualname'. I don't see it in the exception log. Does it make sense? logger.conf: [logger_ovsdbapp] level=ERROR handlers=logfile propagate=0 qualname=ovsdbapp
This is an exception in ovsdbapp code, but it is not logged by ovsdbapp code. Ovsdbapp does not catch it, so it crashes the ovsdbapp transaction thread, making ovsdbapp unusable. To prevent this have a hook in the provider code that detects and reports such situations, so this exception is logged by the provider. The same hook will shutdown the provider, which will then be restarted by systemd. This is prefered to trying to fix ovsdbapp.
Just noting: 'root' is the default logger which ovirt-provider-ovn uses, just in case we find messages/errors from this source.
This bugzilla is included in oVirt 4.2.2 release, published on March 28th 2018. Since the problem described in this bug report should be resolved in oVirt 4.2.2 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.