Bug 1636867

Created attachment 1491526 [details]
ovn log

Verified on - ovirt-provider-ovn-1.2.17-0.20181016103251.git3d03ecf.el7.noarch and 4.3.0-0.0.master.20181016132820.gite60d148.el7

This bugzilla is included in oVirt 4.2.7 release, published on November 2nd 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.7 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Closed by mistake, moving back to qa -> verified

Even this issue was fixed already, it needs to be verified and probably fixed again.

The bug has failedQA. Still exist on -
rhvm-4.3.0-0.6.alpha2.el7.noarch
ovirt-provider-ovn-1.2.17-1.el7ev.noarch
rhv-openvswitch-ovn-common-2.10-3.el7ev.noarch
rhv-openvswitch-ovn-central-2.10-3.el7ev.noarch

Note that now, the behavior is even worse then before. After failing to create such network, the ovn provider need a restart, becasue it's no more possible to communicate with him or to create networks on him. Also the OVS database is completely dead. Not sure yet how to recover from this.

TimeoutException: Commands [<ovsdbapp.schema.ovn_northbound.commands.LsAddCommand object at 0x7fe1c4298cd0>] exceeded timeout 100 seconds
TimeoutException: Commands [<ovsdbapp.schema.ovn_northbound.commands.LsAddCommand object at 0x7fe1c4298cd0>] exceeded timeout 100 seconds

Error while executing action AddNetworkOnProvider: Failed to communicate with the external provider, see log for additional details.


{
  "network" : {
    "name" : "רשת_חיצונית",
    "admin_state_up" : true,
    "port_security_enabled" : true
  }
}


Traceback (most recent call last):
  File "/usr/share/ovirt-provider-ovn/ovirt_provider_ovn.py", line 65, in run_with_except_hook
    run_original(*args2, **kwargs2)
  File "/usr/lib64/python2.7/threading.py", line 765, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 95, in run
    self.idl.run()
  File "/usr/lib64/python2.7/site-packages/ovs/db/idl.py", line 230, in run
    msg = self._session.recv()
  File "/usr/lib64/python2.7/site-packages/ovs/jsonrpc.py", line 567, in recv
    error, msg = self.rpc.recv()
  File "/usr/lib64/python2.7/site-packages/ovs/jsonrpc.py", line 301, in recv
    self.input = self.input[self.parser.feed(self.input):]
UnicodeEncodeError: 'ascii' codec can't encode characters in position 158-160: ordinal not in range(128)


2018-12-24 08:17:30,827 root OVS database connection to %(db_schema)s failed with error: '%(error)s'. Verify that OVS and related services are available and that the relevant configuration options are correct.
Traceback (most recent call last):
  File "/usr/share/ovirt-provider-ovn/handlers/base_handler.py", line 134, in _handle_request
    method, path_parts, content
  File "/usr/share/ovirt-provider-ovn/handlers/selecting_handler.py", line 175, in handle_request
    return self.call_response_handler(handler, content, parameters)
  File "/usr/share/ovirt-provider-ovn/handlers/neutron.py", line 35, in call_response_handler
    with NeutronApi() as ovn_north:
  File "/usr/share/ovirt-provider-ovn/neutron/neutron_api.py", line 63, in __init__
    self.ovsidl, self.idl = ovn_connection.connect()
  File "/usr/share/ovirt-provider-ovn/ovn_connection.py", line 48, in connect
    timeout=100
  File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 31, in __init__
    self.start_connection(connection)
  File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 43, in start_connection
    raise connection_exception

Created attachment 1516542 [details]
new ovn log

openvswitch-2.9.0-56 from https://cbs.centos.org/koji/buildinfo?buildID=24381 is working, but unfortunately, this build is not included in ovirt-4.2.8, and ovirt-4.3 requires openvswitch >= 2.10

The issue seems to only affect the C json parser; falling back to the python json parser could be a valid workaround.

This can be achieved by removing the C parser from the filesystem, and restarting the provider:
> rm /usr/lib64/python2.7/site-packages/ovs/_json.so
> systemctl restart ovirt-provider-ovn

Keep in mind that the C json parser is *a lot* faster than the python one.

This requires us to move to ovs-2.11, so I would not rush it.

Michael, does this issue is solved with ovs-2.11?

(In reply to Dominik Holler from comment #11)
> Michael, does this issue is solved with ovs-2.11?

Yes it is solved with 2.11 indeed

Verified on - 4.3.5-0.1.el7 with:
openvswitch2.11-2.11.0-9.el7fdp.x86_64
rhv-openvswitch-2.11-4.el7ev.noarch
rhv-openvswitch-ovn-central-2.11-4.el7ev.noarch
rhv-openvswitch-ovn-common-2.11-4.el7ev.noarch
rhv-python-openvswitch-2.11-4.el7ev.noarch
openvswitch-selinux-extra-policy-1.0-13.el7fdp.noarch
ovn2.11-2.11.0-16.el7fdp.x86_64
ovn2.11-central-2.11.0-16.el7fdp.x86_64
ovirt-provider-ovn-1.2.22-1.el7ev.noarch

This bugzilla is included in oVirt 4.3.5 release, published on July 30th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.5 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.