Bug 1839717

Summary:

OVN ML2 driver driver error: no attribute 'OVN_NEUTRON_OWNER_TO_PORT_TYPE'

Product:

Red Hat OpenStack

Reporter:

Bruna Bonguardo <bbonguar>

Component:

python-networking-ovn

Assignee:

Maciej Józefczyk <mjozefcz>

Status:

CLOSED ERRATA

QA Contact:

Roman Safronov <rsafrono>

Severity:

urgent

Docs Contact:

Priority:

urgent

Version:

13.0 (Queens)

CC:

apevec, cgoncalves, ihrachys, lhh, lpeer, majopela, mgarciac, mjozefcz, rsafrono, scohen, spower

Target Milestone:

z12

Keywords:

Triaged, ZStream

Target Release:

13.0 (Queens)

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

python-networking-ovn-4.0.4-6.el7ost

Doc Type:

No Doc Update

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2020-06-24 11:53:09 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

Bug Blocks:

1789836, 1830734

Attachments:

Description	Flags
Log: home/stack/overcloud_install.log	none

Description Bruna Bonguardo 2020-05-25 10:20:53 UTC

Description of problem:

When deploying Openstack osp13 z12 with Customized Job (https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/OSPD-Customized-Deployment-virt/), overcloud is failing with error:

Failure caused by error in tasks: octavia_post_workflow -> Failure caused by error in tasks: config_octavia

Version-Release number of selected component (if applicable):
osp13 core_puddle=2020-05-19.2


Steps to Reproduce:
1. Deploy OSP13 with https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/OSPD-Customized-Deployment-virt/
2. Add the octavia.yaml file to overcloud_templates.
3. The deployment fails on the "Overcloud" install stage.

Actual results:
home/stack/overcloud_install.log: http://pastebin.test.redhat.com/868181

Additional info:
I deployed the same deployment again, but without Octavia, to confirm if it is indeed because of Octavia the deployment is failing.

Comment 1 Carlos Goncalves 2020-05-25 10:29:28 UTC

Neutron failed to create port:
  Request Failed: internal server error while processing your request.", "Neutron server returns request_ids: [\'req-826f3ad5-ce2b-4a1c-aecf-01b8c6ed39b9\']"

Could you please share Neutron logs from overcloud?
This may be a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1797229

Comment 2 Carlos Goncalves 2020-05-25 10:36:30 UTC

I found the job build in https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/OSPD-Customized-Deployment-virt/15550 and had a look at the sosreports.

2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers [req-826f3ad5-ce2b-4a1c-aecf-01b8c6ed39b9 e248af562ffd4ef4a7d070948f99260e ccbdda43fa594af2a6fc815f69f025e1 - default default] Mechanism driver 'ovn' failed in create_port_post
commit: AttributeError: 'module' object has no attribute 'OVN_NEUTRON_OWNER_TO_PORT_TYPE'
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers Traceback (most recent call last):
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/managers.py", line 455, in _call_on_drivers
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers     getattr(driver.obj, method_name)(context)
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/networking_ovn/ml2/mech_driver.py", line 477, in create_port_postcommit
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers     self._ovn_client.create_port(port)
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/networking_ovn/common/ovn_client.py", line 261, in create_port
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers     port_info = self._get_port_options(port)
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/networking_ovn/common/ovn_client.py", line 243, in _get_port_options
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers     port_type = ovn_const.OVN_NEUTRON_OWNER_TO_PORT_TYPE.get(
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers AttributeError: 'module' object has no attribute 'OVN_NEUTRON_OWNER_TO_PORT_TYPE'
2020-05-24 16:51:23.809 29 ERROR neutron.plugins.ml2.managers 
2020-05-24 16:51:23.811 29 ERROR neutron.plugins.ml2.plugin [req-826f3ad5-ce2b-4a1c-aecf-01b8c6ed39b9 e248af562ffd4ef4a7d070948f99260e ccbdda43fa594af2a6fc815f69f025e1 - default default] mechanism_manager.create_port_postcommit failed, de
leting port '8be41630-e07b-43cf-a3b5-38b75f07b6cc': MechanismDriverError
[...]
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation [req-826f3ad5-ce2b-4a1c-aecf-01b8c6ed39b9 e248af562ffd4ef4a7d070948f99260e ccbdda43fa594af2a6fc815f69f025e1 - default default] POST failed.: MechanismDriverError
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last):
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/pecan/core.py", line 678, in __call__
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.invoke_controller(controller, args, kwargs, state)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/pecan/core.py", line 569, in invoke_controller
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     result = controller(*args, **kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 91, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     setattr(e, '_RETRY_EXCEEDED', True)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 87, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 147, in wrapper
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     ectxt.value = e.inner_exc
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 135, in wrapper
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 126, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     LOG.debug("Retry wrapper got retriable exception: %s", e)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 122, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return f(*dup_args, **dup_kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/pecan_wsgi/controllers/utils.py", line 76, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 159, in post
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return self.create(resources)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 177, in create
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return {key: creator(*creator_args, **creator_kwargs)}
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 638, in inner
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return f(self, context, *args, **kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 161, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return method(*args, **kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 91, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     setattr(e, '_RETRY_EXCEEDED', True)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 87, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 147, in wrapper
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     ectxt.value = e.inner_exc
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 135, in wrapper
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 126, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     LOG.debug("Retry wrapper got retriable exception: %s", e)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 122, in wrapped
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return f(*dup_args, **dup_kwargs)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py", line 1356, in create_port
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     return self._after_create_port(context, result, mech_context)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py", line 1369, in _after_create_port
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.delete_port(context, result['id'], l3_port_check=False)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py", line 1364, in _after_create_port
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self.mechanism_manager.create_port_postcommit(mech_context)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/managers.py", line 693, in create_port_postcommit
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     self._call_on_drivers("create_port_postcommit", context)
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/managers.py", line 473, in _call_on_drivers
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation     errors=errors
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation MechanismDriverError
2020-05-24 16:51:24.422 29 ERROR neutron.pecan_wsgi.hooks.translation 
2020-05-24 16:51:24.425 29 DEBUG neutron.pecan_wsgi.hooks.notifier [req-826f3ad5-ce2b-4a1c-aecf-01b8c6ed39b9 e248af562ffd4ef4a7d070948f99260e ccbdda43fa594af2a6fc815f69f025e1 - default default] No notification will be sent due to unsuccessful status code: 500 after /usr/lib/python2.7/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79
2020-05-24 16:51:24.463 29 INFO neutron.wsgi [req-826f3ad5-ce2b-4a1c-aecf-01b8c6ed39b9 e248af562ffd4ef4a7d070948f99260e ccbdda43fa594af2a6fc815f69f025e1 - default default] 172.17.1.137 "POST /v2.0/ports HTTP/1.1" status: 500  len: 344 time: 2.4275782

Comment 3 Bruna Bonguardo 2020-05-25 10:55:07 UTC

Created attachment 1691901 [details]
Log: home/stack/overcloud_install.log

Comment 5 Maciej Józefczyk 2020-05-25 13:23:07 UTC

After recent pull from u/s some change deleted constant that has been used by downstream only cherry-pick from stable/rocky to downstream stable/stein.

Fix for that is included in python-networking-ovn-4.0.4-6.el7ost.

Comment 8 Jakub Libosvar 2020-05-26 09:13:27 UTC

*** Bug 1839811 has been marked as a duplicate of this bug. ***

Comment 14 Roman Safronov 2020-06-02 17:03:28 UTC

Verified on  2020-05-28.2 with  python-networking-ovn-4.0.4-6.el7ost.noarch
Verified that the issue does not occur. Environment can be installed successfully and no OVN_NEUTRON_OWNER_TO_PORT_TYPE ML2 driver error or other errors in neutron server logs after creating networks, routers and launching instances.

Comment 17 errata-xmlrpc 2020-06-24 11:53:09 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2724