Bug 1703740

The issue is copy of
https://bugzilla.redhat.com/show_bug.cgi?id=1699839 closed as WORKSFORME. I succeeded to reproduce the issue. The issue occurs only on environment with DVR disabled.

Description of problem:
Impossible to run an instance connected to the external network. Instance created in ERROR state, exception seen in neutron logs (see below)
The issue occurs only on environment without DVR. 

Version-Release number of selected component (if applicable):
13.0-RHEL-7/2019-04-23.1
python-networking-ovn-4.0.3-6.el7ost.noarch
python2-ovsdbapp-0.10.3-1.el7ost.noarch 


How reproducible:
100%

Steps to Reproduce:
1. Create an external network 
(note, I tried to create shared and not shared network, same result)
example command
openstack network create --provider-network-type flat --provider-physical-network datacentre --external external
or for shared one
openstack network create --provider-network-type flat --provider-physical-network datacentre --share --external external

2. Create an external network subnet 
(note, i tried to create with dhcp enabled and not enabled, same result)
openstack subnet create --no-dhcp --gateway 10.0.0.1 --dns-nameserver 10.0.0.1 --network external --subnet-range 10.0.0.0/24 --allocation-pool start=10.0.0.151,end=10.0.0.200 external-subnet
or with dhcp enabled
openstack subnet create --gateway 10.0.0.1 --dns-nameserver 10.0.0.1 --network external --subnet-range 10.0.0.0/24 --allocation-pool start=10.0.0.151,end=10.0.0.200 external-subnet


3. Create security group and try to create an instance
openstack server create --flavor m1.nano --image cirros35 --security-group overcloud_sg   --nic net-id=external  vm_1


Actual results:
Instance created in ERROR state, exception seen in neutron logs (see below)

Expected results:
No exceptions, instance created and state is ACTIVE

Additional info:
from neutron/server.log 

2019-04-25 17:03:00.960 31 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=3): DbRemoveCommand(if_exists=False, column=records, keyvalues={}, record=41f91cc1-d829-4d64-b153-e5017c3d08c7, v
alues=(u'host-10-0-30-154',), table=DNS) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2019-04-25 17:03:00.963 31 ERROR ovsdbapp.backend.ovs_idl.transaction [req-1852f3d0-bb39-4897-8956-d56f9bf0d413 5985bfbf2b74471e8d000a3ff6ff124d 5c8c2f21083a4a6f8be7a631e396ee84 - default default] Traceback (mos
t recent call last):
  File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 99, in run
    txn.results.put(txn.do_commit())
  File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 86, in do_commit
    command.run_idl(txn)
  File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 310, in run_idl
    if isinstance(getattr(record, self.column), dict):
  File "/usr/lib64/python2.7/site-packages/ovs/db/idl.py", line 816, in __getattr__
    del dmap[key]
KeyError: u'host-10-0-30-154.ovn.test'

2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers [req-1852f3d0-bb39-4897-8956-d56f9bf0d413 5985bfbf2b74471e8d000a3ff6ff124d 5c8c2f21083a4a6f8be7a631e396ee84 - default default] Mechanism driver 'ovn' failed in update_port_postcommit: KeyError: u'host-10-0-30-154.ovn.test'
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers Traceback (most recent call last):
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/managers.py", line 427, in _call_on_drivers
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     getattr(driver.obj, method_name)(context)
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/networking_ovn/ml2/mech_driver.py", line 531, in update_port_postcommit
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     self._ovn_client.update_port(port, port_object=original_port)
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/networking_ovn/common/ovn_client.py", line 504, in update_port
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     self.add_txns_to_remove_port_dns_records(txn, port_object)
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     self.gen.next()
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/networking_ovn/ovsdb/impl_idl_ovn.py", line 139, in transaction
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     yield t
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     self.gen.next()
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/ovsdbapp/api.py", line 102, in transaction
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     del self._nested_txns_map[cur_thread_id]
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/ovsdbapp/api.py", line 59, in __exit__
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     self.result = self.commit()
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers   File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 62, in commit
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers     raise result.ex
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers KeyError: u'host-10-0-30-154.ovn.test'
2019-04-25 17:03:00.964 31 ERROR neutron.plugins.ml2.managers 
2019-04-25 17:03:00.966 31 ERROR neutron.plugins.ml2.plugin [req-1852f3d0-bb39-4897-8956-d56f9bf0d413 5985bfbf2b74471e8d000a3ff6ff124d 5c8c2f21083a4a6f8be7a631e396ee84 - default default] mechanism_manager.update_port_postcommit failed for port 9e879f5b-d129-4fcf-8dfd-9d6c1d2feffb: MechanismDriverError
2019-04-25 17:03:00.983 31 DEBUG oslo_concurrency.lockutils [req-1852f3d0-bb39-4897-8956-d56f9bf0d413 5985bfbf2b74471e8d000a3ff6ff124d 5c8c2f21083a4a6f8be7a631e396ee84 - - -] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc.dispatch_events" :: held 0.122s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285


Note, in this specific environment public network uses 10.0.30.0/24 but issue happens on environments with default subnet 10.0.0.0/24 as well, see original issue https://bugzilla.redhat.com/show_bug.cgi?id=1699839