Bug 1042535

Summary: [RFE][neutron]: Fix races in db-based plugin
Product: Red Hat OpenStack Reporter: RHOS Integration <rhos-integ>
Component: RFEsAssignee: RHOS Maint <rhos-maint>
Status: CLOSED UPSTREAM QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: markmc, yeylon
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
URL: https://blueprints.launchpad.net/neutron/+spec/fix-races-of-db-based-plugin
Whiteboard: upstream_milestone_none upstream_status_unknown upstream_definition_obsolete
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-19 17:41:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description RHOS Integration 2013-12-13 00:17:33 UTC
Cloned from launchpad blueprint https://blueprints.launchpad.net/neutron/+spec/fix-races-of-db-based-plugin.

Description:

Fix races in db-based plugin.
Especially error recovery path and rpc notification.

For example typical db-based plug does for network creation,
- super().create_network() = QuantumDbPluginV2.create_network()
  this finishes db transaction.
  After the transaction is committed, the network is visible to outside.
  That is, list-network returns this network.
- then talk to plugin specific controller.
  This can takes a while and can fail.
- If the above fails, start new transaction to delete the network or mark it as error.

This causes race. The way to address is to introduce new status BUILD(already exists) and DELETING.

Another example. From openvswitch plugin. update_port()
- update db
  start transaction and update db.
- query db
- send notification
If two requests to update the same port are sent simaltaneously, it can result in race.
With db locking(row locking), we can assume the transactions are serialized.
But ad the next query to check if notification is necessary results can reorder the threads.
So the notifications can sent in reverse order. 
OF-controller based plugins can talked to their controller during update. So possible way to address this is
- do all under single transaction
or
- serialize the update somehow if necessary

Specification URL (additional information):

https://docs.google.com/file/d/0B4LNMvjOzyDuU2xNd0piS3JBMHM/edit