Description of problem: Nova is not aware NUMA locality of physical NICs when launching a nova instance. For example, when creating an instance nova attempts to pin the instance to a particular NUMA node, however nova is not aware of the locallity of the DPDK enables interfaces and will launch and instance pinned to a particular numa node, even if the virtual nic is local to a different NUMA node.

Because of this proper DKDP tuning requires all interfaces (bonded or not) to be in same NUMA node, and your NovaPinSet must only specify cores from one NUMA node.


Version-Release number of selected component (if applicable):
OSP 11


How reproducible:
Via director, deploy OSP 11 and configure two DPDK enabled interfaces, one local to each NUMA node on a Compute Node.  Configure NovaPinSet to include cores from each NUMA node. Create network port and launch an instance. Nova is not aware of NUMA locality of the DPDK interfaces and can pin an instance to a NUMA node that is not local to the Network Interface


Actual results:

New Instance can be pinned to cores in one NUMA node, and its VIF can be local to another NUMA node.  



Expected results:

Ideally one should be able to include cores from both CPUs (NUMA nodes) and when an instance is launched Nova looks at the NUMA locality of the network and isolates the instance to the correct NUMA node.




Additional info:


Furthermore, In dual socket systems, the only way to ensure that a Nova instance does not cross NUMA boundaries is to set the NovaPinSet to use only the cores from one physical CPUs. The cores from the second NUMA node will have to remain unused by Nova to ensure proper NUMA allignment. 

Note that this configuration will require a DPDK enabled interfaces attached to two separate networks. One network/interface for NUMA node 0 pinned instances, and one network/interface for NUMA node 1 pinned instances. I do not think that it will be possible to bond interfaces in dispirit NUMA nodes due to how network traffic will be routed back to the Compute host when only one network is presented externally.