Bug 1447081

Summary: Backport mtu_request option added to the ovs interface in ovs 2.6 to OSP 10
Product: Red Hat OpenStack Reporter: Andreas Karis <akaris>
Component: python-os-vifAssignee: Sahid Ferdjaoui <sferdjao>
Status: CLOSED ERRATA QA Contact: awaugama
Severity: urgent Docs Contact:
Priority: urgent    
Version: 10.0 (Newton)CC: akaris, awaugama, dcadzow, eglynn, jjoyce, jschluet, jthomas, mlinden, mschuppe, sferdjao, sgordon, slinaber, tvignaud, vcojot
Target Milestone: z4Keywords: FeatureBackport, Triaged, ZStream
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-os-vif-1.2.1-2.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-06 17:06:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1468620    

Description Andreas Karis 2017-05-01 16:36:46 UTC 
Description of problem:
Please backport https://github.com/openstack/os-vif/commit/9a14c18c2163f8f90c797150d12e11b2aad8c1ee

Otherwise, customers which are already going to use OVS 2.6 for OSP 10 won't be able to raise MTU to jumbo frames size on vhost_user_ports

Thanks,

Andreas
Comment 2 Vincent S. Cojot 2017-07-06 18:22:13 UTC 
The upstream patch that I got doesn't apply cleanly to Newton 

https://github.com/openstack/os-vif/commit/01da454fc8f50f2c86200df377dab4d21cecb753.patch

I could attempt a backport but since I don't know this part of the code, chances are I wouldn't be able to verify it properly..

[root@instack site-packages]# patch -p1 < /tmp/cecb753.patch 
patching file vif_plug_ovs/linux_net.py
Hunk #1 FAILED at 75.
Hunk #2 succeeded at 119 (offset -6 lines).
Hunk #3 succeeded at 167 (offset -6 lines).
1 out of 3 hunks FAILED -- saving rejects to file vif_plug_ovs/linux_net.py.rej
patching file vif_plug_ovs/ovs.py
Hunk #1 FAILED at 82.
Hunk #2 succeeded at 80 with fuzz 2 (offset -16 lines).
Hunk #3 FAILED at 130.
2 out of 3 hunks FAILED -- saving rejects to file vif_plug_ovs/ovs.py.rej
can't find file to patch at input line 124
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/vif_plug_ovs/tests/test_plugin.py b/vif_plug_ovs/tests/test_plugin.py
|index d442591..590b831 100644
|--- a/vif_plug_ovs/tests/test_plugin.py
|+++ b/vif_plug_ovs/tests/test_plugin.py
--------------------------
Comment 3 Vincent S. Cojot 2017-07-06 18:50:17 UTC 
Do we also need https://review.openstack.org/#/q/topic:405532?
Comment 12 Andreas Karis 2017-07-12 23:18:28 UTC 
Hi,

I tested this in my virtual lab, and it's not working:

[root@overcloud-compute-0 ~]# grep 2000 /etc/neutron/* -R
/etc/neutron/cisco_device_manager_plugin.ini:# slot_capacity=2000
/etc/neutron/cisco_device_manager_plugin.ini:# slot_capacity=2000
/etc/neutron/cisco_device_manager_plugin.ini:# slot_capacity=2000
/etc/neutron/ml2_conf_cisco.ini:# Example: 100:1000,2000:6000
/etc/neutron/neutron.conf:global_physnet_mtu=2000
/etc/neutron/neutron.conf:#wait_timeout = 2000
/etc/neutron/neutron.conf:#check_timeout = 20000

[root@overcloud-controller-0 osvif]# grep 2000 /etc/neutron/* -R
/etc/neutron/cisco_device_manager_plugin.ini:# slot_capacity=2000
/etc/neutron/cisco_device_manager_plugin.ini:# slot_capacity=2000
/etc/neutron/cisco_device_manager_plugin.ini:# slot_capacity=2000
/etc/neutron/ml2_conf_cisco.ini:# Example: 100:1000,2000:6000
/etc/neutron/neutron.conf:global_physnet_mtu=2000
/etc/neutron/neutron.conf:#wait_timeout = 2000
/etc/neutron/neutron.conf:#check_timeout = 20000


[root@overcloud-compute-0 ~]# rpm -qa | egrep 'neutron|nova'
python-neutron-tests-9.3.1-2.el7ost.noarch
python-nova-14.0.7-6.el7ost.noarch
openstack-nova-conductor-14.0.7-6.el7ost.noarch
openstack-nova-spicehtml5proxy-14.0.7-6.el7ost.noarch
openstack-neutron-9.3.1-2.el7ost.noarch
openstack-neutron-openvswitch-9.3.1-2.el7ost.noarch
openstack-neutron-bigswitch-lldp-9.42.1-3.el7ost.noarch
openstack-nova-cells-14.0.7-6.el7ost.noarch
openstack-nova-14.0.7-6.el7ost.noarch
puppet-nova-9.5.0-6.el7ost.noarch
python-neutronclient-6.0.0-2.el7ost.noarch
python-neutron-lib-0.4.0-1.el7ost.noarch
openstack-neutron-common-9.3.1-2.el7ost.noarch
openstack-neutron-bigswitch-agent-9.42.1-3.el7ost.noarch
python-neutron-lbaas-9.2.0-2.el7ost.noarch
openstack-neutron-ml2-9.3.1-2.el7ost.noarch
openstack-nova-compute-14.0.7-6.el7ost.noarch
openstack-nova-api-14.0.7-6.el7ost.noarch
openstack-nova-placement-api-14.0.7-6.el7ost.noarch
python-nova-tests-14.0.7-6.el7ost.noarch
python-neutron-9.3.1-2.el7ost.noarch
openstack-nova-migration-14.0.7-6.el7ost.noarch
openstack-nova-scheduler-14.0.7-6.el7ost.noarch
openstack-nova-novncproxy-14.0.7-6.el7ost.noarch
openstack-nova-serialproxy-14.0.7-6.el7ost.noarch
openstack-neutron-lbaas-9.2.0-2.el7ost.noarch
openstack-neutron-metering-agent-9.3.1-2.el7ost.noarch
openstack-nova-console-14.0.7-6.el7ost.noarch
openstack-nova-cert-14.0.7-6.el7ost.noarch
python-novaclient-6.0.0-3.el7ost.noarch
openstack-nova-common-14.0.7-6.el7ost.noarch
openstack-nova-network-14.0.7-6.el7ost.noarch
openstack-neutron-sriov-nic-agent-9.3.1-2.el7ost.noarch
puppet-neutron-9.5.0-2.el7ost.noarch


I restarted openstack-nova-compute on the compute node and all neutron and nova on the controller.

[root@overcloud-controller-0 osvif]# rpm -qa | egrep 'neutron|nova'
python-neutron-tests-9.3.1-2.el7ost.noarch
openstack-nova-common-14.0.7-6.el7ost.noarch
openstack-nova-network-14.0.7-6.el7ost.noarch
openstack-neutron-9.3.1-2.el7ost.noarch
openstack-neutron-openvswitch-9.3.1-2.el7ost.noarch
openstack-neutron-bigswitch-lldp-9.42.1-3.el7ost.noarch
openstack-nova-api-14.0.7-6.el7ost.noarch
python-nova-tests-14.0.7-6.el7ost.noarch
puppet-nova-9.5.0-6.el7ost.noarch
python-neutronclient-6.0.0-2.el7ost.noarch
python-neutron-lib-0.4.0-1.el7ost.noarch
openstack-neutron-common-9.3.1-2.el7ost.noarch
openstack-neutron-bigswitch-agent-9.42.1-3.el7ost.noarch
python-neutron-lbaas-9.2.0-2.el7ost.noarch
openstack-neutron-ml2-9.3.1-2.el7ost.noarch
openstack-nova-migration-14.0.7-6.el7ost.noarch
openstack-nova-scheduler-14.0.7-6.el7ost.noarch
openstack-nova-novncproxy-14.0.7-6.el7ost.noarch
openstack-nova-serialproxy-14.0.7-6.el7ost.noarch
python-neutron-9.3.1-2.el7ost.noarch
python-nova-14.0.7-6.el7ost.noarch
openstack-nova-console-14.0.7-6.el7ost.noarch
openstack-nova-conductor-14.0.7-6.el7ost.noarch
openstack-nova-cert-14.0.7-6.el7ost.noarch
openstack-nova-spicehtml5proxy-14.0.7-6.el7ost.noarch
openstack-neutron-lbaas-9.2.0-2.el7ost.noarch
openstack-neutron-metering-agent-9.3.1-2.el7ost.noarch
openstack-nova-cells-14.0.7-6.el7ost.noarch
openstack-nova-14.0.7-6.el7ost.noarch
python-novaclient-6.0.0-3.el7ost.noarch
openstack-nova-compute-14.0.7-6.el7ost.noarch
openstack-nova-placement-api-14.0.7-6.el7ost.noarch
openstack-neutron-sriov-nic-agent-9.3.1-2.el7ost.noarch
puppet-neutron-9.5.0-2.el7ost.noarch
[root@overcloud-controller-0 osvif]# 

crudini --set /etc/nova/nova.conf DEFAULT debug true
crudini --set /etc/neutron/neutron.conf DEFAULT debug true
systemctl list-units | egrep 'neutron|nova' | awk '{print $1}' | xargs -I {} systemctl restart {}

Cleared logs on controller and compute.

for i in /var/log/nova/*.log /var/log/neutron/*.log;do echo $i; > $i ;done

Booted a new server:

[stack@undercloud-4 ~]$ NETID=a3d0cafa-e35b-4835-83c7-d50c97b5756b
[stack@undercloud-4 ~]$  nova boot --nic net-id=$NETID --image rhel --flavor m1.small --key-name id_rsa rhel-test
neutron net-show provider+--------------------------------------+-------------------------------------------------+
| Property                             | Value                                           |
+--------------------------------------+-------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                          |
| OS-EXT-AZ:availability_zone          |                                                 |
| OS-EXT-SRV-ATTR:host                 | -                                               |
| OS-EXT-SRV-ATTR:hostname             | rhel-test                                       |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                               |
| OS-EXT-SRV-ATTR:instance_name        |                                                 |
| OS-EXT-SRV-ATTR:kernel_id            |                                                 |
| OS-EXT-SRV-ATTR:launch_index         | 0                                               |
| OS-EXT-SRV-ATTR:ramdisk_id           |                                                 |
| OS-EXT-SRV-ATTR:reservation_id       | r-og1stb1i                                      |
| OS-EXT-SRV-ATTR:root_device_name     | -                                               |
| OS-EXT-SRV-ATTR:user_data            | -                                               |
| OS-EXT-STS:power_state               | 0                                               |
| OS-EXT-STS:task_state                | scheduling                                      |
| OS-EXT-STS:vm_state                  | building                                        |
| OS-SRV-USG:launched_at               | -                                               |
| OS-SRV-USG:terminated_at             | -                                               |
| accessIPv4                           |                                                 |
| accessIPv6                           |                                                 |
| adminPass                            | 9Uy38KtiaejW                                    |
| config_drive                         |                                                 |
| created                              | 2017-07-12T23:13:57Z                            |
| description                          | -                                               |
| flavor                               | m1.small (e97f54c3-8e95-4baa-bf0d-8dfd1dcbd193) |
| hostId                               |                                                 |
| host_status                          |                                                 |
| id                                   | 715caeb8-2420-4101-a055-22860fc777de            |
| image                                | rhel (1c98dbab-c0ae-4e8a-a90d-e91b787363a6)     |
| key_name                             | id_rsa                                          |
| locked                               | False                                           |
| metadata                             | {}                                              |
| name                                 | rhel-test                                       |
| os-extended-volumes:volumes_attached | []                                              |
| progress                             | 0                                               |
| security_groups                      | default                                         |
| status                               | BUILD                                           |
| tags                                 | []                                              |
| tenant_id                            | cddf29ffccb9427d846fd4949f8caa79                |
| updated                              | 2017-07-12T23:13:58Z                            |
| user_id                              | 9e74abc1832240f4ac793cf2c49cb8b1                |
+--------------------------------------+-------------------------------------------------+
[stack@undercloud-4 ~]$ neutron net-show provider1
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2017-07-11T18:21:26Z                 |
| description               |                                      |
| id                        | a3d0cafa-e35b-4835-83c7-d50c97b5756b |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| is_default                | False                                |
| mtu                       | 2000                                 |
| name                      | provider1                            |
| port_security_enabled     | True                                 |
| project_id                | cddf29ffccb9427d846fd4949f8caa79     |
| provider:network_type     | vlan                                 |
| provider:physical_network | dpdk                                 |
| provider:segmentation_id  | 905                                  |
| qos_policy_id             |                                      |
| revision_number           | 8                                    |
| router:external           | True                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | 5ac3649c-ac56-4055-a286-7c85f6ef9bdc |
|                           | a3617909-bfee-45e0-b25d-13aa4a4ca092 |
| tags                      |                                      |
| tenant_id                 | cddf29ffccb9427d846fd4949f8caa79     |
| updated_at                | 2017-07-11T18:21:38Z                 |
+---------------------------+--------------------------------------+
[stack@undercloud-4 ~]$ sleep 30 ; nova list
+--------------------------------------+-----------+--------+------------+-------------+----------------------------------------------------+
| ID                                   | Name      | Status | Task State | Power State | Networks                                           |
+--------------------------------------+-----------+--------+------------+-------------+----------------------------------------------------+
| 715caeb8-2420-4101-a055-22860fc777de | rhel-test | ACTIVE | -          | Running     | provider1=2000:10::f816:3eff:fe5e:2bfb, 10.0.0.102 |
+--------------------------------------+-----------+--------+------------+-------------+----------------------------------------------------+

ovs database:

[root@overcloud-compute-0 log]# ovs-vsctl show
4ee49e09-b477-43c4-b8a4-c7dcdfa0a1cb
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-link
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port br-link
            Interface br-link
                type: internal
        Port phy-br-link
            Interface phy-br-link
                type: patch
                options: {peer=int-br-link}
        Port "dpdk0"
            Interface "dpdk0"
                type: dpdk
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
        Port int-br-link
            Interface int-br-link
                type: patch
                options: {peer=phy-br-link}
        Port "vhuff839ac7-7a"
            tag: 2
            Interface "vhuff839ac7-7a"
                type: dpdkvhostuser
    ovs_version: "2.6.1"
[root@overcloud-compute-0 log]# ovs-vsctl list Interface vhuff839ac7-7a | grep -i mtumtu                 : 1500
mtu_request         : []


Logs:

[root@overcloud-compute-0 log]# grep 2000 /var/log/nova/*
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.229 6647 DEBUG neutronclient.v2_0.client [req-5ad4c129-2ee3-47fc-8a60-3dbc82370d12 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] GET call to neutron for http://172.16.2.5:9696/v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 used request id req-8757c1a7-5b76-43df-bdef-ceb27436633c _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.563 6647 DEBUG nova.network.base_api [req-5ad4c129-2ee3-47fc-8a60-3dbc82370d12 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] [instance: 715caeb8-2420-4101-a055-22860fc777de] Updating instance_info_cache with network_info: [{"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": false, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null}] update_instance_cache_with_nw_info /usr/lib/python2.7/site-packages/nova/network/base_api.py:43
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.624 6647 DEBUG nova.compute.manager [req-5ad4c129-2ee3-47fc-8a60-3dbc82370d12 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] [instance: 715caeb8-2420-4101-a055-22860fc777de] Instance network_info: |[{"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": false, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null}]| _allocate_network_async /usr/lib/python2.7/site-packages/nova/compute/manager.py:1403
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.626 6647 DEBUG nova.virt.libvirt.driver [req-5ad4c129-2ee3-47fc-8a60-3dbc82370d12 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] [instance: 715caeb8-2420-4101-a055-22860fc777de] Start _get_guest_xml network_info=[{"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": false, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk': {'bus': 'virtio', 'boot_index': '1', 'type': 'disk', 'dev': u'vda'}, 'root': {'bus': 'virtio', 'boot_index': '1', 'type': 'disk', 'dev': u'vda'}}} image_meta=ImageMeta(checksum='37c3b02671c5c392c88a2d3bb33e26b3',container_format='bare',created_at=2017-07-11T19:12:34Z,direct_url=<?>,disk_format='qcow2',id=1c98dbab-c0ae-4e8a-a90d-e91b787363a6,min_disk=0,min_ram=0,name='rhel',owner='cddf29ffccb9427d846fd4949f8caa79',properties=ImageMetaProps,protected=<?>,size=581369856,status='active',tags=<?>,updated_at=2017-07-11T19:12:38Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'swap': None, 'root_device_name': u'/dev/vda', 'ephemerals': [], 'block_device_mapping': []} _get_guest_xml /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:4715
/var/log/nova/nova-compute.log:',key_name='id_rsa',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='overcloud-compute-0.localdomain',locked=False,locked_by=None,memory_mb=1024,metadata={},migration_context=<?>,new_flavor=None,node='overcloud-compute-0.localdomain',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='cddf29ffccb9427d846fd4949f8caa79',ramdisk_id='',reservation_id='r-og1stb1i',root_device_name='/dev/vda',root_gb=16,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={image_base_image_ref='1c98dbab-c0ae-4e8a-a90d-e91b787363a6',image_container_format='bare',image_disk_format='qcow2',image_min_disk='16',image_min_ram='0',network_allocated='True'},tags=<?>,task_state='spawning',terminated_at=None,updated_at=2017-07-12T23:14:00Z,user_data=None,user_id='9e74abc1832240f4ac793cf2c49cb8b1',uuid=715caeb8-2420-4101-a055-22860fc777de,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": false, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null} virt_type=kvm get_config /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:518
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.675 6647 DEBUG nova.network.os_vif_util [req-5ad4c129-2ee3-47fc-8a60-3dbc82370d12 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] Converting VIF {"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": false, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null} nova_to_osvif_vif /usr/lib/python2.7/site-packages/nova/network/os_vif_util.py:421
/var/log/nova/nova-compute.log:',key_name='id_rsa',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='overcloud-compute-0.localdomain',locked=False,locked_by=None,memory_mb=1024,metadata={},migration_context=<?>,new_flavor=None,node='overcloud-compute-0.localdomain',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='cddf29ffccb9427d846fd4949f8caa79',ramdisk_id='',reservation_id='r-og1stb1i',root_device_name='/dev/vda',root_gb=16,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={image_base_image_ref='1c98dbab-c0ae-4e8a-a90d-e91b787363a6',image_container_format='bare',image_disk_format='qcow2',image_min_disk='16',image_min_ram='0',network_allocated='True'},tags=<?>,task_state='spawning',terminated_at=None,updated_at=2017-07-12T23:14:00Z,user_data=None,user_id='9e74abc1832240f4ac793cf2c49cb8b1',uuid=715caeb8-2420-4101-a055-22860fc777de,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": false, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null} plug /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:777
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.683 6647 DEBUG nova.network.os_vif_util [req-5ad4c129-2ee3-47fc-8a60-3dbc82370d12 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] Converting VIF {"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": false, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null} nova_to_osvif_vif /usr/lib/python2.7/site-packages/nova/network/os_vif_util.py:421
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.703 6647 DEBUG oslo.privsep.daemon [-] privsep: request[125578608]: (3, 'vif_plug_ovs.linux_net.create_ovs_vif_port', (u'br-int', 'vhuff839ac7-7a', 'ff839ac7-7a58-4adc-b9f0-34058bb0f5f9', u'fa:16:3e:5e:2b:fb', '715caeb8-2420-4101-a055-22860fc777de', 2000), {'interface_type': 'dpdkvhostuser', 'timeout': 120}) out_of_band /usr/lib/python2.7/site-packages/oslo_privsep/daemon.py:194
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.774 6647 DEBUG oslo_concurrency.processutils [req-5ad4c129-2ee3-47fc-8a60-3dbc82370d12 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set br-int mtu 2000 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:344
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.937 6647 DEBUG oslo_concurrency.processutils [req-5ad4c129-2ee3-47fc-8a60-3dbc82370d12 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] CMD "sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set br-int mtu 2000" returned: 0 in 0.164s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:374
/var/log/nova/nova-compute.log:2017-07-12 23:14:06.577 6647 DEBUG neutronclient.v2_0.client [req-9749e1dd-024d-46cd-b183-28758bb5a79f 3c8be2632af34d7785a6adbd68ea38f8 a5e75eaaa154433185812d21b466b1e1 - - -] GET call to neutron for http://172.16.2.5:9696/v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 used request id req-710067c9-ecb2-4af4-a9ec-604a18c5d7f5 _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127
/var/log/nova/nova-compute.log:2017-07-12 23:14:06.971 6647 DEBUG nova.network.base_api [req-9749e1dd-024d-46cd-b183-28758bb5a79f 3c8be2632af34d7785a6adbd68ea38f8 a5e75eaaa154433185812d21b466b1e1 - - -] [instance: 715caeb8-2420-4101-a055-22860fc777de] Updating instance_info_cache with network_info: [{"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": false, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null}] update_instance_cache_with_nw_info /usr/lib/python2.7/site-packages/nova/network/base_api.py:43
/var/log/nova/nova-compute.log:2017-07-12 23:14:21.075 6647 DEBUG neutronclient.v2_0.client [req-54930f3d-d4e2-4bdf-b24a-d34ebce03740 - - - - -] GET call to neutron for http://172.16.2.5:9696/v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 used request id req-dcdf9cf9-20e2-440a-92db-513e692dd6dd _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127
/var/log/nova/nova-compute.log:2017-07-12 23:14:21.455 6647 DEBUG nova.network.base_api [req-54930f3d-d4e2-4bdf-b24a-d34ebce03740 - - - - -] [instance: 715caeb8-2420-4101-a055-22860fc777de] Updating instance_info_cache with network_info: [{"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": true, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null}] update_instance_cache_with_nw_info /usr/lib/python2.7/site-packages/nova/network/base_api.py:43
/var/log/nova/nova-compute.log:2017-07-12 23:15:16.051 6647 DEBUG neutronclient.v2_0.client [req-54930f3d-d4e2-4bdf-b24a-d34ebce03740 - - - - -] GET call to neutron for http://172.16.2.5:9696/v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 used request id req-39ef60fc-c965-4c47-ac62-a2792ba2a7a9 _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127
/var/log/nova/nova-compute.log:2017-07-12 23:15:16.484 6647 DEBUG nova.network.base_api [req-54930f3d-d4e2-4bdf-b24a-d34ebce03740 - - - - -] [instance: 715caeb8-2420-4101-a055-22860fc777de] Updating instance_info_cache with network_info: [{"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": true, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null}] update_instance_cache_with_nw_info /usr/lib/python2.7/site-packages/nova/network/base_api.py:43
/var/log/nova/nova-compute.log:2017-07-12 23:16:17.163 6647 DEBUG neutronclient.v2_0.client [req-54930f3d-d4e2-4bdf-b24a-d34ebce03740 - - - - -] GET call to neutron for http://172.16.2.5:9696/v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 used request id req-9304b826-e906-4986-8d82-c1133be6a499 _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127
/var/log/nova/nova-compute.log:2017-07-12 23:16:17.492 6647 DEBUG nova.network.base_api [req-54930f3d-d4e2-4bdf-b24a-d34ebce03740 - - - - -] [instance: 715caeb8-2420-4101-a055-22860fc777de] Updating instance_info_cache with network_info: [{"profile": {}, "ovs_interfaceid": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 6, "type": "fixed", "floating_ips": [], "address": "2000:10::f816:3eff:fe5e:2bfb"}], "version": 6, "meta": {"dhcp_server": "2000:10::f816:3eff:fe28:a422"}, "dns": [], "routes": [], "cidr": "2000:10::/64", "gateway": {"meta": {}, "version": 6, "type": "gateway", "address": "2000:10::250"}}, {"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.0.0.102"}], "version": 4, "meta": {"dhcp_server": "10.0.0.100"}, "dns": [{"meta": {}, "version": 4, "type": "dns", "address": "8.8.8.8"}], "routes": [], "cidr": "10.0.0.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cddf29ffccb9427d846fd4949f8caa79", "mtu": 2000}, "id": "a3d0cafa-e35b-4835-83c7-d50c97b5756b", "label": "provider1"}, "devname": "tapff839ac7-7a", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhuff839ac7-7a", "vhostuser_mode": "client"}, "address": "fa:16:3e:5e:2b:fb", "active": true, "type": "vhostuser", "id": "ff839ac7-7a58-4adc-b9f0-34058bb0f5f9", "qbg_params": null}] update_instance_cache_with_nw_info /usr/lib/python2.7/site-packages/nova/network/base_api.py:43
[root@overcloud-compute-0 log]# grep 2000 /var/log/neutron/*
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:13:49.590 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a25f7b,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=172000000,duration_sec=161,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:14:06.193 6590 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] Port ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 updated. Details: {u'profile': {}, u'network_qos_policy_id': None, u'qos_policy_id': None, u'allowed_address_pairs': [], u'admin_state_up': True, u'network_id': u'a3d0cafa-e35b-4835-83c7-d50c97b5756b', u'segmentation_id': 905, u'device_owner': u'compute:None', u'physical_network': u'dpdk', u'mac_address': u'fa:16:3e:5e:2b:fb', u'device': u'ff839ac7-7a58-4adc-b9f0-34058bb0f5f9', u'port_security_enabled': True, u'port_id': u'ff839ac7-7a58-4adc-b9f0-34058bb0f5f9', u'fixed_ips': [{u'subnet_id': u'a3617909-bfee-45e0-b25d-13aa4a4ca092', u'ip_address': u'10.0.0.102'}, {u'subnet_id': u'5ac3649c-ac56-4055-a286-7c85f6ef9bdc', u'ip_address': u'2000:10::f816:3eff:fe5e:2bfb'}], u'network_type': u'vlan'}
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:14:07.359 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0xe,msg_len=0x78,xid=0xc4a25f99,OFPFlowMod(buffer_id=4294967295,command=0,cookie=13833480551765693049L,cookie_mask=0,flags=0,hard_timeout=0,idle_timeout=0,instructions=[OFPInstructionActions(actions=[OFPActionOutput(len=16,max_len=0,port=4294967290,type=0)],len=24,type=4)],match=OFPMatch(oxm_fields={'ip_proto': 58, 'icmpv6_type': 136, 'eth_type': 34525, 'ipv6_nd_target': '2000:10::f816:3eff:fe5e:2bfb', 'in_port': 8}),out_group=0,out_port=0,priority=2,table_id=24) result None _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:14:30.420 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a25fb7,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=2000000,duration_sec=202,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:14:54.431 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a25fcf,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=12000000,duration_sec=226,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:14:58.430 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a25fd3,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=12000000,duration_sec=230,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:15:02.430 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a25fd7,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=12000000,duration_sec=234,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:15:26.438 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a25fef,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=20000000,duration_sec=258,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:15:28.440 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a25ff1,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=22000000,duration_sec=260,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:15:30.440 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a25ff3,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=22000000,duration_sec=262,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:16:02.450 6590 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-d23bdd63-4e85-4ca4-927f-04fa52e3655f - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0xc4a26013,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=13833480551765693049L,duration_nsec=32000000,duration_sec=294,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:93
[root@overcloud-compute-0 log]# 


grep 20[root@overcloud-controller-0 osvif]# grep 2000 /var/log/nova/*.log
/var/log/nova/nova-conductor.log:2017-07-12 23:15:18.161 367634 DEBUG oslo_messaging._drivers.amqpdriver [req-920904f8-f164-4ca4-9cd6-441d92c2cd1f - - - - -] sending reply msg_id: 79119c63de554ac8a386ab8b352be6cf reply queue: reply_b1b7e07cb5a94ebebe7087aad2617098 time elapsed: 0.0520140820008s _send_reply /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:90
[root@overcloud-controller-0 osvif]# grep 2000 /var/log/neutron/*.log
/var/log/neutron/dhcp-agent.log:2017-07-12 23:14:02.535 364758 INFO neutron.agent.dhcp.agent [req-f59fa3b0-4931-4c4e-a19b-e2c5fde8a8c7 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] Trigger reload_allocations for port admin_state_up=True, allowed_address_pairs=[], binding:host_id=, binding:profile=, binding:vif_details=, binding:vif_type=unbound, binding:vnic_type=normal, created_at=2017-07-12T23:14:01Z, description=, device_id=, device_owner=, extra_dhcp_opts=[], fixed_ips=[{u'subnet_id': u'a3617909-bfee-45e0-b25d-13aa4a4ca092', u'ip_address': u'10.0.0.102'}, {u'subnet_id': u'5ac3649c-ac56-4055-a286-7c85f6ef9bdc', u'ip_address': u'2000:10::f816:3eff:fe5e:2bfb'}], id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9, mac_address=fa:16:3e:5e:2b:fb, name=, network_id=a3d0cafa-e35b-4835-83c7-d50c97b5756b, port_security_enabled=True, project_id=cddf29ffccb9427d846fd4949f8caa79, qos_policy_id=None, revision_number=6, security_groups=[u'69c77ff2-e42e-4e3c-8aee-52e017c80b98'], status=DOWN, tenant_id=cddf29ffccb9427d846fd4949f8caa79, updated_at=2017-07-12T23:14:01Z
/var/log/neutron/dhcp-agent.log:2017-07-12 23:14:03.589 364758 INFO neutron.agent.dhcp.agent [req-8d6ac3f8-9286-4f17-9be0-2f36fa4e5378 11e9d25d4e904b4283cb208dbfb1aa35 a5e75eaaa154433185812d21b466b1e1 - - -] Trigger reload_allocations for port admin_state_up=True, allowed_address_pairs=[], binding:host_id=overcloud-compute-0.localdomain, binding:profile=, binding:vif_details=, binding:vif_type=unbound, binding:vnic_type=normal, created_at=2017-07-12T23:14:01Z, description=, device_id=715caeb8-2420-4101-a055-22860fc777de, device_owner=compute:None, extra_dhcp_opts=[], fixed_ips=[{u'subnet_id': u'a3617909-bfee-45e0-b25d-13aa4a4ca092', u'ip_address': u'10.0.0.102'}, {u'subnet_id': u'5ac3649c-ac56-4055-a286-7c85f6ef9bdc', u'ip_address': u'2000:10::f816:3eff:fe5e:2bfb'}], id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9, mac_address=fa:16:3e:5e:2b:fb, name=, network_id=a3d0cafa-e35b-4835-83c7-d50c97b5756b, port_security_enabled=True, project_id=cddf29ffccb9427d846fd4949f8caa79, qos_policy_id=None, revision_number=7, security_groups=[u'69c77ff2-e42e-4e3c-8aee-52e017c80b98'], status=DOWN, tenant_id=cddf29ffccb9427d846fd4949f8caa79, updated_at=2017-07-12T23:14:02Z
/var/log/neutron/server.log:2017-07-12 23:14:01.540 366844 DEBUG neutron.db.db_base_plugin_common [req-f59fa3b0-4931-4c4e-a19b-e2c5fde8a8c7 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] Allocated IP 2000:10::f816:3eff:fe5e:2bfb (a3d0cafa-e35b-4835-83c7-d50c97b5756b/5ac3649c-ac56-4055-a286-7c85f6ef9bdc/ff839ac7-7a58-4adc-b9f0-34058bb0f5f9) _store_ip_allocation /usr/lib/python2.7/site-packages/neutron/db/db_base_plugin_common.py:115
/var/log/neutron/server.log:2017-07-12 23:14:04.227 366844 INFO neutron.wsgi [req-8757c1a7-5b76-43df-bdef-ceb27436633c 11e9d25d4e904b4283cb208dbfb1aa35 a5e75eaaa154433185812d21b466b1e1 - - -] 172.16.2.11 - - [12/Jul/2017 23:14:04] "GET /v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 HTTP/1.1" 200 193 0.030244
/var/log/neutron/server.log:2017-07-12 23:14:06.180 366845 DEBUG neutron.plugins.ml2.rpc [req-fc804438-3cbe-4451-b9fe-dc20592079ef - - - - -] Returning: {'profile': {}, 'network_qos_policy_id': None, 'allowed_address_pairs': [], 'segmentation_id': 905, 'device_owner': u'compute:None', 'device': u'ff839ac7-7a58-4adc-b9f0-34058bb0f5f9', 'port_security_enabled': True, 'fixed_ips': [{'subnet_id': u'a3617909-bfee-45e0-b25d-13aa4a4ca092', 'ip_address': u'10.0.0.102'}, {'subnet_id': u'5ac3649c-ac56-4055-a286-7c85f6ef9bdc', 'ip_address': u'2000:10::f816:3eff:fe5e:2bfb'}], 'qos_policy_id': None, 'admin_state_up': True, 'network_id': u'a3d0cafa-e35b-4835-83c7-d50c97b5756b', 'port_id': u'ff839ac7-7a58-4adc-b9f0-34058bb0f5f9', 'physical_network': u'dpdk', 'mac_address': u'fa:16:3e:5e:2b:fb', 'network_type': u'vlan'} get_device_details /usr/lib/python2.7/site-packages/neutron/plugins/ml2/rpc.py:130
/var/log/neutron/server.log:2017-07-12 23:14:06.576 366844 INFO neutron.wsgi [req-710067c9-ecb2-4af4-a9ec-604a18c5d7f5 11e9d25d4e904b4283cb208dbfb1aa35 a5e75eaaa154433185812d21b466b1e1 - - -] 172.16.2.11 - - [12/Jul/2017 23:14:06] "GET /v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 HTTP/1.1" 200 193 0.021239
/var/log/neutron/server.log:2017-07-12 23:14:21.073 366844 INFO neutron.wsgi [req-dcdf9cf9-20e2-440a-92db-513e692dd6dd 11e9d25d4e904b4283cb208dbfb1aa35 a5e75eaaa154433185812d21b466b1e1 - - -] 172.16.2.11 - - [12/Jul/2017 23:14:21] "GET /v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 HTTP/1.1" 200 193 0.021574
/var/log/neutron/server.log:2017-07-12 23:15:16.049 366844 INFO neutron.wsgi [req-39ef60fc-c965-4c47-ac62-a2792ba2a7a9 11e9d25d4e904b4283cb208dbfb1aa35 a5e75eaaa154433185812d21b466b1e1 - - -] 172.16.2.11 - - [12/Jul/2017 23:15:16] "GET /v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 HTTP/1.1" 200 193 0.046585
/var/log/neutron/server.log:2017-07-12 23:16:17.162 366844 INFO neutron.wsgi [req-9304b826-e906-4986-8d82-c1133be6a499 11e9d25d4e904b4283cb208dbfb1aa35 a5e75eaaa154433185812d21b466b1e1 - - -] 172.16.2.11 - - [12/Jul/2017 23:16:17] "GET /v2.0/floatingips.json?fixed_ip_address=2000%3A10%3A%3Af816%3A3eff%3Afe5e%3A2bfb&port_id=ff839ac7-7a58-4adc-b9f0-34058bb0f5f9 HTTP/1.1" 200 193 0.020877
[root@overcloud-controller-0 osvif]#
Comment 13 Andreas Karis 2017-07-12 23:22:44 UTC 
Note that I'm testing in a virtual lab (KVM in KVM). I already updated the bugreport with the below findings and I hope that engineering can give a few more insights. By the way, my OVS is not crashing, but I'm also not seeing any `set` for the mtu:

[root@overcloud-compute-0 log]# 
[root@overcloud-compute-0 log]# grep -i set /var/log/neutron/* | grep -i inter
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:14:04.725 6590 DEBUG neutron.agent.linux.async_process [-] Output received from [ovsdb-client monitor Interface name,ofport,external_ids --format=json]: {"data":[["2a3f8c7f-fc25-422d-a706-6fb0d24ba156","insert","vhuff839ac7-7a",["set",[]],["map",[["attached-mac","fa:16:3e:5e:2b:fb"],["iface-id","ff839ac7-7a58-4adc-b9f0-34058bb0f5f9"],["iface-status","active"],["vm-uuid","715caeb8-2420-4101-a055-22860fc777de"]]]]],"headings":["row","action","name","ofport","external_ids"]} _read_stdout /usr/lib/python2.7/site-packages/neutron/agent/linux/async_process.py:233
/var/log/neutron/openvswitch-agent.log:2017-07-12 23:14:04.776 6590 DEBUG neutron.agent.linux.async_process [-] Output received from [ovsdb-client monitor Interface name,ofport,external_ids --format=json]: {"data":[["2a3f8c7f-fc25-422d-a706-6fb0d24ba156","old",null,["set",[]],null],["","new","vhuff839ac7-7a",8,["map",[["attached-mac","fa:16:3e:5e:2b:fb"],["iface-id","ff839ac7-7a58-4adc-b9f0-34058bb0f5f9"],["iface-status","active"],["vm-uuid","715caeb8-2420-4101-a055-22860fc777de"]]]]],"headings":["row","action","name","ofport","external_ids"]} _read_stdout /usr/lib/python2.7/site-packages/neutron/agent/linux/async_process.py:233


-- Andreas
Comment 15 Andreas Karis 2017-07-13 14:28:04 UTC 
Hi,

It indeed seems to be that line 170:
~~~
[root@overcloud-compute-0 ~]# grep 'MTU not' /var/log/nova/*
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.772 6647 DEBUG oslo.privsep.daemon [-] MTU not set on vhuff839ac7-7a interface of type dpdkvhostuser. out_of_band /usr/lib/python2.7/site-packages/oslo_privsep/daemon.py:194
~~~

I have this in the logs. Testing this now.

- Andreas
Comment 16 Andreas Karis 2017-07-13 14:53:21 UTC 
I retested in my lab. In my lab, this failed due to:
~~~
[root@overcloud-compute-0 ~]# grep 'MTU not' /var/log/nova/*
/var/log/nova/nova-compute.log:2017-07-12 23:14:04.772 6647 DEBUG oslo.privsep.daemon [-] MTU not set on vhuff839ac7-7a interface of type dpdkvhostuser. out_of_band /usr/lib/python2.7/site-packages/oslo_privsep/daemon.py:194
~~~

Which comes from line 170 in here:
vif_plug_ovs/linux_net.py
~~~
 139         processutils.execute('brctl', 'stp', bridge, 'off')
 140         syspath = '/sys/class/net/%s/bridge/multicast_snooping'
 141         syspath = syspath % bridge
 142         processutils.execute('tee', syspath, process_input='0',
 143                              check_exit_code=[0, 1])
 144         disv6 = ('/proc/sys/net/ipv6/conf/%s/disable_ipv6' %
 145                  bridge)
 146         if os.path.exists(disv6):
 147             processutils.execute('tee',
 148                                  disv6,
 149                                  process_input='1',
 150                                  check_exit_code=[0, 1])
 151 
 152 
 153 @privsep.vif_plug.entrypoint
 154 def delete_bridge(bridge, dev):
 155     if device_exists(bridge):
 156         processutils.execute('brctl', 'delif', bridge, dev)
 157         processutils.execute('ip', 'link', 'set', bridge, 'down')
 158         processutils.execute('brctl', 'delbr', bridge)
 159 
 160 
 161 @privsep.vif_plug.entrypoint
 162 def add_bridge_port(bridge, dev):
 163     processutils.execute('ip', 'link', 'set', bridge, 'up')
 164     processutils.execute('brctl', 'addif', bridge, dev)
 165 
 166 
 167 def _update_device_mtu(dev, mtu, interface_type=None):
 168     # Note at present there is no support for setting the
 169     # mtu for vhost-user type ports.
 170     if mtu and interface_type != constants.OVS_VHOSTUSER_INTERFACE_TYPE:
 171         _set_device_mtu(dev, mtu)
 172     else:
 173         LOG.debug("MTU not set on %(interface_name)s interface "
 174                   "of type %(interface_type)s.",
 175                   {'interface_name': dev,
 176                    'interface_type': interface_type})
 177 
 178 
 179 def _set_device_mtu(dev, mtu):
 180     """Set the device MTU."""
 181     processutils.execute('ip', 'link', 'set', dev, 'mtu', mtu,
~~~

I modified the following code on the compute:
~~~
[root@overcloud-compute-0 ~]# diff -u /usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py{.bck,}
--- /usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py.bck	2017-07-13 14:46:45.446696319 +0000
+++ /usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py	2017-07-13 14:47:06.768621542 +0000
@@ -167,7 +167,8 @@
 def _update_device_mtu(dev, mtu, interface_type=None):
     # Note at present there is no support for setting the
     # mtu for vhost-user type ports.
-    if mtu and interface_type != constants.OVS_VHOSTUSER_INTERFACE_TYPE:
+    if mtu:
+         # and interface_type != constants.OVS_VHOSTUSER_INTERFACE_TYPE:
         _set_device_mtu(dev, mtu)
     else:
         LOG.debug("MTU not set on %(interface_name)s interface "
~~~

I restarted all nova and neutron:
~~~
systemctl list-units | egrep 'neutron|nova' | awk '{print $1}' | xargs -I {} systemctl restart {}
~~~

And then I got:
~~~
2017-07-13 14:51:51.895 86118 DEBUG oslo_messaging._drivers.amqpdriver [req-9b101426-e638-43ca-a81a-03266c489c2c 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] CAST unique_id: 164f7321e0c44e38aa4f277e24fc5561 NOTIFY exchange 'nova' topic 'notifications.error' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:479
2017-07-13 14:51:51.898 86118 DEBUG nova.compute.manager [req-9b101426-e638-43ca-a81a-03266c489c2c 9e74abc1832240f4ac793cf2c49cb8b1 cddf29ffccb9427d846fd4949f8caa79 - - -] [instance: e578ebe0-a119-484d-bc57-b89911d0ecaa] Build of instance e578ebe0-a119-484d-bc57-b89911d0ecaa was re-scheduled: Failure running os_vif plugin plug method: Failed to plug VIF VIFVHostUser(active=False,address=fa:16:3e:4f:14:d3,has_traffic_filtering=False,id=02e5eba8-6e11-4574-98c7-72ff21e60e20,mode='client',network=Network(a3d0cafa-e35b-4835-83c7-d50c97b5756b),path='/var/run/openvswitch/vhu02e5eba8-6e',plugin='ovs',port_profile=VIFPortProfileBase,preserve_on_delete=False). Got error: Unexpected error while running command.
Command: ip link set vhu02e5eba8-6e mtu 2000
Exit code: 1
Stdout: u''
Stderr: u'Cannot find device "vhu02e5eba8-6e"\n' _do_build_and_run_instance /usr/lib/python2.7/site-packages/nova/compute/manager.py:1802
~~~

Which obviously means that this is trying to trigger ip link set and not the DPDK command.

- Andreas
Comment 17 Andreas Karis 2017-07-13 14:54:56 UTC 
Which is logical, because:
~~~
 179 def _set_device_mtu(dev, mtu):
 180     """Set the device MTU."""
 181     processutils.execute('ip', 'link', 'set', dev, 'mtu', mtu,
 182                          check_exit_code=[0, 2, 254])
~~~
Comment 18 Andreas Karis 2017-07-13 14:57:44 UTC 
I just looked at the code, and it contains none of these changes:
https://github.com/openstack/os-vif/commit/9a14c18c2163f8f90c797150d12e11b2aad8c1ee
Comment 19 Andreas Karis 2017-07-13 14:58:57 UTC 
E.g.:

+def _update_device_mtu(dev, mtu, interface_type=None, timeout=120):
+    if not mtu:
+        return
+    if interface_type not in [
+        constants.OVS_VHOSTUSER_INTERFACE_TYPE,
+        constants.OVS_VHOSTUSER_CLIENT_INTERFACE_TYPE]:
         _set_device_mtu(dev, mtu)
+    elif _ovs_supports_mtu_requests(timeout=timeout):
+        _set_mtu_request(dev, mtu, timeout=timeout)
     else:
         LOG.debug("MTU not set on %(interface_name)s interface "
                   "of type %(interface_type)s.",
                   {'interface_name': dev,
                    'interface_type': interface_type})

vs what we currently have:
~~~
167 def _update_device_mtu(dev, mtu, interface_type=None):
 168     # Note at present there is no support for setting the
 169     # mtu for vhost-user type ports.
 170     if mtu and interface_type != constants.OVS_VHOSTUSER_INTERFACE_TYPE:
 171         _set_device_mtu(dev, mtu)
 172     else:
 173         LOG.debug("MTU not set on %(interface_name)s interface "
 174                   "of type %(interface_type)s.",
 175                   {'interface_name': dev,
 176                    'interface_type': interface_type})
~~~
Comment 43 errata-xmlrpc 2017-09-06 17:06:29 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2653