Bug 1651719

Summary: After FFU SRIOV is not working - neutron-sriov-nic-agent not containerized
Product: Red Hat OpenStack Reporter: Priscila <pveiga>
Component: openstack-neutronAssignee: Assaf Muller <amuller>
Status: CLOSED DUPLICATE QA Contact: Roee Agiman <ragiman>
Severity: high Docs Contact:
Priority: unspecified    
Version: 13.0 (Queens)CC: amuller, bhaley, chrisw, lbezdick, nyechiel, skramaja, yroblamo
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-13 16:20:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Priscila 2018-11-20 16:07:13 UTC 
Description of problem:

In OSPD-10,  SRIOV services are defined in the compute role itself. For the fast forward upgrade, when the undercloud moves to 13 the SRIOV services are no longer present in the compute role. Due to which after running ffwd upgrade on overcloud nodes, all the compute nodes with sriov enabled are ending up as compute nodes with sriov disabled.

Version-Release number of selected component (if applicable):


How reproducible: Always


Steps to Reproduce:
1. Follow https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/fast_forward_upgrades 

2. 
openstack overcloud deploy --templates -e /home/stack/templates/neutron-sriov.yaml

Later, by following the fast forward upgrade document we ran the below commands to upgrade the overcloud to 13. 
 openstack overcloud container image prepare --namespace=registry.access.redhat.com/rhosp13 --prefix=openstack-  --tag-from-label {version}-{release} --output-env-file=/home/stack/templates/overcloud_images.yaml
openstack overcloud ffwd-upgrade prepare --templates -e /home/stack/templates/neutron-sriov.yaml -e /home/stack/templates/overcloud_images.yaml
openstack overcloud ffwd-upgrade run
openstack overcloud upgrade run --nodes Controller --skip-tags validation
openstack overcloud upgrade run --nodes Compute --skip-tags validation

Before starting the upgrade, neutron-sriov-nic-agent on all compute nodes is in running state.
After upgrade, neutron-sriov-nic-agent on all compute nodes is in failed state.

neutron-sriov.yaml for 10
----------------------------------------------------------------------
## A Heat environment that can be used to deploy SR-IOV
resource_registry:
  OS::TripleO::Services::NeutronSriovAgent: /usr/share/openstack-tripleo-heat-templates/puppet/services/neutron-sriov-agent.yaml


parameter_defaults:

  # Add PciPassthroughFilter to the scheduler default filters
  NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter']
  NovaSchedulerAvailableFilters: ['nova.scheduler.filters.all_filters']
  #NovaSchedulerAvailableFilters: ["nova.scheduler.filters.all_filters","nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter"]
  # Provide the vendorid:productid of the VFs
  NeutronSupportedPCIVendorDevs: ['8086:154c','8086:10ca','8086:1520']
  NeutronPhysicalDevMappings: 'physnet1:eno4'
  # Number of VFs that needs to be configured for a physical interface
  NeutronSriovNumVFs: 'eno4:7'
----------------------------------------------------------------------

neutron-sriov.yaml for 13
----------------------------------------------------------------------
## A Heat environment that can be used to deploy SR-IOV
resource_registry:
  OS::TripleO::Services::NeutronSriovAgent: /usr/share/openstack-tripleo-heat-templates/docker/services/neutron-sriov-agent.yaml
  OS::TripleO::Services::NeutronSriovHostConfig: /usr/share/openstack-tripleo-heat-templates/puppet/services/neutron-sriov-host-config.yaml

parameter_defaults:

  # Add PciPassthroughFilter to the scheduler default filters
  NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter']
  NovaSchedulerAvailableFilters: ['nova.scheduler.filters.all_filters']
  #NovaSchedulerAvailableFilters: ["nova.scheduler.filters.all_filters","nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter"]
  # Provide the vendorid:productid of the VFs
  NeutronSupportedPCIVendorDevs: ['8086:154c','8086:10ca','8086:1520']
  NeutronPhysicalDevMappings: 'physnet1:eno4'
  # Number of VFs that needs to be configured for a physical interface
  NeutronSriovNumVFs: 'eno4:7'
----------------------------------------------------------------------

Actual results:

Nov 16 06:57:28 overcloud-compute-0 yum[230208]: Updated: 1:openstack-neutron-sriov-nic-agent-12.0.4-2.el7ost.noarch
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: Traceback (most recent call last):
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/bin/neutron-sriov-nic-agent", line 6, in <module>
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.cmd.eventlet.plugins.sriov_nic_neutron_agent import main
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/cmd/eventlet/plugins/sriov_nic_neutron_agent.py", line 15, in <module>
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: import neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent \
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py", line 36, in <module>
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.agent import rpc as agent_rpc
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/agent/rpc.py", line 27, in <module>
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.agent import resource_cache
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/agent/resource_cache.py", line 20, in <module>
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.api.rpc.callbacks.consumer import registry as registry_rpc
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/api/rpc/callbacks/consumer/registry.py", line 15, in <module>
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.api.rpc.callbacks import resource_manager
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/api/rpc/callbacks/resource_manager.py", line 21, in <module>
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.api.rpc.callbacks import resources
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/api/rpc/callbacks/resources.py", line 29, in <module>
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: NETWORK = network.Network.obj_name()
Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: AttributeError: 'module' object has no attribute 'Network'
Nov 16 07:01:17 overcloud-compute-0 systemd: neutron-sriov-nic-agent.service: main process exited, code=exited, status=1/FAILURE
Nov 16 07:01:17 overcloud-compute-0 systemd: Unit neutron-sriov-nic-agent.service entered failed state.
Nov 16 07:01:17 overcloud-compute-0 systemd: neutron-sriov-nic-agent.service failed.

Expected results:

The expected behavior is to upgrade overcloud from 10 to 13 with compute nodes having sriov enabled.

Additional info: