Description of problem: In OSPD-10, SRIOV services are defined in the compute role itself. For the fast forward upgrade, when the undercloud moves to 13 the SRIOV services are no longer present in the compute role. Due to which after running ffwd upgrade on overcloud nodes, all the compute nodes with sriov enabled are ending up as compute nodes with sriov disabled. Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. Follow https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/fast_forward_upgrades 2. openstack overcloud deploy --templates -e /home/stack/templates/neutron-sriov.yaml Later, by following the fast forward upgrade document we ran the below commands to upgrade the overcloud to 13. openstack overcloud container image prepare --namespace=registry.access.redhat.com/rhosp13 --prefix=openstack- --tag-from-label {version}-{release} --output-env-file=/home/stack/templates/overcloud_images.yaml openstack overcloud ffwd-upgrade prepare --templates -e /home/stack/templates/neutron-sriov.yaml -e /home/stack/templates/overcloud_images.yaml openstack overcloud ffwd-upgrade run openstack overcloud upgrade run --nodes Controller --skip-tags validation openstack overcloud upgrade run --nodes Compute --skip-tags validation Before starting the upgrade, neutron-sriov-nic-agent on all compute nodes is in running state. After upgrade, neutron-sriov-nic-agent on all compute nodes is in failed state. neutron-sriov.yaml for 10 ---------------------------------------------------------------------- ## A Heat environment that can be used to deploy SR-IOV resource_registry: OS::TripleO::Services::NeutronSriovAgent: /usr/share/openstack-tripleo-heat-templates/puppet/services/neutron-sriov-agent.yaml parameter_defaults: # Add PciPassthroughFilter to the scheduler default filters NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter'] NovaSchedulerAvailableFilters: ['nova.scheduler.filters.all_filters'] #NovaSchedulerAvailableFilters: ["nova.scheduler.filters.all_filters","nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter"] # Provide the vendorid:productid of the VFs NeutronSupportedPCIVendorDevs: ['8086:154c','8086:10ca','8086:1520'] NeutronPhysicalDevMappings: 'physnet1:eno4' # Number of VFs that needs to be configured for a physical interface NeutronSriovNumVFs: 'eno4:7' ---------------------------------------------------------------------- neutron-sriov.yaml for 13 ---------------------------------------------------------------------- ## A Heat environment that can be used to deploy SR-IOV resource_registry: OS::TripleO::Services::NeutronSriovAgent: /usr/share/openstack-tripleo-heat-templates/docker/services/neutron-sriov-agent.yaml OS::TripleO::Services::NeutronSriovHostConfig: /usr/share/openstack-tripleo-heat-templates/puppet/services/neutron-sriov-host-config.yaml parameter_defaults: # Add PciPassthroughFilter to the scheduler default filters NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter'] NovaSchedulerAvailableFilters: ['nova.scheduler.filters.all_filters'] #NovaSchedulerAvailableFilters: ["nova.scheduler.filters.all_filters","nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter"] # Provide the vendorid:productid of the VFs NeutronSupportedPCIVendorDevs: ['8086:154c','8086:10ca','8086:1520'] NeutronPhysicalDevMappings: 'physnet1:eno4' # Number of VFs that needs to be configured for a physical interface NeutronSriovNumVFs: 'eno4:7' ---------------------------------------------------------------------- Actual results: Nov 16 06:57:28 overcloud-compute-0 yum[230208]: Updated: 1:openstack-neutron-sriov-nic-agent-12.0.4-2.el7ost.noarch Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: Traceback (most recent call last): Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/bin/neutron-sriov-nic-agent", line 6, in <module> Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.cmd.eventlet.plugins.sriov_nic_neutron_agent import main Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/cmd/eventlet/plugins/sriov_nic_neutron_agent.py", line 15, in <module> Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: import neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent \ Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py", line 36, in <module> Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.agent import rpc as agent_rpc Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/agent/rpc.py", line 27, in <module> Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.agent import resource_cache Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/agent/resource_cache.py", line 20, in <module> Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.api.rpc.callbacks.consumer import registry as registry_rpc Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/api/rpc/callbacks/consumer/registry.py", line 15, in <module> Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.api.rpc.callbacks import resource_manager Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/api/rpc/callbacks/resource_manager.py", line 21, in <module> Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: from neutron.api.rpc.callbacks import resources Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: File "/usr/lib/python2.7/site-packages/neutron/api/rpc/callbacks/resources.py", line 29, in <module> Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: NETWORK = network.Network.obj_name() Nov 16 07:01:16 overcloud-compute-0 neutron-sriov-nic-agent: AttributeError: 'module' object has no attribute 'Network' Nov 16 07:01:17 overcloud-compute-0 systemd: neutron-sriov-nic-agent.service: main process exited, code=exited, status=1/FAILURE Nov 16 07:01:17 overcloud-compute-0 systemd: Unit neutron-sriov-nic-agent.service entered failed state. Nov 16 07:01:17 overcloud-compute-0 systemd: neutron-sriov-nic-agent.service failed. Expected results: The expected behavior is to upgrade overcloud from 10 to 13 with compute nodes having sriov enabled. Additional info: