Bug 1983958

Summary: Impossible to launch a SR-IOV VM since SRIOV agent cannot retrieve port information from server
Product: Red Hat OpenStack Reporter: Roman Safronov <rsafrono>
Component: openstack-neutronAssignee: OSP Team <rhos-maint>
Status: CLOSED DUPLICATE QA Contact: Eran Kuris <ekuris>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 16.2 (Train)CC: chrisw, eolivare, hakhande, jeckersb, lmiccini, oblaut, pgrist, ralonsoh, scohen, supadhya, yrachman
Target Milestone: ---Keywords: AutomationBlocker, Regression, TestBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-22 16:33:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1880120, 1970051    

Description Roman Safronov 2021-07-20 09:47:15 UTC 
Description of problem:
Not possible to launch a VM with SR-IOV port. In SR-IOV agent can be seen that it's cannot retrieve port information from server. RPC calls are not arriving.

[root@computesriov-0 neutron]# grep -nrw "Loop iteration exceeded interval"
sriov-nic-agent.log.1:1347:2021-07-19 13:15:00.651 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 90.15017771720886)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:1518:2021-07-19 13:16:16.655 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 76.00449132919312)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:1695:2021-07-19 13:18:37.329 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 140.67360615730286)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:1716:2021-07-19 13:22:42.743 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 245.41391921043396)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:1816:2021-07-19 13:25:10.070 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 147.32630228996277)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:1894:2021-07-19 13:33:18.967 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 488.8963186740875)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:2559:2021-07-19 13:44:18.703 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 659.7358005046844)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:3443:2021-07-19 13:57:26.718 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 637.9330930709839)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:3830:2021-07-19 14:19:18.326 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 622.917150259018)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:4394:2021-07-19 14:40:44.072 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 621.1504912376404)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:5698:2021-07-19 15:23:32.343 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 602.2097053527832)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501
sriov-nic-agent.log.1:14583:2021-07-19 18:26:08.213 27567 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [req-e6dd617e-ac3f-41a4-9191-147c9bd19e0d - - - - -] Loop iteration exceeded interval (2 vs. 640.2052080631256)! daemon_loop /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py:501


Version-Release number of selected component (if applicable):
RHOS-16.2-RHEL-8-20210713.n.0
python3-neutron-15.3.5-2.20210608154812.el8ost.2.noarch
python3-networking-ovn-7.4.2-2.20210601204819.el8ost.6.noarch
ovn-2021-21.03.0-40.el8fdp.x86_64

How reproducible:
100%

Steps to Reproduce:
1. OSP hybrid setup installed with OVN SRIOV configuration. Make sure that external network, image, flavor, keypair and security group exist.
2. Create SRIOV VF port
example command:
 openstack port create  --network nova --vnic-type direct vf1 --security-group overcloud_sg
3. Create VM using that port
example command:
 openstack server create --flavor rhel-flavor --image rhel-8 --nic port-id=vf1 vm_vf  --key-name test-key

Actual results:
VM not created successfully. VM status is ERROR

Expected results:
Successful VM creation
Comment 9 Yariv 2021-07-21 15:28:35 UTC 
This is NFV blocker,

SRIOV NFV tests, are PASSING with ovs back-end
SRIOV NFV tests, are FAILING with ovn back-end