RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2105141 - kubernetes-nmstate-operator deletes Virtual Functions created by sriov-fec-operator
Summary: kubernetes-nmstate-operator deletes Virtual Functions created by sriov-fec-op...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: nmstate
Version: 8.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Gris Ge
QA Contact: Mingyu Shi
URL:
Whiteboard: Telco: RAN
: 2105884 (view as bug list)
Depends On:
Blocks: 2139698 2139699 2139701
TreeView+ depends on / blocked
 
Reported: 2022-07-08 04:19 UTC by Yuki Okada
Modified: 2023-05-16 09:24 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 2139698 2139699 2139701 (view as bug list)
Environment:
Last Closed: 2023-05-16 08:26:38 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker NMT-384 0 None None None 2023-03-14 15:46:45 UTC
Red Hat Issue Tracker RHELPLAN-129541 0 None None None 2022-07-28 21:54:11 UTC
Red Hat Product Errata RHBA-2023:2772 0 None None None 2023-05-16 08:27:16 UTC

Description Yuki Okada 2022-07-08 04:19:35 UTC 
Description of problem:
sriov-fec-operator[1][2], which is provided by Intel, provides a functionality to creates Virtual Functions for Forward Error Correction(FEC) devices.
But those Virtual Functions are deleted when kubernetes-nmstate-operator is installed and configured.

[1] https://catalog.redhat.com/software/operators/detail/6001a748e4e3f23b0b6ad765
[2] https://github.com/smart-edge-open/sriov-fec-operator/blob/main/spec/sriov-fec-operator.md

Version-Release number of selected component (if applicable):
OCP 4.10.13
sriov-fec.v2.2.1
kubernetes-nmstate-operator.4.10.0-202205120735

How reproducible:
100% at customer's site

Steps to Reproduce:
1. Install "SEO SR-IOV Operator for Wireless FEC Accelerators (sriov-fec-operator)" from OperatorHub
2. Configure "SriovFecClusterConfig" resource as explained in https://github.com/smart-edge-open/sriov-fec-operator/blob/main/spec/sriov-fec-operator.md#acc100-fec
3. See if virtual functions are created

  $ oc get sriovfecnodeconfig <nodename> -o yaml    # Sample status is here: https://github.com/smart-edge-open/sriov-fec-operator/blob/main/spec/sriov-fec-operator.md#sample-status-for-wireless-fec-acc100
  $ oc describe node <nodename>                     # You will see "intel.com/intel_fec_acc100" entry in Capacity section.

4. Install "Kubernetes NMState Operator" from OperatorHub
5. Apply "NodeNetworkConfigurationPolicy" manifest to create an interface on nodes https://docs.openshift.com/container-platform/4.10/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.html#virt-creating-interface-on-nodes_k8s_nmstate-updating-node-network-config
6. Restart nodes
7. See if virtual functions exist or not using the same commands as 3.

Actual results:
virtual functions created by sriov-fec-operator are deleted when kubernetes-nmstate-operator is configured.

Expected results:
virtual functions are NOT deleted.

Additional info:
Comment 4 Ben Nemec 2022-07-28 21:49:47 UTC 
*** Bug 2105884 has been marked as a duplicate of this bug. ***
Comment 5 Ben Nemec 2022-07-28 21:52:18 UTC 
Moving to nmstate team since we think this will require changes there to avoid stepping on the sriov configurations.
Comment 9 Gris Ge 2022-09-14 13:55:45 UTC 
This might be caused by nmstate-1.0 try to crate NetworkManager connections for SR-IOV VFs even they are not mentioned.

Still debuging.
Comment 10 Gris Ge 2022-09-14 14:59:07 UTC 
Hi Yuki Okada,

Can we have customer try these rpm ?

https://people.redhat.com/fge/bz_2105141/nmstate-1.0.2-19.bz2105141.el8_4.noarch.rpm
https://people.redhat.com/fge/bz_2105141/python3-libnmstate-1.0.2-19.bz2105141.el8_4.noarch.rpm
Comment 14 Gris Ge 2022-10-09 05:37:03 UTC 
Patch been merged in upstream: https://github.com/nmstate/nmstate/pull/2042
Comment 23 Mingyu Shi 2022-11-02 07:30:15 UTC 
Also tested driver: mlx5_core, i40e, ixgbe.
On these drivers, when applying new state without `sr-iov` subtree, everying works fine with both old or new nmstate -- the VFs created in sysfs are preserved.

When removing the interface (state: absent), they behaves differently:
The older nmstate would delete VF too, while the new nmstate(in #comment21) would preserve them.
Comment 24 Gris Ge 2022-11-02 07:35:14 UTC 
Yes. That's expected behaviour.
Comment 39 errata-xmlrpc 2023-05-16 08:26:38 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2772
Note You need to log in before you can comment on or make changes to this bug.