RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2151624 - SR-IOV VF parameter configuration fails
Summary: SR-IOV VF parameter configuration fails
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: nmstate
Version: 9.0
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: rc
: ---
Assignee: Gris Ge
QA Contact: Mingyu Shi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-12-07 16:30 UTC by Carlos Goncalves
Modified: 2023-12-12 03:59 UTC (History)
6 users (show)

Fixed In Version: nmstate-2.2.3-2.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-09 07:31:50 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
journalctl log (14.39 KB, text/plain)
2022-12-07 16:31 UTC, Carlos Goncalves 		no flags Details
knmstate policy failure log (3.88 KB, text/plain)
2022-12-07 16:31 UTC, Carlos Goncalves 		no flags Details
knmstate policy CR (376 bytes, text/plain)
2022-12-07 16:32 UTC, Carlos Goncalves 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Github nmstate nmstate pull 2165 0 None open sriov: Support modify only defined VF 2022-12-19 09:36:53 UTC
Red Hat Issue Tracker NMT-126 0 None None None 2023-01-22 15:28:58 UTC
Red Hat Issue Tracker RHELPLAN-141579 0 None None None 2022-12-07 16:43:49 UTC
Red Hat Product Errata RHBA-2023:2190 0 None None None 2023-05-09 07:32:04 UTC

Description Carlos Goncalves 2022-12-07 16:30:22 UTC 
Description of problem:
As an OpenShift user, I want to configure my SR-IOV NIC with 4 virtual functions and set the fourth one (index 3) with trust on. I use Kubernetes-nmstate with the desired network configuration (see NodeNetworkConfigurationPolicy CR file attached). nmstatectl returns an error.

For about a minute, the four VFs are created and VF 3 is set to trust on as desired:

# ip link show ens1f0
2: ens1f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 64:4c:36:11:c6:e0 brd ff:ff:ff:ff:ff:ff
    vf 0     link/ether 6a:ec:6b:a5:03:ea brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 1     link/ether 1a:07:3e:b9:34:e3 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 2     link/ether 86:70:73:75:d3:27 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 3     link/ether a2:93:0d:97:64:0c brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust on

This configuration is rolled back by NetworkManager in approximately 1 minute.


Version-Release number of selected component (if applicable):
- Red Hat Enterprise Linux CoreOS 412.86.202212030032-0
- Kernel 4.18.0-372.32.1.el8_6.x86_64
- OpenShift 4.12.0-0.nightly-2022-12-03-062812

How reproducible:
100%

Steps to Reproduce:
1. oc apply -f knmstate-sriov-vf3-trust-policy.yaml
2.
3.

Actual results:
See journalctl and k-nmstate failure log files attached.

# oc get nnce                                                                                                                                                                                                
NAME                                    STATUS    REASON                                                                                                                                                                                               
cnfdc8-worker-1sriov-vf3-trust-policy   Failing   FailedToConfigure
Comment 1 Carlos Goncalves 2022-12-07 16:31:04 UTC 
Created attachment 1930858 [details]
journalctl log
Comment 2 Carlos Goncalves 2022-12-07 16:31:38 UTC 
Created attachment 1930859 [details]
knmstate policy failure log
Comment 3 Carlos Goncalves 2022-12-07 16:32:14 UTC 
Created attachment 1930860 [details]
knmstate policy CR
Comment 5 Gris Ge 2022-12-13 12:51:41 UTC 
Hi Carlos,

Could you try:

```
apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: sriov-vf3-trust-policy
spec:
  nodeSelector:
    kubernetes.io/hostname: "cnfdc8-worker-1"
  desiredState:
    interfaces:
      - name: ens1f0
        type: ethernet
        ethernet:
          sr-iov:
            total-vfs: 4
            vfs:
              - id: 0
              - id: 1
              - id: 2
              - id: 3
                trust: true
```
Comment 6 Carlos Goncalves 2022-12-15 11:58:44 UTC 
Gris, that helped.

# oc get nncp
NAME                     STATUS      REASON
sriov-vf3-trust-policy   Available   SuccessfullyConfigured

# ip link show ens1f0
2: ens1f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 64:4c:36:11:c6:e0 brd ff:ff:ff:ff:ff:ff
    vf 0     link/ether 6a:ec:6b:a5:03:ea brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 1     link/ether 1a:07:3e:b9:34:e3 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 2     link/ether 86:70:73:75:d3:27 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 3     link/ether a2:93:0d:97:64:0c brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust on
Comment 7 Gris Ge 2022-12-18 11:24:31 UTC 
Hi Carlos,

Thanks for the confirmation!

I will patch nmstate to be smart on auto convert your original desire state 
to what I mentioned in comment #5.
Comment 8 Gris Ge 2022-12-19 09:38:21 UTC 
Patch send to upstream: https://github.com/nmstate/nmstate/pull/2165


For VF config

 * `vfs: []` will revert all VF configs back to default.
 * Non-empty `vfs` will merge with current for undefined VF.
Comment 12 Gris Ge 2023-02-16 08:25:17 UTC 
New patch send to upstream: https://github.com/nmstate/nmstate/pull/2241

The fix will shipped by next rebase build.
Comment 13 Mingyu Shi 2023-02-23 07:00:24 UTC 
Verified with:
nmstate-2.2.7-1.el9.x86_64
nispor-1.2.10-1.el9.x86_64
NetworkManager-1.42.0-1.el9.x86_64
openvswitch2.15-2.15.0-79.el9fdp.x86_64
Linux dell-per740-68.rhts.eng.pek2.redhat.com 5.14.0-252.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 1 09:46:00 EST 2023 x86_64 x86_64 x86_64 GNU/Linux
Comment 15 errata-xmlrpc 2023-05-09 07:31:50 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2190
Note You need to log in before you can comment on or make changes to this bug.