Bug 1880268 - [nfp] xdp-loader unload failed after load a sample xdp program
Summary: [nfp] xdp-loader unload failed after load a sample xdp program
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: xdp-tools
Version: 8.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.4
Assignee: Toke Høiland-Jørgensen
QA Contact: Zhiqian Guan
Marc Muehlfeld
URL:
Whiteboard:
Depends On:
Blocks: 1680409
TreeView+ depends on / blocked
 
Reported: 2020-09-18 07:06 UTC by Zhiqian Guan
Modified: 2021-05-18 16:10 UTC (History)
2 users (show)

Fixed In Version: xdp-tools-1.1.0-1.el8.x86_64
Doc Type: Bug Fix
Doc Text:
.Unloading XDP programs no longer fails on Netronome network cards that use the `nfp` driver Previously, the `nfp` driver for Netronome network cards contained a bug. As a consequence, unloading eXpress Data Path (XDP) programs failed if you used such a card and loaded the XDP program using the `IFLA_XDP_EXPECTED_FD` feature with the `XDP_FLAGS_REPLACE` flag. For example, this affected XDP programs that were loaded using the `libxdp` library. This bug has been fixed. As a result, unloading an XDP program from Netronome network cards works as expected.
Clone Of:
Environment:
Last Closed: 2021-05-18 16:09:59 UTC
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2021:1925 0 None None None 2021-05-18 16:10:07 UTC

Description Zhiqian Guan 2020-09-18 07:06:22 UTC
Description of problem:

I cannot unload an xdp program which attached to a nfp card with native mode by using "xdp-loader unload $nic -a"

How reproducible:
Always


Steps to Reproduce:

[root@netqe30 xdp_tools]# xdp-loader load ens3np0 xdp_prog_kern.o
[root@netqe30 xdp_tools]# xdp-loader unload ens3np0 -a
libbpf: Kernel error message: program loaded with different flags
libxdp: Error attaching XDP program to ifindex 10: Device or resource busy
Unable to detach XDP program: Device or resource busy
[root@netqe30 xdp_tools]# xdp-loader status
CURRENT XDP PROGRAM STATUS:

Interface        Prio  Program name     Mode     ID   Tag               Chain actions
-------------------------------------------------------------------------------------
lo               <no XDP program>
eno1             <no XDP program>
ens1f0           <no XDP program>
eno2             <no XDP program>
ens1f1           <no XDP program>
eno3             <no XDP program>
eno4             <no XDP program>
ens2f0           <no XDP program>
ens2f1           <no XDP program>
ens3np0                xdp_dispatcher   native   665  d51e469e988d81da
 =>              50    xdp_pass_func             670  3b185187f1855c4c  XDP_PASS
ens3np1          <no XDP program>

[root@netqe30 xdp_tools]# xdp-loader unload ens3np0 -a -vv
Setting rlimit to minimum 1048576
libxdp: Verified XDP dispatcher version 1 <= 1
libxdp: DATASEC '.xdp_run_config' not found.
libxdp: Acquired lock from /sys/fs/bpf/xdp with fd 6
libxdp: Reading multiprog component programs from pinned directory
libxdp: DATASEC '.xdp_run_config' not found.
libxdp: Released lock fd 6
libxdp: Found multiprog with id 665 and 1 component progs
libxdp: Replacing XDP fd 4 with -1 on ifindex 10
libbpf: Kernel error message: program loaded with different flags
libxdp: Error attaching XDP program to ifindex 10: Device or resource busy
libxdp: XDP already loaded on device
Unable to detach XDP program: Device or resource busy
[root@netqe30 xdp_tools]#


Actual results:
unload failed

Expected results:
can successfully unload the xdp program

Comment 1 Zhiqian Guan 2020-09-18 07:09:46 UTC
file bug for tracking the unresolved known issue of xdp-tools, details please see: https://bugzilla.redhat.com/show_bug.cgi?id=1820670, comment96, comment97

Comment 2 Toke Høiland-Jørgensen 2020-09-18 10:26:31 UTC
For this (and the other nfp-related bug), I will likely need access to a machine with an nfp device - Zhiqian, is there a particular qe box I can use for that, and how do I get access to that? :)

Comment 3 Zhiqian Guan 2020-09-21 09:28:26 UTC
I'm also sharing the nfp test bed with others, let me check with him first :~)

Comment 5 Zhiqian Guan 2020-10-19 07:35:50 UTC
Hi Toke,

Is there any plan ddl for this bug? I'm also going to set the ITR to 8.4.0 and need info to set the ITM field:~)

Comment 6 Toke Høiland-Jørgensen 2020-10-21 10:15:52 UTC
(In reply to Zhiqian Guan from comment #5)
> Hi Toke,
> 
> Is there any plan ddl for this bug? I'm also going to set the ITR to 8.4.0
> and need info to set the ITM field:~)

Hmm, the ITM includes QE verification, right? So I think it's simplest if we do this along with the 5.10 backport, as there are kernel features we'll need in the version of xdp-tools we ship in 8.4, so if we can just do everything at once that would be simpler...

So set this to whatever the 5.10 backport ends up as +1 (or 2)?

(same for the other xdp-tools issues)

Comment 7 Zhiqian Guan 2020-10-22 02:09:35 UTC
(In reply to Toke Høiland-Jørgensen from comment #6)
> (In reply to Zhiqian Guan from comment #5)
> > Hi Toke,
> > 
> > Is there any plan ddl for this bug? I'm also going to set the ITR to 8.4.0
> > and need info to set the ITM field:~)
> 
> Hmm, the ITM includes QE verification, right? So I think it's simplest if we
> do this along with the 5.10 backport, as there are kernel features we'll
> need in the version of xdp-tools we ship in 8.4, so if we can just do
> everything at once that would be simpler...
> 
> So set this to whatever the 5.10 backport ends up as +1 (or 2)?
> 
> (same for the other xdp-tools issues)

OK, got it:)

Comment 8 Zhiqian Guan 2020-11-16 03:06:22 UTC
Hi Toke,

Since the 5.10 backport development for ebpf/xdp will be done by ITM12, so I'll set this bug to ITM 15, sync up with 5.10 backport ITM. Same as other xdp-tools bugs. otherwise, it will pass the Beta. it may need you to fix them at least by ITM 13or14, is that OK for you?

Comment 9 Toke Høiland-Jørgensen 2020-11-16 23:28:21 UTC
Sure, that should be fine :)

Comment 10 Toke Høiland-Jørgensen 2021-01-04 21:05:11 UTC
Scratch build what should fix this available here: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=34045153

Will turn it into a proper build as soon as I figure out how to get a new branch created in dist-git...

Comment 15 Zhiqian Guan 2021-01-28 06:42:08 UTC
This issue won't be reproduced in the new version

[root@netqe30 ~]# rpm -q xdp-tools
xdp-tools-1.1.0-1.el8.x86_64
[root@netqe30 ~]# uname -r
4.18.0-277.el8.x86_64
[root@netqe30 ~]# rpm -q libxdp
libxdp-1.1.0-1.el8.x86_64
[root@netqe30 ~]# rpm -q libbpf
libbpf-0.0.8-4.el8.x86_64
[root@netqe30 ~]#
[root@netqe30 ~]# ethtool -i ens3np0
driver: nfp
version: 4.18.0-277.el8.x86_64
firmware-version: 0.0.3.5 0.22 bpf-2.0.6.124 ebpf
expansion-rom-version:
bus-info: 0000:5e:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
[root@netqe30 ~]# xdp-loader load ens3np0 xdp_prog_kern.o
[root@netqe30 ~]# xdp-loader status
CURRENT XDP PROGRAM STATUS:

Interface        Prio  Program name     Mode     ID   Tag               Chain actions
-------------------------------------------------------------------------------------
lo               <no XDP program>
eno1             <no XDP program>
ens1f0           <no XDP program>
eno2             <no XDP program>
ens1f1           <no XDP program>
eno3             <no XDP program>
eno4             <no XDP program>
ens2f0           <no XDP program>
ens2f1           <no XDP program>
ens3np0                xdp_dispatcher   native   81   d51e469e988d81da
 =>              50    xdp_pass_func             93   3b185187f1855c4c  XDP_PASS
ens3np1          <no XDP program>

[root@netqe30 ~]# xdp-loader unload ens3np0 -a
[root@netqe30 ~]# xdp-loader status
CURRENT XDP PROGRAM STATUS:

Interface        Prio  Program name     Mode     ID   Tag               Chain actions
-------------------------------------------------------------------------------------
lo               <no XDP program>
eno1             <no XDP program>
ens1f0           <no XDP program>
eno2             <no XDP program>
ens1f1           <no XDP program>
eno3             <no XDP program>
eno4             <no XDP program>
ens2f0           <no XDP program>
ens2f1           <no XDP program>
ens3np0          <no XDP program>
ens3np1          <no XDP program>

Comment 16 Zhiqian Guan 2021-01-28 06:44:32 UTC
base on Comment15, set this to VERIFIED

Comment 20 errata-xmlrpc 2021-05-18 16:09:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (xdp-tools bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:1925


Note You need to log in before you can comment on or make changes to this bug.