RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1721779 - Add bpftool net list support
Summary: Add bpftool net list support
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: sos
Version: 8.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.1
Assignee: Pavel Moravec
QA Contact: Miroslav Hradílek
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-06-19 04:09 UTC by Hangbin Liu
Modified: 2023-02-12 22:23 UTC (History)
8 users (show)

Fixed In Version: sos-3.8-2.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
https://github.com/sosreport/sos/pull/1712
Last Closed: 2020-04-28 17:01:55 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github sosreport sos pull 1712 0 'None' closed [kernel] collect "bpftool net list" 2020-08-11 12:18:03 UTC
Red Hat Issue Tracker RHELPLAN-31228 0 None None None 2023-02-12 22:23:15 UTC
Red Hat Product Errata RHEA-2020:1900 0 None None None 2020-04-28 17:02:44 UTC

Description Hangbin Liu 2019-06-19 04:09:32 UTC 
Description of problem:
Since bug 1700846, we updated bpf to upstream v5.0 and add net list support for bpftool. It would be helpful if we could get customer's xdp and tc attachment info.

# bpftool net help
Usage: bpftool net { show | list } [dev <devname>]
       bpftool net help
Note: Only xdp and tc attachments are supported now.
      For progs attached to cgroups, use "bpftool cgroup"
      to dump program attachments. For program types
      sk_{filter,skb,msg,reuseport} and lwt/seg6, please
      consult iproute2.

# bpftool net list
xdp:
enp3s0(7) driver id 1
enp3s0d1(8) driver id 2

tc:
veth0(10) clsact/egress test_tunnel_kern.o:[gre_set_tunnel] id 3
veth1(11) clsact/ingress test_tunnel_kern.o:[gre_get_tunnel] id 4
Comment 1 Pavel Moravec 2019-06-19 05:54:21 UTC 
Hello,
just collecting "bpftool net list" is required? No need to iterate over devices from the list output and collect "bpftool net show <device>" ?

In case the net show is also required, can I assume each line from "bpftool net list" that is not blank and not "xdp:" and not "tc:" contains device name before the very first "(" (or provide a different precise specification how to generate list of such devices)?

Thanks for clarification.
Comment 2 Hangbin Liu 2019-06-19 06:25:41 UTC 
(In reply to Pavel Moravec from comment #1)
> Hello,
> just collecting "bpftool net list" is required? No need to iterate over
> devices from the list output and collect "bpftool net show <device>" ?

No, I think just "bpftool net list" is enough. there is no more detail for "bpftool net show <device>".

Hi Jiri, what do you think?
Comment 3 Jiri Benc 2019-06-27 09:35:24 UTC 
Yes, "bpftool net list" is enough.
Comment 4 Pavel Moravec 2019-06-27 11:46:41 UTC 
Thx for clarification.

devel_ack+ to 8.2, upstream PR raised.
Comment 6 Pavel Moravec 2019-12-18 07:30:17 UTC 
Hi,
can I expect with OtherQE verification here as well, please?
Comment 7 Hangbin Liu 2019-12-18 07:46:30 UTC 
(In reply to Pavel Moravec from comment #6)
> Hi,
> can I expect with OtherQE verification here as well, please?

Do we have a test package now?

Thanks
Hangbin
Comment 8 Pavel Moravec 2019-12-18 08:23:44 UTC 
(In reply to Hangbin Liu from comment #7)
> (In reply to Pavel Moravec from comment #6)
> > Hi,
> > can I expect with OtherQE verification here as well, please?
> 
> Do we have a test package now?
> 
> Thanks
> Hangbin

Not yet. The bureaucratic logic is qa_ack+ (where OtherQA helps) => pm_ack+/release+ => I can put to dist-git => I can build a package.

So far, a 8.2 sos candidate build 3.8-1 is without this fix. I plan a re-spin probably in mid Jan.
Comment 9 Jiri Benc 2019-12-18 08:54:08 UTC 
Hmm, isn't this a duplicate of bug 1768956 (or vice versa)? Sorry for that, I completely forgot we have this bug open.

The description of this bug made me realize that we're not capturing 'bpftool cgroup tree' output. I guess I should file a separate bug report/RFE for that, right?
Comment 12 Pavel Moravec 2019-12-22 14:27:16 UTC 
(In reply to Jiri Benc from comment #9)
> Hmm, isn't this a duplicate of bug 1768956 (or vice versa)? Sorry for that,
> I completely forgot we have this bug open.

This BZ is for collecting "bpftool net list" itself, while bug 1768956 requests to collect the same per each namespace.


> 
> The description of this bug made me realize that we're not capturing
> 'bpftool cgroup tree' output. I guess I should file a separate bug
> report/RFE for that, right?

A new BZ is preferred. On RHEL7 or 8 or both? Just this command "system wide" or also per namespace?
Comment 13 Jiri Benc 2020-01-03 14:32:02 UTC 
(In reply to Pavel Moravec from comment #12)
> A new BZ is preferred.

Filed bug 1787586.

> On RHEL7 or 8 or both?

RHEL 8 only, this is not available on RHEL 7.

> Just this command "system wide" or also per namespace?

System wide. Added this info to the newly filed bug, too.

Thanks!
Comment 15 Pavel Moravec 2020-01-13 14:53:30 UTC 
(In reply to Hangbin Liu from comment #7)
> (In reply to Pavel Moravec from comment #6)
> > Hi,
> > can I expect with OtherQE verification here as well, please?
> 
> Do we have a test package now?
> 
> Thanks
> Hangbin

Now we have :) Could you please verify the bugfix against below package? Thanks in advance.


A yum repository for the build of sos-3.8-2.el8 (task 25733431) is available at:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.8/2.el8/

You can install the rpms locally by putting this .repo file in your /etc/yum.repos.d/ directory:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.8/2.el8/sos-3.8-2.el8.repo

RPMs and build logs can be found in the following locations:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.8/2.el8/noarch/

The full list of available rpms is:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.8/2.el8/noarch/sos-3.8-2.el8.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.8/2.el8/noarch/sos-3.8-2.el8.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.8/2.el8/noarch/sos-audit-3.8-2.el8.noarch.rpm

Build output will be available for the next 21 days.
Comment 16 Hangbin Liu 2020-01-14 03:34:26 UTC 
Hi Pavel,

Thanks for the package, here is the test result:

# rpm -q sos
sos-3.8-2.el8.noarch

# cat sos_commands/networking/bpftool_net_list
xdp:
test(6) driver id 68

tc:
test(6) clsact/ingress xdp_pass_kern.o:[xdp] id 65
test(6) clsact/ingress xdp_pass_kern.o:[xdp] id 64

flow_dissector:

# cat sos_commands/networking/ip_netns_exec_test_bpftool_net_list
xdp:
veth0(5) driver id 69

tc:

flow_dissector:
Comment 17 Jiri Benc 2020-01-16 19:43:26 UTC 
I ran a few kselftests, seems to work okay for xdp, tc and flow_dissector:

$ for f in sosreport-*/sos_commands/networking/*bpftool_net*; do echo $f; cat $f; done
sosreport-localhost-2-2020-01-16-inqeubt/sos_commands/networking/bpftool_net_list
xdp:

tc:

flow_dissector:

sosreport-localhost-2-2020-01-16-inqeubt/sos_commands/networking/ip_netns_exec_ns-vePDNH_bpftool_net_list
xdp:

tc:

flow_dissector:
id 4
sosreport-localhost-5-2020-01-16-hgmfhcn/sos_commands/networking/bpftool_net_list
xdp:
veth1(7) generic id 74

tc:

flow_dissector:

sosreport-localhost-5-2020-01-16-hgmfhcn/sos_commands/networking/ip_netns_exec_xdp_ns0_bpftool_net_list
xdp:
veth0(8) generic id 70

tc:

flow_dissector:

sosreport-localhost-6-2020-01-16-ivlznto/sos_commands/networking/bpftool_net_list
xdp:

tc:

flow_dissector:

sosreport-localhost-6-2020-01-16-ivlznto/sos_commands/networking/ip_netns_exec_ns-970-1_bpftool_net_list
xdp:

tc:
veth1(3) clsact/egress test_tc_tunnel.o:[encap_ip6tnl_none] id 3

flow_dissector:

sosreport-localhost-6-2020-01-16-ivlznto/sos_commands/networking/ip_netns_exec_ns-970-2_bpftool_net_list
xdp:

tc:

flow_dissector:
Comment 18 Jiri Benc 2020-01-16 19:49:43 UTC 
One thing I noticed: there's no sos_commands/kernel/bpftool_prog_list and sos_commands/kernel/bpftool_map_list, only the json (-j) versions are there. While the json files should be used by sos itself to parse the program and map ids, it's not the ideal output for an engineer looking at the report, as it's not easily human consumable. Shouldn't both version (-j and non-j) be captured? It's not urgent, both contain the same data and with the help of json_reformat it is kind of workable but it doesn't seem ideal. What do you think?

To illustrate that on a simple example (only one program loaded):

[root@localhost ~]# bpftool prog list
3: sched_cls  tag dd562a23ff170aab  gpl
        loaded_at 2020-01-16T20:36:45+0100  uid 0
        xlated 520B  jited 328B  memlock 4096B
[root@localhost ~]# bpftool prog -j list
[{"id":3,"type":"sched_cls","tag":"dd562a23ff170aab","gpl_compatible":true,"loaded_at":1579203405,"uid":0,"bytes_xlated":520,"jited":true,"bytes_jited":328,"bytes_memlock":4096}]
Comment 19 Pavel Moravec 2020-01-21 08:32:32 UTC 
Good point (that revealed a regression just in upstream :) ), I raised https://github.com/sosreport/sos/pull/1923

Until somebody disagrees, we will:
- collect just JSON format in 8.2
- fix the upstream regression (map/prog missing in either format in stored data, info available indirectly only via individual maps/progs)
- collect both JSON and human readable data since 8.3
Comment 20 Jiri Benc 2020-01-21 10:05:58 UTC 
(In reply to Pavel Moravec from comment #19)
> Until somebody disagrees, we will:
> - collect just JSON format in 8.2
> - fix the upstream regression (map/prog missing in either format in stored
> data, info available indirectly only via individual maps/progs)
> - collect both JSON and human readable data since 8.3

Sounds good to me, thanks a lot for taking care of this!
Comment 23 errata-xmlrpc 2020-04-28 17:01:55 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:1900
Note You need to log in before you can comment on or make changes to this bug.