Bug 1787586

Summary: Add 'bpftool cgroup tree' support
Product: Red Hat Enterprise Linux 8 Reporter: Jiri Benc <jbenc>
Component: sosAssignee: Pavel Moravec <pmoravec>
Status: CLOSED ERRATA QA Contact: Miroslav HradĂ­lek <mhradile>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.2CC: agk, bmr, cww, haliu, jcastillo, jhunsaker, mhradile, plambri, sbradley
Target Milestone: rcKeywords: OtherQA
Target Release: 8.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: sos-3.9.1-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-04 01:57:52 UTC Type: Enhancement
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jiri Benc 2020-01-03 14:30:47 UTC
With expansion of eBPF functionality, the bpf programs can influence more and more the behavior of the system. sosreport already calls bpftool. But the bpftool tool is getting new functionality, too, which should be utilized by sosreport to provide more info to the support engineers.

bpftool got support for listing all eBPF programs attached to cgroups. Such programs can influence various system calls of programs in the given cgroup. This is being used mostly by container networking solutions (such as Cilium).

Please run 'bpftool cgroup tree'. This should be run system-wide (i.e., no need to be run per name space).

I'm not exactly sure whether it belongs to plugins/kernel.py or plugins/networking.py. Although it is currently used by networking related programs, the idea behind bpf cgroups is general and can expand to non-networking features in the future.

Comment 1 Jake Hunsaker 2020-01-03 15:40:44 UTC
While by itself this is trivial to add, at this point I wouldn't be against moving all the bpftool calls into a new `bpftool` plugin. We've already shuffled some of the calls a couple times, and the distinction between whether to keep it in networking or kernel is getting smaller.

Comment 2 Jose Castillo 2020-01-09 11:36:13 UTC
Jiri, a quick note to let you know that I'm working on this at the moment via pull request https://github.com/sosreport/sos/pull/1907

Comment 3 Pavel Moravec 2020-05-27 09:00:13 UTC
This will be automatically fixed in RHEL8.3 due to rebase of sos to 3.9-1 that includes the upstream fix. To verify it by yourself against a candidate package:

A yum repository for the build of sos-3.9.1-1.el8 (task 28858375) is available at:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/1.el8/

You can install the rpms locally by putting this .repo file in your /etc/yum.repos.d/ directory:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/1.el8/sos-3.9.1-1.el8.repo

RPMs and build logs can be found in the following locations:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/1.el8/noarch/

The full list of available rpms is:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/1.el8/noarch/sos-3.9.1-1.el8.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/1.el8/noarch/sos-3.9.1-1.el8.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.9.1/1.el8/noarch/sos-audit-3.9.1-1.el8.noarch.rpm

The repository will be available for the next 60 days. Scratch build output will be deleted
earlier, based on the Brew scratch build retention policy.

Comment 4 Pavel Moravec 2020-07-01 10:25:31 UTC
Hello,
this bug should be fixed in 8.3 but there is no capacity to verify the bugfix.

Let me know if you can / are interested in verifying the fix by yourself (i.e. if OtherQE could happen).

Comment 5 Jiri Benc 2020-07-01 10:36:22 UTC
(In reply to Pavel Moravec from comment #4)
> Let me know if you can / are interested in verifying the fix by yourself
> (i.e. if OtherQE could happen).

Sure, no problem. I'll try to verify today, as I'm on PTO from tomorrow on.

Comment 6 Jiri Benc 2020-07-01 11:39:11 UTC
Tested with RHEL-8.3.0-InternalSnapshot-2.0, which contains sos-3.9.1-2.el8.noarch, and with the newest 8.3 kernel (kernel-4.18.0-214.el8.x86_64).

On a freshly booted system:

# cat sosreport-localhost-0-2020-07-01-dgsaqye/sos_commands/ebpf/bpftool_cgroup_tree
Error: cgroup v2 isn't mounted

With a sample cgroup bpf program running:

# cat sosreport-localhost-1-2020-07-01-exizekj/sos_commands/ebpf/bpftool_cgroup_tree
CgroupPath
ID       AttachType      AttachFlags     Name           
/tmp/cgroupv2-test_cgrp2_sock/sockopts
    10       sock_create

# cat sosreport-localhost-1-2020-07-01-exizekj/sos_commands/ebpf/bpftool_prog_list
10: cgroup_sock  tag ea3142f3d9d82e18  gpl
        loaded_at 2020-07-01T13:12:35+0200  uid 0
        xlated 144B  jited 108B  memlock 4096B

Looks good to me.

Comment 13 errata-xmlrpc 2020-11-04 01:57:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (sos bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4534