Bug 1848056

Summary: extend sosreport to run "bpftool struct_ops dump"
Product: Red Hat Enterprise Linux 8 Reporter: Yauheni Kaliuta <ykaliuta>
Component: sosAssignee: Pavel Moravec <pmoravec>
Status: CLOSED ERRATA QA Contact: Miroslav HradĂ­lek <mhradile>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.4CC: agk, bmr, jbenc, jcastillo, mhradile, plambri, sbradley
Target Milestone: rcKeywords: OtherQA
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: sos-4.0-2.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-18 14:47:21 UTC Type: Feature Request
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 1 Jose Castillo 2020-06-17 15:42:01 UTC
Hello Yauheni. Is there a pull request already upstream?

Comment 2 Yauheni Kaliuta 2020-06-17 16:04:07 UTC
(In reply to Jose Castillo from comment #1)
> Hello Yauheni. Is there a pull request already upstream?

I, actually, do not know. This requirement came from review of bpf patchset to RHEL8.

Comment 3 Jiri Benc 2020-06-17 16:14:11 UTC
I don't think there is. If you don't have a capacity for writing an upstream patch, we can submit one, it's no problem, it should be rather easy. We have enough time for that, the target is RHEL 8.4 (as it's too late for RHEL 8.3 at this point).

Comment 4 Jose Castillo 2020-06-17 16:17:12 UTC
(In reply to Jiri Benc from comment #3)
> I don't think there is. If you don't have a capacity for writing an upstream
> patch, we can submit one, it's no problem, it should be rather easy. We have
> enough time for that, the target is RHEL 8.4 (as it's too late for RHEL 8.3
> at this point).

I'll write it then, I just need to finish another one today and will start with it after it.

Comment 5 Jose Castillo 2020-06-17 16:30:22 UTC
Done via https://github.com/sosreport/sos/pull/2116 . Check it when you have a minute, in case I missed anything.

Comment 6 Jiri Benc 2020-06-17 17:41:33 UTC
That was fast! It looks good to me.

We'll want to test it, I based the command line on bpftool patch review; I've still not finished my review of the bpf kernel and bpftool rebase and haven't had an opportunity to try executing the command, yet. I now checked again the bpftool patch and the bpftool man page and this should be indeed the command to run. A real test is a real test, though.

Comment 7 Jose Castillo 2020-06-17 18:00:17 UTC
Agreed, we need to test the patch. I don't know too much about bpftool so I'll need your help to test it. Can I try it in a Fedora VM?

Comment 8 Jiri Benc 2020-06-17 18:26:30 UTC
(In reply to Jose Castillo from comment #7)
> Agreed, we need to test the patch. I don't know too much about bpftool so
> I'll need your help to test it. Can I try it in a Fedora VM?

You'll need a kernel v5.7 or newer. To verify that the command indeed dumps the structs, you'll need a tcp_congestion_ops bpf program loaded; there is one in the kernel source tree but I'm not sure Fedora builds it. 'bpftool struct_ops register bpf_cubic.o' is the command to load it in case it does.

Comment 9 Jiri Benc 2020-06-17 18:27:45 UTC
I'll verify the command if you're not in hurry - I will get to that only next week, I'm afraid.

Comment 11 Pavel Moravec 2020-10-29 11:57:08 UTC
Hello,
can you please verify the bug against below build (not sure if our QE will have capacity to verify by themselves)? Thanks in advance.


A yum repository for the build of sos-4.0-2.el8 (task 32548242) is available at:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/

You can install the rpms locally by putting this .repo file in your /etc/yum.repos.d/ directory:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/sos-4.0-2.el8.repo

RPMs and build logs can be found in the following locations:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/

The full list of available rpms is:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/sos-4.0-2.el8.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/sos-4.0-2.el8.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/4.0/2.el8/noarch/sos-audit-4.0-2.el8.noarch.rpm

The repository will be available for the next 60 days. Scratch build output will be deleted
earlier, based on the Brew scratch build retention policy.

Comment 12 Yauheni Kaliuta 2020-10-29 12:42:10 UTC
Huh, if I knew how :) I'll try, thanks.

Comment 13 Pavel Moravec 2020-10-29 13:19:32 UTC
(In reply to Yauheni Kaliuta from comment #12)
> Huh, if I knew how :) I'll try, thanks.

Very brief check:

# rpm -q sos
sos-4.0-2.el8.noarch
# sos report -o ebpf --batch --build

sosreport (version 4.0)

WARNING: unable to set option for disabled or non-existing plugin (apache)

This command will collect diagnostic and configuration information from
this Red Hat Enterprise Linux system and installed applications.

An archive containing the collected information will be generated in
/var/tmp/sos.np2zgfza and may be provided to a Red Hat support
representative.

Any information provided to Red Hat will be treated in accordance with
the published support policies at:

  https://access.redhat.com/support/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.


 Setting up archive ...
 Setting up plugins ...
 Running plugins. Please wait ...

  Starting 1/1   ebpf            [Running: ebpf]

  Finished running plugins


Your sosreport build tree has been generated in:
	/var/tmp/sosreport-pmoravec-rhel8-2020-10-29-wuhklug


# find /var/tmp/sosreport-pmoravec-rhel8-2020-10-29-wuhklug | grep struct
/var/tmp/sosreport-pmoravec-rhel8-2020-10-29-wuhklug/sos_commands/ebpf/bpftool_struct_ops_dump
# cat /var/tmp/sosreport-pmoravec-rhel8-2020-10-29-wuhklug/sos_commands/ebpf/bpftool_struct_ops_dump
Usage: bpftool [OPTIONS] OBJECT { COMMAND | help }
       bpftool batch file FILE
       bpftool version

       OBJECT := { prog | map | cgroup | perf | net | feature | btf }
       OPTIONS := { {-j|--json} [{-p|--pretty}] | {-f|--bpffs} |
	            {-m|--mapcompat} | {-n|--nomount} }
#


So the command is collected, but bpftool (latest released version 4.18.0-193.28.1.el8_2) does not know that subcommand..?

Comment 14 Pavel Moravec 2020-10-29 14:14:28 UTC
OK, newer bpftool (bpftool-4.18.0-240.8.el8.x86_64) does know that subcommand:

# cat /var/tmp/sosreport-pmoravec-rhel8-2020-10-29-exhwfgh/sos_commands/ebpf/bpftool_struct_ops_dump
libbpf: failed to find valid kernel BTF
Error: struct_ops requires kernel CONFIG_DEBUG_INFO_BTF=y
#

(then it is rather kernel/config issue that the cmd output is fruitless - output is OK from sos perspective).

Comment 15 Yauheni Kaliuta 2020-10-29 16:04:43 UTC
# rpm -q sos
sos-4.0-2.el8.noarch
# cat /var/tmp/sosreport-ykaliuta-rhel8-2020-10-29-hqebtir/sos_commands/ebpf/bpftool_struct_ops_dump
[]

Comment 20 errata-xmlrpc 2021-05-18 14:47:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (sos bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:1604