Bug 2145105

(In reply to Zhiqian Guan from comment #0)
> Description of problem:
> bcctool tcpdrop tool reported below error:
> 
> [root@netqe2 ~]# cd /usr/share/bcc/tools/
> [root@netqe2 tools]# ./tcpdrop
> cannot attach kprobe, probe entry may not exist

As mention here, the probe entry does not exist. The reason being that the function has been inlined.

There has been some discussion about adding a tracepoint for dropped packets, but until this happens, there is no reliable way to trace that.

However, the error message could be improved to better inform the user. As a matter of fact, it has been done upstream and the fix will be brought by the next rebase. 

> Traceback (most recent call last):
>   File "./tcpdrop", line 200, in <module>
>     b.attach_kprobe(event="tcp_drop", fn_name="trace_tcp_drop")
>   File "/usr/lib/python3.6/site-packages/bcc/__init__.py", line 841, in
> attach_kprobe
>     (fn_name, event))
> Exception: Failed to attach BPF program b'trace_tcp_drop' to kprobe
> b'tcp_drop'
> [root@netqe2 tools]# uname -r
> 4.18.0-438.el8.x86_64
> [root@netqe2 tools]# rpm -q bcc
> bcc-0.24.0-2.el8.x86_64
> 
> Version-Release number of selected component (if applicable):
> 4.18.0-438.el8.x86_64
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. install bcc tools
> 2. run tcpdrop
> 3.
> 
> Actual results:
> fail to run
> 
> Expected results:
> no error
> 
> Additional info:

Should this be moved to 9.0?

(In reply to Jerome Marchand from comment #1)
> (In reply to Zhiqian Guan from comment #0)
> > Description of problem:
> > bcctool tcpdrop tool reported below error:
> > 
> > [root@netqe2 ~]# cd /usr/share/bcc/tools/
> > [root@netqe2 tools]# ./tcpdrop
> > cannot attach kprobe, probe entry may not exist
> 
> As mention here, the probe entry does not exist. The reason being that the
> function has been inlined.
> 
> There has been some discussion about adding a tracepoint for dropped
> packets, but until this happens, there is no reliable way to trace that.
> 
> However, the error message could be improved to better inform the user. As a
> matter of fact, it has been done upstream and the fix will be brought by the
> next rebase. 
> 
> > Traceback (most recent call last):
> >   File "./tcpdrop", line 200, in <module>
> >     b.attach_kprobe(event="tcp_drop", fn_name="trace_tcp_drop")
> >   File "/usr/lib/python3.6/site-packages/bcc/__init__.py", line 841, in
> > attach_kprobe
> >     (fn_name, event))
> > Exception: Failed to attach BPF program b'trace_tcp_drop' to kprobe
> > b'tcp_drop'

My earlier assessment was wrong. The error message above is not the one from tcpdrop tool, but the generic bcc error message when trying to attach a non-existent kprobe.

As it turned out, on RHEL 8 kernel, get_kprobe_functions("tcp_drop") does not return None due to the presence of tcp_drop_reason and a bug in get_kprobe_functions().
I sent a pull request upstream:
https://github.com/iovisor/bcc/pull/4426


> > [root@netqe2 tools]# uname -r
> > 4.18.0-438.el8.x86_64
> > [root@netqe2 tools]# rpm -q bcc
> > bcc-0.24.0-2.el8.x86_64
> > 
> > Version-Release number of selected component (if applicable):
> > 4.18.0-438.el8.x86_64
> > 
> > How reproducible:
> > 100%
> > 
> > Steps to Reproduce:
> > 1. install bcc tools
> > 2. run tcpdrop
> > 3.
> > 
> > Actual results:
> > fail to run
> > 
> > Expected results:
> > no error
> > 
> > Additional info:

Per my email with Jerome on Jan 9, 2023:

Me:
>> Okay but I ran it against yesterday's 8.8 image and got the same error message as Zhiqian.
Jerome:
> Ah, indeed. Looks like there is another bug in bcc and it has to be
fixed upstream. Then yes, we have to move this bz to 8.9.

The same error message is still here. Do you not care an just want to push this through with same error message for 8.8?

[root@cloud-qe-01 tools]# uname -r
4.18.0-451.el8.x86_64
[root@cloud-qe-01 tools]# rpm -q bcc
bcc-0.24.0-2.el8.x86_64

[root@cloud-qe-01 ~]# cd /usr/share/bcc/tools/
[root@cloud-qe-01 tools]# ./tcpdrop
cannot attach kprobe, probe entry may not exist
Traceback (most recent call last):
  File "./tcpdrop", line 200, in <module>
    b.attach_kprobe(event="tcp_drop", fn_name="trace_tcp_drop")
  File "/usr/lib/python3.6/site-packages/bcc/__init__.py", line 841, in attach_kprobe
    (fn_name, event))
Exception: Failed to attach BPF program b'trace_tcp_drop' to kprobe b'tcp_drop'

(In reply to Jon Trossbach from comment #9)
> Per my email with Jerome on Jan 9, 2023:
> 
> Me:
> >> Okay but I ran it against yesterday's 8.8 image and got the same error message as Zhiqian.
> Jerome:
> > Ah, indeed. Looks like there is another bug in bcc and it has to be
> fixed upstream. Then yes, we have to move this bz to 8.9.
> 
> The same error message is still here. Do you not care an just want to push
> this through with same error message for 8.8?
> 
> [root@cloud-qe-01 tools]# uname -r
> 4.18.0-451.el8.x86_64
> [root@cloud-qe-01 tools]# rpm -q bcc
> bcc-0.24.0-2.el8.x86_64

That should be bcc-0.25.0-2.el8

> 
> [root@cloud-qe-01 ~]# cd /usr/share/bcc/tools/
> [root@cloud-qe-01 tools]# ./tcpdrop
> cannot attach kprobe, probe entry may not exist
> Traceback (most recent call last):
>   File "./tcpdrop", line 200, in <module>
>     b.attach_kprobe(event="tcp_drop", fn_name="trace_tcp_drop")
>   File "/usr/lib/python3.6/site-packages/bcc/__init__.py", line 841, in
> attach_kprobe
>     (fn_name, event))
> Exception: Failed to attach BPF program b'trace_tcp_drop' to kprobe
> b'tcp_drop'

Hey thanks!

I didn't catch the bcc-0.25.0-2.el8 repo in brew earlier.

Jerome had to point me to it:
> there is also some automation that create a repo for all
brew build on RHEL. You can add the repo to /etc/yum.repos.d/:
http://brew-task-repos.usersys.redhat.com/repos/official/bcc/0.25.0/2.el8/bcc-0.25.0-2.el8.repo

I'll know where to look next time.


Results, they look good:

[root@netqe4 ~]# cd /usr/share/bcc/tools/
[root@netqe4 tools]# ./tcpdrop
WARNING: tcp_drop() kernel function not found or traceable. Use tracpoint:skb:kfree_skb instead.
TIME     PID     IP SADDR:SPORT          > DADDR:DPORT          STATE (FLAGS)

^C[root@netqe4 tools]# uname -r
4.18.0-451.el8.x86_64
[root@netqe4 tools]# rpm -q bcc
bcc-0.25.0-2.el8.x86_64

Pre-verification complete.

Looks good moving to verified.

[root@dell-per730-01 ~]# cd /usr/share/bcc/tools/
[root@dell-per730-01 tools]# ./tcpdrop 
WARNING: tcp_drop() kernel function not found or traceable. Use tracpoint:skb:kfree_skb instead.
TIME     PID     IP SADDR:SPORT          > DADDR:DPORT          STATE (FLAGS)

^C[root@dell-per730-01 tools]# uname -r
4.18.0-451.el8.x86_64
[root@dell-per730-01 tools]# rpm -q bcc
bcc-0.25.0-2.el8.x86_64

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (bcc bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2879