Bug 1813599 - Error running 'ip vrf exec': Failed to load BPF prog: 'Operation not permitted'
Summary: Error running 'ip vrf exec': Failed to load BPF prog: 'Operation not permitted'
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: 32
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-14 17:37 UTC by nucleo
Modified: 2021-05-18 18:52 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-18 18:52:09 UTC
Type: Bug


Attachments (Terms of Use)

Description nucleo 2020-03-14 17:37:33 UTC
Description of problem:
Error running 'ip vrf exec' on Fedora 32 Beta.

Version-Release number of selected component (if applicable):
iproute-5.5.0-1.fc32.x86_64
kernel-core-5.6.0-0.rc5.git0.2.fc32.x86_64

Steps to Reproduce:
1. ip link add VRF10 type vrf table 10
2. ip vrf exec VRF10 ip link
3. Run again "ip vrf exec VRF10 ip link"

Actual results:
On step 2 'ip vrf exec' works, on step 3 failed with error message
Failed to load BPF prog: 'Operation not permitted'

Additional info:
After user relogin 'ip vrf exec' again runs correctly at first run and fails at second run.

Comment 1 nucleo 2020-06-18 17:26:43 UTC
Error also after updating to iproute-5.6.0-1.fc32.x86_64

Comment 2 nucleo 2020-07-15 16:09:02 UTC
Looks like in Rwahide this is fixed somehow. 
Is it possible to backport fix to F32?

Comment 3 Phil Sutter 2020-07-27 16:09:09 UTC
(In reply to nucleo from comment #2)
> Looks like in Rwahide this is fixed somehow. 
> Is it possible to backport fix to F32?

Maybe it is kernel-related and iproute version doesn't matter. Anyway, since F32 kernel is 5.7.0 meanwhile, I rebased iproute in F32 as well. Please give it a try if you find time.

Comment 4 nucleo 2020-07-28 19:01:59 UTC
iproute-5.7.0-1.fc32.x86_64 did not fix this problem.

I installed systemd-246~rc2 from Rawhide built for F32
https://koji.fedoraproject.org/koji/taskinfo?taskID=48030330
no more 'Operation not permitted' after that.

Installing systemd-245.7-1.fc32.x86_64 from updates-testing also did not fix this problem,, only Rawhide version fixes.

Comment 5 Zbigniew Jędrzejewski-Szmek 2020-08-01 06:31:11 UTC
openat(AT_FDCWD, "/proc/920/cgroup", O_RDONLY) = 5</proc/920/cgroup>
fstat(5</proc/920/cgroup>, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(5</proc/920/cgroup>, "0::/user.slice/user-1000.slice/session-1.scope\n", 1024) = 47
close(5</proc/920/cgroup>)              = 0
mkdir("/sys", 0755)                     = -1 EEXIST (File exists)
mkdir("/sys", 0755)                     = -1 EEXIST (File exists)
mkdir("/sys/fs", 0755)                  = -1 EEXIST (File exists)
mkdir("/sys/fs", 0755)                  = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup", 0755)           = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup", 0755)           = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice/user-1000.slice", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice/user-1000.slice", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope/vrf", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope/vrf", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope/vrf/VRF10", 0755) = -1 EEXIST (File exists)
mkdir("/sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope/vrf/VRF10", 0755) = -1 EEXIST (File exists)
openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope/vrf/VRF10", O_RDONLY|O_DIRECTORY) = 5</sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope/vrf/VRF10>
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_CGROUP_SOCK, insn_cnt=6, insns=0x7ffd460b6d40, license="GPL", log_level=1, log_size=262144, log_buf="", kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=0</dev/pts/0>, func_info_rec_size=0, func_info=NULL, func_info_cnt=0, line_info_rec_size=0, line_info=NULL, line_info_cnt=0, attach_btf_id=0, attach_prog_fd=0</dev/pts/0>}, 120) = -1 EPERM (Operation not permitted)
write(2</dev/pts/0>, "Failed to load BPF prog: 'Operation not permitted'\n", 51Failed to load BPF prog: 'Operation not permitted'
) = 51
close(5</sys/fs/cgroup/user.slice/user-1000.slice/session-1.scope/vrf/VRF10>) = 0
close(-1)                               = -1 EBADF (Bad file descriptor)

I don't understand why we get the EPERM :(

Comment 6 mb+fedora 2020-09-30 10:33:00 UTC
This problem is still there in up-to-date (as of this morning) Fedora 33.

kernel-5.8.12-300.fc33.aarch64
systemd-246.4-2.fc33.aarch64
iproute-5.8.0-1.fc33.aarch64

Comment 7 ybendito 2020-11-26 17:33:06 UTC
Probably the problem happens due to low memlock limit, I see Fedora has 64K limit by default.
What is the output of 'ulimit -l'?
If it is low, try to set it to something more significant:
Add two lines to /etc/security/limits.conf
*                hard    memlock         16384
*                soft    memlock         16384
reboot and check whether this helps

Comment 8 nucleo 2020-11-26 17:52:40 UTC
This was fixed somehow some time ago and now works for me in updated Fedora 33 (x86_64) without modifying of limits. Maybe only aarch64 affected?

Comment 9 Fedora Program Management 2021-04-29 16:14:22 UTC
This message is a reminder that Fedora 32 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '32'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 32 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 10 Zbigniew Jędrzejewski-Szmek 2021-05-18 18:52:09 UTC
Yeah, this seems to be fixed. Sometime between kernel 5.8 and 5.11.
I can't reproduce with 5.11.7-200.fc33.x86_64 or 5.12.0-198.fc34.x86_64,
but it's reproducible with 5.8.17-300.fc33.x86_64.


Note You need to log in before you can comment on or make changes to this bug.