Bug 2127404

Summary:	Introduce libcap-ng inside rsyslog
Product:	Red Hat Enterprise Linux 9	Reporter:	Attila Lakatos <alakatos>
Component:	rsyslog	Assignee:	Attila Lakatos <alakatos>
Status:	CLOSED ERRATA	QA Contact:	Dalibor Pospíšil <dapospis>
Severity:	unspecified	Docs Contact:	Jan Fiala <jafiala>
Priority:	unspecified
Version:	9.2	CC:	alakatos, dapospis, extras-qa, h1k6zn2m, jafiala, jlieskov, jvymazal, lkundrak, mah.darade, pascal.tempier, pasik, richard.hickson, rsroka, tosykora, zfridric
Target Milestone:	rc	Keywords:	AutoVerified, FutureFeature, Triaged
Target Release:	---	Flags:	pm-rhel: mirror+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	rsyslog-8.2102.0-107.el9	Doc Type:	Enhancement
Doc Text:	.Rsyslog privileges are limited The privileges of the Rsyslog log processing system are now limited to only the privileges explicitly required by Rsyslog. This minimizes security exposure in case of a potential error in input resources, for example, a networking plugin. As a result, Rsyslog has the same functionality but does not have unnecessary privileges.	Story Points:	---
Clone Of:	2127403	Environment:
Last Closed:	2023-05-09 07:44:54 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2127403
Bug Blocks:

Description Attila Lakatos 2022-09-16 08:27:05 UTC

+++ This bug was initially created as a clone of Bug #2127403 +++

Description of the problem:

Actually, rsyslogd is running with full set of capabilities. Whilst it's possible for rsyslog to drop privileges by impersonating as another user and/or group after startup, there are some modules that explicitly require root user rights. Thus, it's necessary to drop the capabilities to only the necessary set, to minimize security exposure in case there was ever a mistake in a networking plugin or some other input resource. The change should be done directly in the code with libcap-ng as rsyslog has many optional modules and some of them require additional capabilities.

Comment 13 pascal.tempier 2023-01-04 17:08:50 UTC

Hello

It seems v8.2102.0-107.el9 doesnt works anymore in docker.

When starting with the default parameter i get this error.

rsyslog internal message (3,-2455): could not transfer  the  specified  internal posix  capabilities settings to the kernel, capng_apply=-5


This with the defaults parameters.

docker inspect 4d0fa0efded6 | grep  -i Cap
            "CapAdd": null,
            "CapDrop": null,

Comment 14 Attila Lakatos 2023-01-05 09:37:59 UTC

(In reply to pascal.tempier from comment #13)
> Hello
> 
> It seems v8.2102.0-107.el9 doesnt works anymore in docker.
> 
> When starting with the default parameter i get this error.
> 
> rsyslog internal message (3,-2455): could not transfer  the  specified 
> internal posix  capabilities settings to the kernel, capng_apply=-5
> 
> 
> This with the defaults parameters.
> 
> docker inspect 4d0fa0efded6 | grep  -i Cap
>             "CapAdd": null,
>             "CapDrop": null,

Hello,

Thanks for the report. The -5 error code means failure in capset syscall. It seems like you have to add Linux capabilities to the container via the "--cap-add=[]" option. The list of needed capabilities are: CAP_AUDIT_READ, CAP_BLOCK_SUSPEND, CAP_CHOWN, CAP_IPC_LOCK, CAP_LEASE, CAP_NET_ADMIN, CAP_NET_BIND_SERVICE, CAP_PERFMON, CAP_SETGID, CAP_SETUID, CAP_SYS_ADMIN, CAP_SYS_CHROOT, CAP_SYS_RESOURCE, CAP_SYSLOG.
Let me know if that helps.

Comment 15 pascal.tempier 2023-01-05 12:05:31 UTC

Hello

This bug was open because before this update, the additional capabilities were not needed.
But they are needed after, while the goal of the update was to reduce the capabilities in use, but now i need to add more capabilities.
See what i mean ?

Comment 16 Attila Lakatos 2023-01-05 14:35:31 UTC

Let me correct my previous answer. I did some digging and found out that there is one special capability that rsyslog is not able to set inside a docker container - CAP_PERFMON. If that was not part of the caps list, everything would be working fine. My investigation towards this issue led to upstream discussion https://github.com/docker/docs/issues/13731:
```
Few month back, the commit for disabling CAP_PERFMON, CAP_BPF, and CAP_CHECKPOINT_RESTORE got reverted (moby/moby#42011), so now we should be able to create containers with these capabilities.
When this feature got disabled in the first place, it didn't get into the changelog (moby/moby#42601 (comment)), so one might think that it should work when it actually can't.
```

Which version of docker are you using? I do believe that the patch responsible for enabling CAP_PERFMON has not been backported yet.

Anyway, I have mixed feelings about the CAP_PERFMON capability. I purposely did not skip it because the capabilities(7) man page mentions:
CAP_PERFMON (since Linux 5.8):
Employ various performance-monitoring mechanisms, including:
* call perf_event_open(2);
* employ various BPF operations that have performance implications.
And the perf_event_open(2) man page says that glibc provides no wrapper for perf_event_open(), necessitating the use of syscall(2), which is indeed used in rsyslog.

I will see if we can disable this capability without breaking rsyslog. However, your problem should be resolved with the latest upstream version of docker.

Comment 17 pascal.tempier 2023-01-09 16:19:20 UTC

Hello


Still the same issue after geting the last docker version

 docker version
Client: Docker Engine - Community
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:28:05 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.22
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.9
  Git commit:       42c8b31
  Built:            Thu Dec 15 22:26:16 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.14
  GitCommit:        9ba4b250366a5ddde94bb7c9d1def331423aa323
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0



The ends of the stacktrace if it helps



brk(NULL)                               = 0x562ca0c7a000
brk(0x562ca0c9b000)                     = 0x562ca0c9b000
access("/etc/gcrypt/fips_enabled", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = 3
newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=0, ...}, AT_EMPTY_PATH) = 0
read(3, "0\n", 1024)                    = 2
close(3)                                = 0
prctl(PR_CAPBSET_READ, CAP_MAC_OVERRIDE) = 0
prctl(PR_CAPBSET_READ, 0x30 /* CAP_??? */) = -1 EINVAL (Invalid argument)
prctl(PR_CAPBSET_READ, CAP_CHECKPOINT_RESTORE) = 0
prctl(PR_CAPBSET_READ, 0x2c /* CAP_??? */) = -1 EINVAL (Invalid argument)
prctl(PR_CAPBSET_READ, 0x2a /* CAP_??? */) = -1 EINVAL (Invalid argument)
prctl(PR_CAPBSET_READ, 0x29 /* CAP_??? */) = -1 EINVAL (Invalid argument)
openat(AT_FDCWD, "/proc/sys/kernel/cap_last_cap", O_RDONLY) = 3
fstatfs(3, {f_type=PROC_SUPER_MAGIC, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=0, f_ffree=0, f_fsid={val=[0, 0]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_RDONLY|ST_NOSUID|ST_NODEV|ST_NOEXEC|ST_RELATIME}) = 0
read(3, "40\n", 7)                      = 3
close(3)                                = 0
getpid()                                = 77
capget({version=0 /* _LINUX_CAPABILITY_VERSION_??? */, pid=0}, NULL) = 0
gettid()                                = 77
capget({version=_LINUX_CAPABILITY_VERSION_3, pid=77}, {effective=1<<CAP_CHOWN|1<<CAP_DAC_OVERRIDE|1<<CAP_FOWNER|1<<CAP_FSETID|1<<CAP_KILL|1<<CAP_SETGID|1<<CAP_SETUID|1<<CAP_SETPCAP|1<<CAP_NET_BIND_SERVICE|1<<CAP_NET_ADMIN|1<<CAP_NET_RAW|1<<CAP_SYS_CHROOT|1<<CAP_MKNOD|1<<CAP_AUDIT_WRITE|1<<CAP_SETFCAP, permitted=1<<CAP_CHOWN|1<<CAP_DAC_OVERRIDE|1<<CAP_FOWNER|1<<CAP_FSETID|1<<CAP_KILL|1<<CAP_SETGID|1<<CAP_SETUID|1<<CAP_SETPCAP|1<<CAP_NET_BIND_SERVICE|1<<CAP_NET_ADMIN|1<<CAP_NET_RAW|1<<CAP_SYS_CHROOT|1<<CAP_MKNOD|1<<CAP_AUDIT_WRITE|1<<CAP_SETFCAP, inheritable=0}) = 0
openat(AT_FDCWD, "/proc/77/status", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=0, ...}, AT_EMPTY_PATH) = 0
read(3, "Name:\trsyslogd\nUmask:\t0022\nState"..., 1024) = 1024
close(3)                                = 0
openat(AT_FDCWD, "/proc/77/status", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=0, ...}, AT_EMPTY_PATH) = 0
read(3, "Name:\trsyslogd\nUmask:\t0022\nState"..., 1024) = 1024
close(3)                                = 0
prctl(PR_CAPBSET_DROP, CAP_CHOWN)       = 0
prctl(PR_CAPBSET_DROP, CAP_DAC_OVERRIDE) = 0
prctl(PR_CAPBSET_DROP, CAP_DAC_READ_SEARCH) = 0
prctl(PR_CAPBSET_DROP, CAP_FOWNER)      = 0
prctl(PR_CAPBSET_DROP, CAP_FSETID)      = 0
prctl(PR_CAPBSET_DROP, CAP_KILL)        = 0
prctl(PR_CAPBSET_DROP, CAP_SETGID)      = 0
prctl(PR_CAPBSET_DROP, CAP_SETUID)      = 0
prctl(PR_CAPBSET_DROP, CAP_SETPCAP)     = 0
prctl(PR_CAPBSET_DROP, CAP_LINUX_IMMUTABLE) = 0
prctl(PR_CAPBSET_DROP, CAP_NET_BIND_SERVICE) = 0
prctl(PR_CAPBSET_DROP, CAP_NET_BROADCAST) = 0
prctl(PR_CAPBSET_DROP, CAP_NET_ADMIN)   = 0
prctl(PR_CAPBSET_DROP, CAP_NET_RAW)     = 0
prctl(PR_CAPBSET_DROP, CAP_IPC_LOCK)    = 0
prctl(PR_CAPBSET_DROP, CAP_IPC_OWNER)   = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_MODULE)  = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_RAWIO)   = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_CHROOT)  = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_PTRACE)  = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_PACCT)   = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_ADMIN)   = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_BOOT)    = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_NICE)    = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_RESOURCE) = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_TIME)    = 0
prctl(PR_CAPBSET_DROP, CAP_SYS_TTY_CONFIG) = 0
prctl(PR_CAPBSET_DROP, CAP_MKNOD)       = 0
prctl(PR_CAPBSET_DROP, CAP_LEASE)       = 0
prctl(PR_CAPBSET_DROP, CAP_AUDIT_WRITE) = 0
prctl(PR_CAPBSET_DROP, CAP_AUDIT_CONTROL) = 0
prctl(PR_CAPBSET_DROP, CAP_SETFCAP)     = 0
prctl(PR_CAPBSET_DROP, CAP_MAC_OVERRIDE) = 0
prctl(PR_CAPBSET_DROP, CAP_MAC_ADMIN)   = 0
prctl(PR_CAPBSET_DROP, CAP_SYSLOG)      = 0
prctl(PR_CAPBSET_DROP, CAP_WAKE_ALARM)  = 0
prctl(PR_CAPBSET_DROP, CAP_BLOCK_SUSPEND) = 0
prctl(PR_CAPBSET_DROP, CAP_AUDIT_READ)  = 0
prctl(PR_CAPBSET_DROP, CAP_PERFMON)     = 0
prctl(PR_CAPBSET_DROP, CAP_BPF)         = 0
prctl(PR_CAPBSET_DROP, CAP_CHECKPOINT_RESTORE) = 0
openat(AT_FDCWD, "/proc/77/status", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=0, ...}, AT_EMPTY_PATH) = 0
read(3, "Name:\trsyslogd\nUmask:\t0022\nState"..., 1024) = 1024
close(3)                                = 0
capset({version=_LINUX_CAPABILITY_VERSION_3, pid=77}, {effective=1<<CAP_CHOWN|1<<CAP_SETGID|1<<CAP_SETUID|1<<CAP_NET_BIND_SERVICE|1<<CAP_NET_ADMIN|1<<CAP_IPC_LOCK|1<<CAP_SYS_CHROOT|1<<CAP_SYS_ADMIN|1<<CAP_SYS_RESOURCE|1<<CAP_LEASE|1<<CAP_SYSLOG|1<<CAP_BLOCK_SUSPEND|1<<CAP_PERFMON, permitted=1<<CAP_CHOWN|1<<CAP_SETGID|1<<CAP_SETUID|1<<CAP_NET_BIND_SERVICE|1<<CAP_NET_ADMIN|1<<CAP_IPC_LOCK|1<<CAP_SYS_CHROOT|1<<CAP_SYS_ADMIN|1<<CAP_SYS_RESOURCE|1<<CAP_LEASE|1<<CAP_SYSLOG|1<<CAP_BLOCK_SUSPEND|1<<CAP_PERFMON, inheritable=0}) = -1 EPERM (Operation not permitted)
write(2, "rsyslog internal message (3,-245"..., 197rsyslog internal message (3,-2455): could not transfer  the  specified  internal posix  capabilities settings to the kernel, capng_apply=-5
 [v8.2102.0-107.el9 try https://www.rsyslog.com/e/2455 ]
) = 197
exit_group(-1)                          = ?
+++ exited with 255 +++

Comment 18 Jonathan Kamens 2023-01-12 08:10:24 UTC

This new functionality causes real problems. I've opened bug 2160380 about one of them.

Comment 25 errata-xmlrpc 2023-05-09 07:44:54 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (rsyslog bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2303

Comment 26 richard.hickson@nokia.com 2023-07-01 17:04:48 UTC

Used the latest rsyslod version 113, and the issue is still not resolved.  Is the current expected only workaround for users running in a docker container
as non-root is to rebuild without the  --enable-libcap-ng option from source

Comment 27 Attila Lakatos 2023-07-03 07:20:15 UTC

(In reply to richard.hickson from comment #26)
> Used the latest rsyslod version 113, and the issue is still not resolved. 
> Is the current expected only workaround for users running in a docker
> container
> as non-root is to rebuild without the  --enable-libcap-ng option from source

Hi, this will be fixed with the next update. Thanks for your understanding.