Description of problem:

The usbguard is a security relevant service which authorizes new devices on a system. When it gets killed, the authorization suddenly is not available and anything is allowed to get plugged-in.

The out-of-memory killer in the kernel can easily kill the usbguard-daemon for example when unauthorized process creates many small processes, which causes the OOM algorithm to see usbguard-daemon as the best candidate to be killed, instead of aiming at some of those unprivileged processes.

Version-Release number of selected component (if applicable):

usbguard-1.0.0-10.el9

How reproducible:

Non deterministic but it is possible to get system to such state.

Steps to Reproduce:
1. Have machine with say 2.5 GB of memory.
2. Disable swap to put more pressure on the memory: swapoff -a
3. systemctl start usbguard
4. In /etc/security/limits.conf add

test		soft nproc	unlimited
test		hard nproc	unlimited
test		soft sigpending	unlimited
test		hard sigpending	unlimited

5. Compile program

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sched.h>
#include <sys/mman.h>

void alloc_memory(int len)
{
    void *ret = mmap(0, len, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
    if (ret == MAP_FAILED) {
        perror("mmap");
        return;
    }
    memset(ret, 'x', len);
}

int main(int argc, char **argv)
{
    if (argc < 3) {
        fprintf(stderr, "usage: %s <bytes> <processes>\n", argv[0]);
        return 1;
    }

    int bytes = atoi(argv[1]);
    int procs = atoi(argv[2]);

    while (procs-- > 0) {
        switch (fork()) {
            case -1:
                perror("first fork");
                return 1;
            case 0:
                switch (fork()) {
                    case -1:
                        perror("second fork");
                        return 1;
                    case 0:
                        alloc_memory(bytes);
                        sleep(1000);
                        return 0;
                    default:
                        return 0;
                }
            default:
                break;
        }
        sched_yield();  // be somewhat inconspicuous
    }
}

// Thanks to Jiří Jabůrek.

6. Log in as the user test.
7. As root, increase the maximum number of processes for the session, something like

   echo 1000000 > /sys/fs/cgroup/user.slice/user-1000.slice/pids.max

   On my system that value is 7039 by default and I did not find a way to increase it before the user logs in.
8. Run the testing program as user test with some reasonable parameters. On my system, running it a couple of times as

   ./test-alloc 10 10000

   will eventually exhaust the memory. You can also use

   systemctl status $$

   to see the memory used, and to poke the system in other ways as well.
   Eventually the system will get slow, OOM killer will kick in, and if sshd won't get killed, you will be able to check the journal.
9. As root, run journalctl -l | grep kill

Actual results:

Jun 15 18:10:15 machine.example.com kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/usbguard.service,task=usbguard-daemon,pid=338919,uid=0
Jun 15 18:10:15 machine.example.com kernel: Out of memory: Killed process 338919 (usbguard-daemon) total-vm:29264kB, anon-rss:552kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:68kB oom_score_adj:0

Expected results:

usbguard is a security and authorization relevant service, it should set its OOMScoreAdjust to some negative value to avoid being killed before the unprivileged processes.

Additional info: