Bug 2068699 - Disable systemd-oomd when MATE desktop is used
Summary: Disable systemd-oomd when MATE desktop is used
Keywords:
Status: ON_QA
Alias: None
Product: Fedora
Classification: Fedora
Component: mate-desktop
Version: 37
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Wolfgang Ulbrich
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 2094477 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-03-26 02:32 UTC by Ian McInerney
Modified: 2022-08-09 13:42 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)

Description Ian McInerney 2022-03-26 02:32:47 UTC
Description of problem:

Fedora 34 included a change proposal that affected all Fedora variants by enabling the systemd-oomd out of memory killer instead of the original earlyoom (https://fedoraproject.org/wiki/Changes/EnableSystemdOomd). This memory killer is cgroups based, so it will kill all processes in the cgroup that it thinks is putting the most memory pressure on the system.

The MATE desktop environment appears to not give launched processes new cgroups, so the systemd-oomd killer will kill the entire graphical session when the memory pressure threshold is reached.

The change proposal people are stubbornly insistent that the change is not broken even though it is causing the graphical session to be killed on other spins because it works fine on GNOME/KDE, and they insist all other spins make modifications to their packages to disable systemd-oomd if it causes issues (even though it sounds like a lot of spins have issues...).


I have personally experienced systemd-oomd killing my graphical MATE session on multiple occasions recently, and it is making me very frustrated because it kills EVERYTHING and I lose the work and windows that were open.

Please disable systemd-oomd in the MATE spin, and when using the MATE desktop until upstream MATE implements cgroups support.



I have been experiencing this on my Fedora 34 laptop.

Comment 1 Ian McInerney 2022-03-26 02:34:41 UTC
Link to similar thread about i3/sway having problems, and where the change owners refuse to see this as an overall detrimental change: https://bugzilla.redhat.com/show_bug.cgi?id=1933494

Comment 2 Wolfgang Ulbrich 2022-03-27 10:05:15 UTC
Can you please file out a report at upstream in result cgroups support can be implemented?
https://github.com/mate-desktop/mate-session-manager/issues
Please give them the same detailed description like here.
In the meanwhile i will look how to disable systemd-oomd in mate desktop spin.
But sorry i am not sure if it is possible to do and how to do it.
Personal i nevser run in this issue with my 32GB RAM system.
Systemd-oomd i enabled and i never noticed X stops working. 

Is it possible to uninstall systemd-oomd and earlyoom needs to be installed, or how did you fix it in your installation?

Comment 3 Wolfgang Ulbrich 2022-03-27 10:45:20 UTC
It seems that preventing systemd-oomd-defaults from installation is the solution.
https://fedoraproject.org/wiki/Changes/EnableSystemdOomd#Should_spins_that_don.27t_put_processes_in_separate_cgroups_be_excluded_from_this_change.3F

```
Should spins that don't put processes in separate cgroups be excluded from this change?

That will be left up to the maintainers of those spins. Based on feedback, the current plan is to enable systemd-oomd with the specified configuration by default to minimize fragmentation on the Fedora install base (the Upgrade/Compatibility section as been updated to reflect this). A separate subpackage, "systemd-oomd-defaults", controls the policy for systemd-oomd and excluding it or removing it (and performing a systemctl daemon-reload) will prevent systemd-oomd from killing anything; without a policy systemd-oomd doesn't act. 
```

Can you please confirm that removing systemd-oomd-defaults fixes the problems for you.
Please enable systemd-oomd before you uninstall systemd-oomd.
I like to know if the TODO works with default unit state.

I can't test it for myself as i never ran in this issue.

Comment 4 Wolfgang Ulbrich 2022-03-27 11:03:06 UTC
And do you use earlyoom instead?

Comment 5 Fedora Update System 2022-04-07 15:28:09 UTC
FEDORA-2022-b66b743517 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-b66b743517

Comment 6 Fedora Update System 2022-04-07 19:05:48 UTC
FEDORA-2022-8d14f0b9ec has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-8d14f0b9ec

Comment 7 Fedora Update System 2022-04-08 20:02:44 UTC
FEDORA-2022-b66b743517 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-b66b743517`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-b66b743517

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2022-04-08 20:52:12 UTC
FEDORA-2022-324adf3898 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-324adf3898`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-324adf3898

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Wolfgang Ulbrich 2022-04-09 18:35:06 UTC
Note: Update must be installed with "dnf update mate-desktop* --best --allowerasing" which will be remove systemd-oomd-defaults.

Comment 10 Fedora Update System 2022-04-11 14:57:13 UTC
FEDORA-2022-8d14f0b9ec has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-8d14f0b9ec`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-8d14f0b9ec

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2022-04-15 14:11:39 UTC
FEDORA-2022-324adf3898 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-324adf3898`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-324adf3898

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2022-04-15 14:31:53 UTC
FEDORA-2022-b66b743517 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-b66b743517`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-b66b743517

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2022-04-17 22:20:48 UTC
FEDORA-2022-324adf3898 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 14 Fedora Update System 2022-04-17 22:41:32 UTC
FEDORA-2022-b66b743517 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 15 Fedora Update System 2022-05-07 04:19:09 UTC
FEDORA-2022-8d14f0b9ec has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Davide Repetto 2022-05-08 13:15:10 UTC
The need to use "--best --allowerasing" means most users won't get it.
Wouldn't it be better to use an "obsoletes", which dnf enforces by default during updates?

Moreover, are we sure systemd-oomd will be domesticated by just removing systemd-oomd-defaults?
I'm not sure, because documentation says it would use it's compile-time defaults, as portrayed in /etc/systemd/oomd.conf.

And from Fedora-MATE_Compiz-Live-x86_64-36-20220506.n.0.iso :

[liveuser@localhost-live ~]$ cat /etc/systemd/oomd.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it under the
#  terms of the GNU Lesser General Public License as published by the Free
#  Software Foundation; either version 2.1 of the License, or (at your option)
#  any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file, or by creating "drop-ins" in
# the oomd.conf.d/ subdirectory. The latter is generally recommended.
# Defaults can be restored by simply deleting this file and all drop-ins.
#
# Use 'systemd-analyze cat-config systemd/oomd.conf' to display the full config.
#
# See oomd.conf(5) for details




Note: My Documentation Source is https://www.freedesktop.org/software/systemd/man/oomd.conf.html

========================================== Excerpt ========================================================
The default configuration is set during compilation, so configuration is only needed when it is necessary to deviate from those defaults. Initially, the main configuration file in /etc/systemd/ contains commented out entries showing the defaults as a guide to the administrator. Local overrides can be created by editing this file or by creating drop-ins, as described below. Using drop-ins for local configuration is recommended over modifications to the main configuration file.
==================================================================================================

Comment 17 Davide Repetto 2022-05-08 13:21:02 UTC
Sorry. I mistakenly deleted the most important bit... This is the output, complete with the compile-time defauls:

[liveuser@localhost-live ~]$ cat /etc/systemd/oomd.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it under the
#  terms of the GNU Lesser General Public License as published by the Free
#  Software Foundation; either version 2.1 of the License, or (at your option)
#  any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file, or by creating "drop-ins" in
# the oomd.conf.d/ subdirectory. The latter is generally recommended.
# Defaults can be restored by simply deleting this file and all drop-ins.
#
# Use 'systemd-analyze cat-config systemd/oomd.conf' to display the full config.
#
# See oomd.conf(5) for details

[OOM]
#SwapUsedLimit=90%
#DefaultMemoryPressureLimit=60%
#DefaultMemoryPressureDurationSec=30s

Comment 18 Wolfgang Ulbrich 2022-05-08 16:32:41 UTC
(In reply to Davide Repetto from comment #16)
> The need to use "--best --allowerasing" means most users won't get it.
> Wouldn't it be better to use an "obsoletes", which dnf enforces by default
> during updates?
> 
Using a conflict is the correct way in fedora to uninstall another package.
And with a new installation you won't get this update question.
So you will see it it only one time when updating an existing installation.

> Moreover, are we sure systemd-oomd will be domesticated by just removing
> systemd-oomd-defaults?
> I'm not sure, because documentation says it would use it's compile-time
> defaults, as portrayed in /etc/systemd/oomd.conf.

See https://fedoraproject.org/wiki/Changes/EnableSystemdOomd#Should_spins_that_don.27t_put_processes_in_separate_cgroups_be_excluded_from_this_change.3F
```
 A separate subpackage, "systemd-oomd-defaults", controls the policy for systemd-oomd and excluding it or removing it (and performing a systemctl daemon-reload) will prevent systemd-oomd from killing anything; without a policy systemd-oomd doesn't act.
```
With a previous update i set a present for earlyoom, means it is enable when installed. Tip from
https://bugzilla.redhat.com/show_bug.cgi?id=1933494#c23 

Well, if you have sufficient memory it is save to uninstall earlyoom, which is possible with last setup.
For my 32GB Ram box i don't use a out-of-memory killer.
```
Recommends:  earlyoom
Conflicts: systemd-oomd-defaults
```
At least the right fix for this problem should be adding cgroup support to mate-desktop, to avoid that systemd-oomd will kill the whole x-session.
There is a upstream report. https://github.com/mate-desktop/mate-desktop/issues/519

Comment 19 Ben Cotton 2022-05-12 16:58:29 UTC
This message is a reminder that Fedora Linux 34 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '34'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 34 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 20 Wolfgang Ulbrich 2022-06-07 17:25:05 UTC
*** Bug 2094477 has been marked as a duplicate of this bug. ***

Comment 21 Ben Cotton 2022-08-09 13:42:28 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 37 development cycle.
Changing version to 37.


Note You need to log in before you can comment on or make changes to this bug.