Bug 1947475
| Summary: | httpd: improve OOM handling w/systemd | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Joe Orton <jorton> |
| Component: | httpd | Assignee: | Luboš Uhliarik <luhliari> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Branislav Náter <bnater> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 9.0 | CC: | bnater, luhliari, praiskup |
| Target Milestone: | beta | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | httpd-2.4.48-10.el9 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-12-07 21:57:54 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Joe Orton
2021-04-08 14:35:16 UTC
Reproducible using mod_oom written by Joe, packaged here: https://copr.fedorainfracloud.org/coprs/praiskup/mod_oom/ My hesitation about this is that when the child is killed by SIGKILL it is otherwise not obvious from the httpd logs that the OOM condition took out a process (and terminated a bunch of active threads). Since the parent will terminate a child via SIGKILL internally, a SIGKILL-killed child is reaped via waitpid() is not logged as an abnormal condition. It is possible we could tweak that logic but not sure. The OOM condition is logged in the journal for the unit, so shows up in "systemctl status httpd" as well as in dmesg etc. Apr 08 15:21:08 systemd[1]: httpd.service: A process of this unit has been killed by the OOM killer. Apr 08 15:22:53 systemd[1]: httpd.service: A process of this unit has been killed by the OOM killer. The question is: is it better for httpd to continue processing requests in other children in an OOM situation, or should it terminate? I think that "continue" is better. A vulnerability which triggers OOM is mitigated from a complete DoS to a partial one. The case Pavel found in production was a intermittent OOM (once per week, for example) and again, it is better in that case for httpd to recover. That OOM has occurred remains obvious from the system logs so should be picked up in monitoring, or, when investigating performance/reliability issues. |