Bug 1798046

Summary: Services with Restart=always fail to restart when it has a dependent service with StopWhenUnneeded=true
Product: Red Hat Enterprise Linux 8 Reporter: David Tardon <dtardon>
Component: systemdAssignee: David Tardon <dtardon>
Status: CLOSED ERRATA QA Contact: Frantisek Sumsal <fsumsal>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.0CC: dtardon, systemd-maint-list
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-04 02:02:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David Tardon 2020-02-04 13:17:56 UTC
This bug was initially created as a copy of Bug #1775291

I am copying this bug because: 
The underlying issue is still present in v239. Although v239 doesn't seem to have any problem with the particular configuration described in the RHEL7 bug, there are variants of it that may still result in unintentionally stopped services.


Description of problem:
When a service is configured with "Restart=always" has a "Requires" link to a service that has "StopWhenUnneeded=true" defined, the restart of the parent service will fail to restart.

Version-Release number of selected component (if applicable):
systemd-219-67.el7_7.2.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create a service "A", in this case I will use the HAProxy service as an example.
2. In the service definition, add "[Service] Restart=always" to this definition.
3. Create another service "B", something simple that "cat /proc/cpuinfo" for example.
4. In the definition for "B", add "[Unit] StopWhenUnneeded=true".
5. Not relevant to the bug, but for completeness, in service "B" definition, add "[Service] Type=oneshot RemainAfterExit=yes".
6. In service "A" definition, add "[Unit] After= <Service "B"> Requires=<Service "B">".
7. systemctl start service "A".
8. Confirm that service "A" started successfully and that the HAProxy processes are running ("haproxy-systemd-wrapper" process is the important one).
9. Identify the "haproxy-systemd-wrapper" process PID. It should have a parent process of 1.
10. kill -9 <haproxy-systemd-wrapper process PID>.
11. Search for the haproxy process in the process list. Note that haproxy did not restart.

To see the expected result, comment out the "StopWhenUnneeded=true" in the service "B" definition.

Actual results:
Nov 20 14:59:57 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: haproxy-d3dce9f6-218c-4d8b-90db-233b939922d3.service: main process exited, code=killed, status=9/KILL
Nov 20 14:59:57 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Unit amphora-netns.service is not needed anymore. Stopping.
Nov 20 14:59:57 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Stopped HAProxy Load Balancer.
Nov 20 14:59:57 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Unit amphora-netns.service is not needed anymore. Stopping.
Nov 20 14:59:57 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Unit haproxy-d3dce9f6-218c-4d8b-90db-233b939922d3.service entered failed state.
Nov 20 14:59:57 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: haproxy-d3dce9f6-218c-4d8b-90db-233b939922d3.service failed.
Nov 20 14:59:57 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Stopped Configure amphora-haproxy network namespace.

The haproxy processes did not restart.

Expected results:
Nov 21 11:47:12 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: haproxy-d3dce9f6-218c-4d8b-90db-233b939922d3.service: main process exited, code=killed, status=9/KILL
Nov 21 11:47:12 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Unit haproxy-d3dce9f6-218c-4d8b-90db-233b939922d3.service entered failed state.
Nov 21 11:47:12 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: haproxy-d3dce9f6-218c-4d8b-90db-233b939922d3.service failed.
Nov 21 11:47:13 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: haproxy-d3dce9f6-218c-4d8b-90db-233b939922d3.service holdoff time over, scheduling restart.
Nov 21 11:47:13 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Stopped HAProxy Load Balancer.
Nov 21 11:47:13 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Starting HAProxy Load Balancer...
Nov 21 11:47:13 amphora-53c81d62-544a-4d65-b264-c942a638ec43 systemd: Started HAProxy Load Balancer.

The haproxy processes have been restarted

Additional info:
This issue is impacting Red Hat OpenStack Platform 13+.

This issue is not observed on instances running Ubuntu 18.04.

It may be related to upstream bug: https://github.com/systemd/systemd/issues/1154

This is directly impacting the ability for HAProxy processes to automatically recover should it have a fault or run out of memory due to excessive connections.

Comment 1 David Tardon 2020-02-04 13:28:29 UTC
PR: https://github.com/systemd-rhel/rhel-8/pull/58

Comment 4 Lukáš Nykrýn 2020-04-02 10:02:05 UTC
fix merged to github master branch -> https://github.com/systemd-rhel/rhel-8/pull/58

Comment 11 errata-xmlrpc 2020-11-04 02:02:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: systemd security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4553