Bug 749687 - systemd cgroup limit management is unreliable
Summary: systemd cgroup limit management is unreliable
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: 16
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-10-27 22:26 UTC by David Strauss
Modified: 2012-01-30 20:58 UTC (History)
7 users (show)

Fixed In Version: systemd-37-11.fc16
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-01-30 20:58:51 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Console log (15.71 KB, text/plain)
2011-10-27 22:30 UTC, David Strauss
no flags Details
F16 verification (26.32 KB, image/png)
2011-10-27 23:16 UTC, David Strauss
no flags Details
tail of messages on F16 (28.75 KB, image/png)
2011-10-27 23:16 UTC, David Strauss
no flags Details

Description David Strauss 2011-10-27 22:26:47 UTC
Description of problem:
When configuring limits (memory, CPU, block I/O, etc.) using systemd's built-in options, the options often fail to apply.

Version-Release number of selected component (if applicable):
Verified as a problem on F16 builds 36-3 and 37-2.

How reproducible:
Reliably.

Steps to Reproduce:
1. Configure a service unit with cgroups limits.
2. systemctl --system daemon-reload
3. systemctl restart NAME.service
4. Check the cgroup limits for the server. They should currently be correct.
5. systemctl restart NAME.service
6. systemctl restart NAME.service # A third restart usually isn't necessary to reproduce the problem, but I did see *once* where it was.
7. Check the cgroup limits for the server. They should currently be correct.

It usually only takes one more restart to lose the ability to write the limits.

Actual results:

cgroups limits get set properly only when a service is (re)started immediately after a systemd daemon reload.

Expected results:

cgroups limits that survive service restarts consistently.

Additional info:

I'm seeing this problem with the F16 packages of systemd 36-3 and 37-2 on F15. I'm currently installing the F16 pre-release to see if I can reproduce it on a clean F16 system.

I've attached a console log showing (1) systemd trying but failing to write to the cgroups for the service (2) various interactions with the service and reloading systemd (3) the service unit file.

Comment 1 David Strauss 2011-10-27 22:30:04 UTC
Created attachment 530584 [details]
Console log

Comment 2 David Strauss 2011-10-27 22:33:59 UTC
(In reply to comment #0)
> Steps to Reproduce:
> 1. Configure a service unit with cgroups limits.
> 2. systemctl --system daemon-reload
> 3. systemctl restart NAME.service
> 4. Check the cgroup limits for the server. They should currently be correct.
> 5. systemctl restart NAME.service
> 6. systemctl restart NAME.service # A third restart usually isn't necessary to
> reproduce the problem, but I did see *once* where it was.
> 7. Check the cgroup limits for the server. They should currently be correct.
> 
> It usually only takes one more restart to lose the ability to write the limits.

Whoops, that's wrong. Here's what I meant to write:

Steps to Reproduce:
1. Configure a service unit with cgroups limits.
2. systemctl --system daemon-reload
3. systemctl restart NAME.service
4. Check the cgroup limits for the server. They should currently be correct.
5. systemctl restart NAME.service
6. systemctl restart NAME.service # A third restart usually isn't necessary to
reproduce the problem, but I did see *once* where it was.
7. Check the cgroup limits for the server. They should currently be *incorrect*.

It usually only takes *two restarts* after the systemd reload to lose the ability to write the limits.

Comment 3 David Strauss 2011-10-27 23:15:55 UTC
Just verified on a fresh installation of F16. See the attached screenshots with the relevant info using a *very* simple service (a simple unit running a Python script that simply sleeps 600 seconds).

Comment 4 David Strauss 2011-10-27 23:16:21 UTC
Created attachment 530587 [details]
F16 verification

Comment 5 David Strauss 2011-10-27 23:16:48 UTC
Created attachment 530588 [details]
tail of messages on F16

Comment 6 Lennart Poettering 2011-11-01 21:04:43 UTC
Fixed now upstream in git.

Comment 7 David Strauss 2011-11-02 01:56:47 UTC
I can vouch for this fix working.

Comment 8 Fedora Update System 2012-01-11 15:01:44 UTC
systemd-37-6.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/systemd-37-6.fc16

Comment 9 Fedora Update System 2012-01-11 20:57:31 UTC
Package systemd-37-6.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-37-6.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-0409/systemd-37-6.fc16
then log in and leave karma (feedback).

Comment 10 Fedora Update System 2012-01-16 02:25:05 UTC
Package systemd-37-7.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-37-7.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-0409/systemd-37-7.fc16
then log in and leave karma (feedback).

Comment 11 Fedora Update System 2012-01-17 20:22:55 UTC
Package systemd-37-8.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-37-8.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-0409/systemd-37-8.fc16
then log in and leave karma (feedback).

Comment 12 Fedora Update System 2012-01-22 22:54:18 UTC
Package systemd-37-10.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-37-10.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-0409/systemd-37-10.fc16
then log in and leave karma (feedback).

Comment 13 Fedora Update System 2012-01-26 22:58:06 UTC
Package systemd-37-11.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-37-11.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-0409/systemd-37-11.fc16
then log in and leave karma (feedback).

Comment 14 Fedora Update System 2012-01-30 20:58:51 UTC
systemd-37-11.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.