Bug 924433 - sysctl.conf cannot be overriden again
Summary: sysctl.conf cannot be overriden again
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: 18
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-03-21 19:00 UTC by Łukasz Trąbiński
Modified: 2013-06-21 18:27 UTC (History)
16 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2013-05-16 02:59:14 UTC


Attachments (Terms of Use)

Description Łukasz Trąbiński 2013-03-21 19:00:20 UTC
Description of problem:

It's very similar problem to Bug #760254
/etc/sysctl.conf or /usr/lib/sysctl.d/fedora.conf or /etc/sysctl.d/00-fedora.conf
not apply changes in system



Version-Release number of selected component (if applicable):

Fedora 18
initscripts-9.42.2-1.fc18.x86_64

How reproducible:

1. Put something to /etc/sysctl.conf or /lib/sysctl.d/fedora.conf  or /etc/sysctl.d/00-fedora.conf
2. Reboot system
3. Check via sysctl changes

  
Actual results:
As you can see, i two copies of /etc/sysctcl in /usr/lib/sysctl.d and /etc/sysctl.d. It's the same file in three different directory

[root@ssh ~]# ll /etc/sysctl.conf
-rw-r--r-- 1 root root 969 Mar 21 16:14 /etc/sysctl.conf

[root@ssh ~]# ll /usr/lib/sysctl.d
total 4
-rw-r--r-- 1 root root 969 Mar 21 18:36 00-fedora.conf

[root@ssh ~]# ll /etc/sysctl.d
total 4
-rw-r--r-- 1 root root 969 Mar 21 18:41 sysctl.conf

[root@ssh ~]# systemctl status systemd-sysctl.service
systemd-sysctl.service - Apply Kernel Variables
          Loaded: loaded (/usr/lib/systemd/system/systemd-sysctl.service; static)
          Active: active (exited) since Thu 2013-03-21 19:07:34 CET; 3min 1s ago
            Docs: man:systemd-sysctl.service(8)
                  man:sysctl.d(5)
         Process: 380 ExecStart=/usr/lib/systemd/systemd-sysctl (code=exited, status=0/SUCCESS)

[root@ssh ~]# cat /usr/lib/sysctl.d/00-fedora.conf 
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv6.conf.all.autoconf = 0
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.all.router_solicitations = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.icmp_ignore_bogus_error_responses = 1

but, you can see below that net.ipv4.ip_forward = 0 

[root@ssh ~]# sysctl -a |grep net.ipv4.ip_forward
net.ipv4.ip_forward = 0

What went wrong?

Comment 1 Lukáš Nykrýn 2013-03-22 09:29:00 UTC
sysctl is now handled by systemd.

Comment 2 Lukáš Nykrýn 2013-03-22 09:55:32 UTC
But I think that this is caused by name of your conf file. 00-fedora.conf precedes in alphabet 00-system.conf, so it is overridden by its values. Try to rename it as 01-fedora.conf

Comment 3 Michal Schmidt 2013-03-22 10:50:24 UTC
But Lukasz also his settings also in /etc/sysctl.conf and that should trump everything.

To debug this, please run this command as root and paste the output here:
SYSTEMD_LOG_LEVEL=debug /usr/lib/systemd/systemd-sysctl

Comment 4 Łukasz Trąbiński 2013-03-22 22:05:39 UTC
Well, I have just renamed /usr/lib/sysctl.d/00-fedora.conf to 01-fedora.conf and now after reboot settings from 01-fedora.conf appear to be applied. I will try make some tests in my other box with fc18 and report here.

Comment 5 Lennart Poettering 2013-04-08 14:17:04 UTC
This is fixed in F19 since a while. (Upstream commit fabe5c0e5fce730aa66e10a9c4f9fdd443d7aeda)

Comment 6 Fedora Update System 2013-04-10 20:14:29 UTC
systemd-201-2.fc18.1 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/systemd-201-2.fc18.1

Comment 7 Fedora Update System 2013-04-11 23:27:34 UTC
Package systemd-201-2.fc18.2:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-201-2.fc18.2'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5452/systemd-201-2.fc18.2
then log in and leave karma (feedback).

Comment 8 Fedora Update System 2013-04-16 00:02:07 UTC
Package systemd-201-2.fc18.4:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-201-2.fc18.4'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5452/systemd-201-2.fc18.4
then log in and leave karma (feedback).

Comment 9 Fedora Update System 2013-04-18 02:39:50 UTC
Package systemd-201-2.fc18.5:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-201-2.fc18.5'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5452/systemd-201-2.fc18.5
then log in and leave karma (feedback).

Comment 10 Ian Pilcher 2013-04-19 07:29:13 UTC
Something is still wonky with systemd-201-2.fc18.5.x86_64.  I'm trying to enable IPv4 forwarding, so I created /etc/sysctl.d/enable-ip-forward.conf, which contains only:

  net.ipv4.ip_forward = 1

But it is unable to override the value set in /usr/lib/sysctl.d/00-system.conf:

[root@localhost sysctl.d]# SYSTEMD_LOG_LEVEL=debug /usr/lib/systemd/systemd-sysctl
parse: /etc/sysctl.conf
parse: /usr/lib/sysctl.d/00-system.conf
parse: /etc/sysctl.d/disable-ipv6.conf
parse: /etc/sysctl.d/enable-ip-forward.conf
Duplicate assignment of net/ipv4/ip_forward in file '/etc/sysctl.d/enable-ip-forward.conf', ignoring.
parse: /usr/lib/sysctl.d/libvirtd.conf
parse: /usr/lib/sysctl.d/nepomuk-inotify.conf
Setting 'net/ipv4/ip_forward' to '0'
Setting 'net/ipv4/conf/default/rp_filter' to '1'
Setting 'net/ipv4/conf/default/accept_source_route' to '0'
Setting 'kernel/sysrq' to '0'
Setting 'kernel/core_uses_pid' to '1'
Setting 'net/bridge/bridge-nf-call-ip6tables' to '0'
Setting 'net/bridge/bridge-nf-call-iptables' to '0'
Setting 'net/bridge/bridge-nf-call-arptables' to '0'
Setting 'net/ipv6/conf/all/disable_ipv6' to '1'
Setting 'fs/aio-max-nr' to '1048576'
Setting 'fs/inotify/max_user_watches' to '524288'

I can workaround the problem by renaming the file to /etc/sysctl.d/00-enable-ip-forward.conf:

[root@localhost sysctl.d]# SYSTEMD_LOG_LEVEL=debug /usr/lib/systemd/systemd-sysctl
parse: /etc/sysctl.conf
parse: /etc/sysctl.d/00-enable-ip-forward.conf
parse: /usr/lib/sysctl.d/00-system.conf
Duplicate assignment of net/ipv4/ip_forward in file '/usr/lib/sysctl.d/00-system.conf', ignoring.
parse: /etc/sysctl.d/disable-ipv6.conf
parse: /usr/lib/sysctl.d/libvirtd.conf
parse: /usr/lib/sysctl.d/nepomuk-inotify.conf
Setting 'net/ipv4/ip_forward' to '1'
Setting 'net/ipv4/conf/default/rp_filter' to '1'
Setting 'net/ipv4/conf/default/accept_source_route' to '0'
Setting 'kernel/sysrq' to '0'
Setting 'kernel/core_uses_pid' to '1'
Setting 'net/bridge/bridge-nf-call-ip6tables' to '0'
Setting 'net/bridge/bridge-nf-call-iptables' to '0'
Setting 'net/bridge/bridge-nf-call-arptables' to '0'
Setting 'net/ipv6/conf/all/disable_ipv6' to '1'
Setting 'fs/aio-max-nr' to '1048576'
Setting 'fs/inotify/max_user_watches' to '524288'

It sure looks like systemd-sysctl is simply processing the files in the order of their relative names, ignoring the documented directory precedence rules.

Comment 11 Michal Schmidt 2013-04-19 15:43:32 UTC
sysctl.d(5) says:

  Files in /etc/ override files with the same name in /usr/lib/ and /run/.

"with the same name" is important.

Further:

  All configuration files are sorted by their filename in alphabetical order,
  regardless in which of the directories they reside, to guarantee that
  a specific configuration file takes precedence over another file with
  an alphabetically later name, if both files contain the same variable setting.

It seems it matches what you're seeing.

I think we should rename 00-system.conf with a higher number to lower its precedence.

Or perhaps we could drop it entirely. There is significant overlap with 50-default.conf shipped by upstream systemd and the bridge-related sysctls are expected to move to a different file (bug 919472).

Comment 12 Michal Schmidt 2013-04-19 15:47:44 UTC
Lukáši, please see my previous comment about 00-system.conf (owned by initscripts).

Comment 13 Bill Nottingham 2013-04-19 16:02:04 UTC
(In reply to comment #11)
> sysctl.d(5) says:
> 
>   Files in /etc/ override files with the same name in /usr/lib/ and /run/.
> 
> "with the same name" is important.
> 
> Further:
> 
>   All configuration files are sorted by their filename in alphabetical order,
>   regardless in which of the directories they reside, to guarantee that
>   a specific configuration file takes precedence over another file with
>   an alphabetically later name, if both files contain the same variable
> setting.
> 
> It seems it matches what you're seeing.

But in terms of providing a clear user interface (especially when overriding packaged files, which can change names), it would be simpler if the algorithm was:

- read all files in override order
- collapse all keys
- apply

That way, an admin can clearly add 99-local.conf (or whatever) to set the keys
they care about without having to guess which packaged config might touch those keys.

Comment 14 Ian Pilcher 2013-04-19 16:56:38 UTC
(In reply to comment #13)
> But in terms of providing a clear user interface (especially when overriding
> packaged files, which can change names), it would be simpler if the
> algorithm was:
> 
> - read all files in override order
> - collapse all keys
> - apply
> 
> That way, an admin can clearly add 99-local.conf (or whatever) to set the
> keys
> they care about without having to guess which packaged config might touch
> those keys.

+1 (bazillion)

As an admin, I should be able to give the files in /etc/sysctl.d names that are meaningful to me.  If I can't, what benefit does the /etc/sysctl.d scheme offer over simply putting everything in /etc/sysctl.conf?

Comment 15 Chris Siebenmann 2013-04-29 17:53:43 UTC
This is a significant change from past behavior. In the past, settings in
any file in /etc/sysctl.d overrode sysctl settings in /usr/lib/. Now they
may not. This makes it extremely difficult for people to reliably override
settings like kernel.sysrq and it also breaks existing systems that have
been doing this reliably so far.

Comment 16 Scott Shambarger 2013-04-30 10:07:49 UTC
I agree, this is a critical issue.  I just attempted an upgrade my F18 router to systemd-201-2 and the networking on my system completely broke.  Relevant entries in /etc/sysctl.d/net.conf are:
---start
# Firewall on bridge.
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1

# Router
net.ipv4.ip_forward = 1

# Controls source route verification, none by default so multicast works
net.ipv4.conf.default.rp_filter = 0
---end

None of these were applied.  So as an FYI, systemd-201-2 as is will break people's systems if pushed to stable.

Comment 17 Scott Shambarger 2013-04-30 21:18:27 UTC
I did a little research, and it appears that systemd-201-2 has changed the documented sysctl.d behavior... man sysctl.d(5) in 197-1 says a config file take precedence over an "alphabetically earlier name" -- 201-2 sysctl.d(5) has changed this to "alphabetically later name"

NOTE: /etc > /run > /usr/lib is still the same, and has _always_ been per-file, with files loaded alphabetically from all directories together -- the change here is the ordering of individual settings in the files (used to be last wins, now it's first wins, similar to .prefix, tmpfiles etc)

I don't see anything in the NEWS file to indicate that the change took place, but I found the commit responsible (by Poettering, fabe5c0e5fce730aa66e10a9c4f9fdd443d7aeda)

I believe the intent of the 00-system.conf file is to ensure defaults (not force settings), and should probably be renamed zz-system.conf to be compatible with this change (is that the best "final" ordering name?).

This makes the bug really against initscripts, and should probably be dependency locked against the newer systemd.

This change will also break any current overrides people are using in their sysctl.d files.. and should probably be clearly noted in the package details!

Comment 18 Michal Schmidt 2013-05-03 17:11:35 UTC
Scott, thanks for researching this.

There really should have been a NEWS item for the inversion of the sysctl.d precedence rules in v198. We should not make such changes in F18 updates. I'm going to undo it in the next F18 updates-testing build.

That still leaves us with apparently counter-intuitive and unpopular rules of sysctl.d overriding. I'll let someone else comment on that.

Comment 19 Fedora Update System 2013-05-07 13:41:44 UTC
systemd-201-2.fc18.6 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/FEDORA-2013-5452/systemd-201-2.fc18.6

Comment 20 Fedora Update System 2013-05-07 13:44:32 UTC
systemd-201-2.fc18.6 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/FEDORA-2013-5452/systemd-201-2.fc18.6

Comment 21 Fedora Update System 2013-05-09 10:03:40 UTC
Package systemd-201-2.fc18.6:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-201-2.fc18.6'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5452/systemd-201-2.fc18.6
then log in and leave karma (feedback).

Comment 22 Fedora Update System 2013-05-16 02:59:14 UTC
systemd-201-2.fc18.6 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 23 Ian Pilcher 2013-05-17 17:45:59 UTC
(In reply to comment #14)
> As an admin, I should be able to give the files in /etc/sysctl.d names that
> are meaningful to me.  If I can't, what benefit does the /etc/sysctl.d
> scheme offer over simply putting everything in /etc/sysctl.conf?

The latest update seems to have changed the behavior again.  Naming the file 00-enable-ip-forward.conf no longer works.

I've given up and moved my settings back to /etc/sysctl.conf.

Comment 24 Michal Schmidt 2013-05-17 18:22:23 UTC
(In reply to comment #23)
> The latest update seems to have changed the behavior again.

Ian,
In comment #18 I said I was going to undo the unexpected change for F18. My goal was to restore the precedence rules that v197 had. Isn't that exactly what happened in systemd-201-2.fc18.6?

Comment 25 Ian Pilcher 2013-05-17 18:32:14 UTC
(In reply to comment #24)
> In comment #18 I said I was going to undo the unexpected change for F18. My
> goal was to restore the precedence rules that v197 had. Isn't that exactly
> what happened in systemd-201-2.fc18.6?

Probably.  I didn't bother to check.

As I mentioned back in comment #14, I don't see the point of having the /etc/sysctl.d directory if the system administrator can't choose expressive file names.


Note You need to log in before you can comment on or make changes to this bug.