Bug 2103517 - Adding file in sysctl.d fails to disable IPV6
Summary: Adding file in sysctl.d fails to disable IPV6
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 36
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-07-04 00:01 UTC by LinuxGeek46@both.org
Modified: 2023-05-25 17:08 UTC (History)
31 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-25 17:08:13 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description LinuxGeek46@both.org 2022-07-04 00:01:56 UTC
Description of problem:


Version-Release number of selected component (if applicable):
250.7-1.fc36


How reproducible:


Steps to Reproduce:
1. Add file 99-local-network.conf to /etc/sysctl.d with the following content.
################################################################################
# Local NetworkManager settings - Specifically to disable IPV6                 #
################################################################################
#
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

2. Reboot the system
3. Check IPV6 status with nmcli

Actual results:
All NICs still show IPV6 active 


Expected results:
IPV6 should be disabled.


Additional info: Lots of additional info here.

########################################################################################
All of my hosts run the Fedora 36 Xfce spin as well as a significant number of packages and configuration changes I perform after the initial default installation. All of my hosts have the most recent updates installed. One of those hosts is my firewall/router and that is the first host on which I disabled IPV6.

You can check to see whether IPV6 is currently active on your Linux host. The nmcli command results below are from my router/firewall host. All the active NICs have both IPV4 and IPV6 addresses. 

[root@wally ~]# nmcli
enp4s0: connected to enp4s0
        "Realtek RTL8111/8168/8411"
        ethernet (r8169), 84:16:F9:04:44:03, hw, mtu 1500
        ip4 default
        inet4 45.20.209.41/29
        route4 45.20.209.40/29 metric 102
        route4 default via 45.20.209.46 metric 102
        inet6 2600:1700:7c0:860:8616:f9ff:fe04:4403/64
        inet6 fe80::8616:f9ff:fe04:4403/64
        route6 fe80::/64 metric 256
        route6 default via fe80::a698:13ff:fee5:fa10 metric 1024
        route6 2600:1700:7c0:860::/64 metric 256

enp1s0: connected to enp1s0
        "Realtek RTL8111/8168/8411"
        ethernet (r8169), 84:16:F9:03:E9:89, hw, mtu 1500
        inet4 192.168.10.1/24
        route4 192.168.10.0/24 metric 101
        inet6 fe80::8616:f9ff:fe03:e989/64
        route6 fe80::/64 metric 256

enp2s0: connected to enp2s0
        "Realtek RTL8111/8168/8411"
        ethernet (r8169), 84:16:F9:03:FD:85, hw, mtu 1500
        inet4 192.168.0.254/24
        route4 192.168.0.0/24 metric 100
        inet6 fe80::8616:f9ff:fe03:fd85/64
        route6 fe80::/64 metric 256

lo: unmanaged
        "lo"
        loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

DNS configuration:
        servers: 192.168.0.52 8.8.8.8 8.8.4.4
        interface: enp4s0

        servers: 192.168.0.52 8.8.8.8
        interface: enp2s0

        servers: 192.168.0.52 8.8.8.8
        interface: enp1s0

Add a Local File to sysctl.d

The /etc/sysctl.conf file can be used to add the configuration settings necessary but it is much better to add a local file to the /etc/sysctl.d directory so that it won’t be overwritten during updates or upgrades. Note that there is already a file named 99-sysctl.conf. You can use that file to set the configuration but I created a different for this purpose, 99-local-network.conf with the following content. That way if the 99-sysctl.conf changes with future updates or upgrades my file will remain untouched. This is not an executable file, it is a configuration file.

################################################################################
# Local NetworkManager settings - Specifically to disable IPV6                 #
################################################################################
#
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

Note: Like many configuration files in configuration directories like this one, the files contained in the directory are read in natural sorted order. That means that the value for a variable declared in a file with a later sort order will override an earlier 
declaration for the same variable. 

A reboot is usually used to cause these changes to take effect but I later learned how to do so without a reboot. I rebooted my system and ran the nmcli command again, this time with the following results. 

[root@wally ~]# nmcli
enp4s0: connected to enp4s0
        "Realtek RTL8111/8168/8411"
        ethernet (r8169), 84:16:F9:04:44:03, hw, mtu 1500
        ip4 default
        inet4 45.20.209.41/29
        route4 45.20.209.40/29 metric 101
        route4 default via 45.20.209.46 metric 101

enp1s0: connected to enp1s0
        "Realtek RTL8111/8168/8411"
        ethernet (r8169), 84:16:F9:03:E9:89, hw, mtu 1500
        inet4 192.168.10.1/24
        route4 192.168.10.0/24 metric 102

enp2s0: connected to enp2s0
        "Realtek RTL8111/8168/8411"
        ethernet (r8169), 84:16:F9:03:FD:85, hw, mtu 1500
        inet4 192.168.0.254/24
        route4 192.168.0.0/24 metric 100

lo: unmanaged
        "lo"
        loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

DNS configuration:
        servers: 192.168.0.52 8.8.8.8 8.8.4.4
        interface: enp4s0

        servers: 192.168.0.52 8.8.8.8
        interface: enp2s0

        servers: 192.168.0.52 8.8.8.8
        interface: enp1s0

This shows that my simple fix worked.

Except …
… that solution only works on one of my twelve Linux computers. After writing the part of this article above, I started installing this fix on all of my other hosts. I had only done one when I realized that this did not always work. I then tried it on one of my VMs and it failed there as well. As best I can tell it only works on one host, the one I use as my firewall and router.

After some additional research on a VM I discovered that these settings can also be issued as commands using sysctl so that a reboot would not be needed. I could enter those commands from the command line and IPV6 would be deactivated.
 
[root@f36vm ~]# sysctl -w net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.all.disable_ipv6 = 1
[root@f36vm ~]# sysctl -w net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6 = 1
[root@f36vm ~]# nmcli
enp0s3: connected to Wired connection 1
        "Intel 82540EM"
        ethernet (e1000), 08:00:27:07:CD:FE, hw, mtu 1500
        ip4 default
        inet4 192.168.0.136/24
        route4 192.168.0.0/24 metric 100
        route4 default via 192.168.0.254 metric 100

lo: unmanaged
<SNIP>

It took some time to research this and determine that the file I created was not being read – or at least not processed – by sysctl during the Linux startup. Or that, if it was, it was being ignored or the settings were being overwritten later by the same variables with different values.  After this I knew that there was no deeper problem preventing it. 


I tested by rebooting the VM and running the following command to set the variables in all of the locations listed in the sysctl man page.

[root@f36vm ~]# sysctl --system
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
kernel.yama.ptrace_scope = 0
* Applying /usr/lib/sysctl.d/50-coredump.conf ...
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
kernel.core_pipe_limit = 16
fs.suid_dumpable = 2
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.sysrq = 16
kernel.core_uses_pid = 1
net.ipv4.conf.default.rp_filter = 2
sysctl: setting key "net.ipv4.conf.all.rp_filter": Invalid argument
net.ipv4.conf.default.accept_source_route = 0
sysctl: setting key "net.ipv4.conf.all.accept_source_route": Invalid argument
net.ipv4.conf.default.promote_secondaries = 1
sysctl: setting key "net.ipv4.conf.all.promote_secondaries": Invalid argument
net.ipv4.ping_group_range = 0 2147483647
net.core.default_qdisc = fq_codel
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
fs.protected_regular = 1
fs.protected_fifos = 1
* Applying /usr/lib/sysctl.d/50-libkcapi-optmem_max.conf ...
net.core.optmem_max = 81920
* Applying /usr/lib/sysctl.d/50-libreswan.conf ...
net.ipv6.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0
* Applying /usr/lib/sysctl.d/50-pid-max.conf ...
kernel.pid_max = 4194304
* Applying /etc/sysctl.d/99-local-network.conf ...
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.conf ...
[root@f36vm ~]#

Then checking the status of the NIC with the nmcli command showed that IPV6 had been disabled and that IPV4 was still up and running. Yes, I do see the other errors in that data stream but I am ignoring them for now. Be sure to read the man page for the sysctl command as it is quite interesting. It provides a method for processing all of the sysctl configuration files with the --service option.

The solution

Now that I understood the true nature of the problem I could create a real solution – even if it might be only a temporary circumvention. But I was able to do so in a way that is in keeping with the original intent of the Linux startup sequence and the sysctl.d method for configuring the kernel. 
I left my new configuration file in place in /etc/sysctl.d. I created the simple Bash script shown below that would run the sysctl --system command and stored it in /usr/local/bin. I then tested that a number of times to ensure that it continued to work as expected.
#!/bin/bash
<SNIP – discarded a bunch of comments to save space>
sysctl --system

I tested this script multiple times before proceeding to ensure that it worked. If you do this be sure to test it multiple times, both with and without rebooting to return to the original kernel configuration. 

Creating the Service

The real key to this solution was to create a new systemd service that would work similarly to the old rc.local SystemV script. In this case I called it the MyStartup.service and I renamed the script I created above and called it MyStartup.sh. To create the service itself I created a new systemd unit file in the /usr/local/lib/systemd/system directory which I also had to create. This service will run once at startup. I named this new file MyStartup.service and added the following content. 
<SNIP – discarded a bunch of comments to save space>
[Unit]
Description=Runs /usr/local/bin/MyStartup.sh

[Service]
ExecStart=/usr/local/bin/MyStartup.sh

[Install]
WantedBy=multi-user.target


The new service is enabled by the command below. Note that the systemctl command searches the new directory by default without any options, arguments, or prodding from me to locate the new unit file.

[root@f36vm ~]# systemctl enable MyStartup.service 
Created symlink /etc/systemd/system/multi-user.target.wants/MyStartup.service → /usr/local/lib/systemd/system/MyStartup.service.

Testing

As soon as the VM rebooted I did a login as root and ran the following command to check the status of IPV6 but it was still active. After a bit more testing I determined that the service was running the commands too soon so I added a command to the MyStartup.sh script sleep for 25 seconds before running the commands. 

#!/bin/bash
<SNIP – discarded a bunch of comments to save space>
# Wait a bit for things to start up and settle. It doesn't work without this.
sleep 25
# Run the sysctl command. 
sysctl --system

I rebooted again and verified the status as soon as I was able to login with the result that the service was still sleeping.

[root@f36vm ~]# systemctl status MyStartup.service 
● MyStartup.service - Runs /usr/local/bin/MyStartup.sh
     Loaded: loaded (/usr/local/lib/systemd/system/MyStartup.service; enabled; vendor preset: disabled)
     Active: active (running) since Fri 2022-07-01 13:24:22 EDT; 14s ago
   Main PID: 667 (MyStartup.sh)
      Tasks: 2 (limit: 14129)
     Memory: 592.0K
        CPU: 1ms
     CGroup: /system.slice/MyStartup.service
             ├─ 667 /bin/bash /usr/local/bin/MyStartup.sh
             └─ 669 sleep 25

After waiting some additional time I checked again and the service had completed successfully with the results clearly shown in the last few journal entries.

Jul 01 13:24:22 f36vm.both.org systemd[1]: Started MyStartup.service - Runs /usr/local/bin/MyStartup.sh.
[root@f36vm ~]# systemctl status MyStartup.service 
○ MyStartup.service - Runs /usr/local/bin/MyStartup.sh
     Loaded: loaded (/usr/local/lib/systemd/system/MyStartup.service; enabled; vendor preset: disabled)
     Active: inactive (dead) since Fri 2022-07-01 13:24:47 EDT; 358ms ago
    Process: 667 ExecStart=/usr/local/bin/MyStartup.sh (code=exited, status=0/SUCCESS)
   Main PID: 667 (code=exited, status=0/SUCCESS)
        CPU: 9ms

Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: net.ipv4.conf.all.send_redirects = 0
Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: net.ipv4.conf.all.accept_redirects = 0
Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: * Applying /usr/lib/sysctl.d/50-pid-max.conf ...
Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: kernel.pid_max = 4194304
Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: * Applying /etc/sysctl.d/99-local-network.conf ...
Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: net.ipv6.conf.all.disable_ipv6 = 1
Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: net.ipv6.conf.default.disable_ipv6 = 1
Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: * Applying /etc/sysctl.d/99-sysctl.conf ...
Jul 01 13:24:47 f36vm.both.org MyStartup.sh[1020]: * Applying /etc/sysctl.conf ...
Jul 01 13:24:47 f36vm.both.org systemd[1]: MyStartup.service: Deactivated successfully.
[root@f36vm ~]#

You can see in the the data output above that the configuration statements in the 99-sysctl.conf file were applied. I also used the nmcli command to verify that IPV6 has been disabled.


Final thoughts

Although just installing the file with the correct kernel parameters and rebooting worked fine on one of my hosts it failed on all of the others I tried. I looked for differences that might explain why it failed on the others while working on my router/firewall, I found nothing that provided a clue as to why this is so. I discovered that this problem exists on hosts with newly installed Fedora 36 without any of the changes and post-installation configuration that I always perform as well as those with those changes.

Comment 1 Zbigniew Jędrzejewski-Szmek 2022-07-04 07:40:51 UTC
Sorry, this is an essay, not a bug report. I'm can't figure out what issue you are trying
to report. 

You have some race during configuration setting. Maybe it's that you set .default. after .all.,
maybe something else. sysctl doesn't support the full syntax of the config files, use systemd-sysctl
instead.

Try to use the same pattern as other settings in /usr/lib/sysctl.d/50-default.conf.
I guess something like this should work:

net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.*.disable_ipv6 = 1
-net.ipv6.conf.all.disable_ipv6

Use 'systemd-analyze cat-config sysctl.d/' to check that you have the expected configuration
and no stray files.

Comment 2 LinuxGeek46@both.org 2022-07-04 12:23:35 UTC
I apologize sir - I do tend to be verbose. I was trying to ensure you had as much information as I could provide. 

I did try these statements in my file and they do work when I run sysctl --system. The following is the result of the systemd-analyze cat-config sysctl.d/ command. Other than disabling IPV6 redirects for XFRM/IPsec, the output below is the only other mention of IPV6. No stray files were found. 

# /etc/sysctl.d/99-local-network.conf
################################################################################
# Local Network settings - Specifically to disable IPV6                        #
# Install this file in the /etc/sysctl.d directory.                            #
################################################################################
#
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1

Replacing my 2 lines with your 3 lines in my /etc/sysctl.d/99-local-network.conf file results in the same with IPV6 still activated.

[root@f36vm ~]# nmcli
enp0s3: connected to Wired connection 1
        "Intel 82540EM"
        ethernet (e1000), 08:00:27:07:CD:FE, hw, mtu 1500
        ip4 default
        inet4 192.168.0.136/24
        route4 192.168.0.0/24 metric 100
        route4 default via 192.168.0.254 metric 100
        inet6 fe80::ffa4:fcef:19f7:8a5e/64
        route6 fe80::/64 metric 1024

Even running sysctl --system from CLI with your 3 lines does not disable IPV6 although running that command with my 2 lines from CLI does disable IPV6. But the MyStartup.service does not disable IPV6 unless I add a delay before running the sysctl --system command in the script launched by the service.

I agree that the problem is likely a race condition during late startup but that should not occur. I am happy to provide any additional information you need.Let me know how I can help.

Comment 3 LinuxGeek46@both.org 2022-07-04 12:24:48 UTC
PS: I will research systemd-sysctl and let you know the results.

Thx.

Comment 4 LinuxGeek46@both.org 2022-08-16 18:50:08 UTC
FYI ... I have been experimenting with this recently. I have also found that attempting to set vm.swappiness this way does not work. However I have also discovered that using a new service ( Like rc,local ) MyStartup.service which waits 2 seconds and then runs MyStartup.sh successfully sets all needed kernelopts. I hope this information helps:


[root@testvm3 system]# cat /etc/systemd/system/multi-user.target.wants/MyStartup.service
#!/bin/bash
################################################################################
#                            MyStartup.service                                 #
#                                                                              #
# This file is the service unit file for MyStartup. It calls the script,       #
# /usr/local/bin/ MyStartup.sh which runs any commands or other scripts        #
# required at startup time.                                                    #
#                                                                              #
#                                                                              #
#                                                                              #
# This program should be placed in /usr/local/lib/systemd/system/.             #
# Create a symlink to it from the /etc/systemd/system directory.               #
#                                                                              #
# Change History                                                               #
# 2022/07/01 David Both    Original code.                                      #
#                                                                              #
################################################################################
################################################################################
################################################################################
#                                                                              #
#  Copyright (C) 2022 David Both                                               #
#  LinuxGeek46                                                        #
#                                                                              #
#  This program is free software; you can redistribute it and/or modify        #
#  it under the terms of the GNU General Public License as published by        #
#  the Free Software Foundation; either version 2 of the License, or           #
#  (at your option) any later version.                                         #
#                                                                              #
#  This program is distributed in the hope that it will be useful,             #
#  but WITHOUT ANY WARRANTY; without even the implied warranty of              #
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the               #
#  GNU General Public License for more details.                                #
#                                                                              #
#  You should have received a copy of the GNU General Public License           #
#  along with this program; if not, write to the Free Software                 #
#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA   #
#                                                                              #
################################################################################
################################################################################

[Unit]
Description=Runs /usr/local/bin/MyStartup.sh

[Service]
ExecStart=/usr/local/bin/MyStartup.sh

[Install]
WantedBy=multi-user.target


################################################################################
################################################################################

[root@testvm3 system]# cat /usr/local/bin/MyStartup.sh
#!/bin/bash
################################################################################
#                               MyStartup.sh                                   #
#                                                                              #
# This script is used during systemd startup or from the command line to       #
# run local startup commands and scripts as necessary. It can be run from the  #
# command line but should be run as a systemd service at Linux startup.        #
#                                                                              #
# This file should be located in /usr/local/bin.                               #
#                                                                              #
# This file is called by the local MyStartup.service during Linux startup.     #
#                                                                              #
# Change History                                                               #
# 2022/07/01 David Both    Original code.                                      #
#                                                                              #
#                                                                              #
################################################################################
################################################################################
################################################################################
#                                                                              #
#  Copyright (C) 2022 David Both                                               #
#  LinuxGeek46                                                        #
#                                                                              #
#  This program is free software; you can redistribute it and/or modify        #
#  it under the terms of the GNU General Public License as published by        #
#  the Free Software Foundation; either version 2 of the License, or           #
#  (at your option) any later version.                                         #
#                                                                              #
#  This program is distributed in the hope that it will be useful,             #
#  but WITHOUT ANY WARRANTY; without even the implied warranty of              #
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the               #
#  GNU General Public License for more details.                                #
#                                                                              #
#  You should have received a copy of the GNU General Public License           #
#  along with this program; if not, write to the Free Software                 #
#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA   #
#                                                                              #
################################################################################
################################################################################
################################################################################

# Place your startup commands here.

################################################################################
# Activate all kernelopts files in /etc/sysctl.d                               #
################################################################################
sleep 2
sysctl -p /etc/sysctl.d/*

################################################################################

Comment 5 David Tardon 2022-09-27 08:53:28 UTC
The settings are applied by systemd-sysctl on boot, as can be checked by

# cat /proc/sys/net/ipv6/conf/default/disable_ipv6

or

# sysctl -n net.ipv6.conf.all.disable_ipv6

. But the .default. setting is only applied to interfaces created after that. The .all. setting should affect all existing interfaces, but it seems it doesn't -> moving to kernel for investigation.

Comment 6 Ben Cotton 2023-04-25 17:32:46 UTC
This message is a reminder that Fedora Linux 36 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 36 on 2023-05-16.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '36'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 36 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 7 Ludek Smid 2023-05-25 17:08:13 UTC
Fedora Linux 36 entered end-of-life (EOL) status on 2023-05-16.

Fedora Linux 36 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.