Red Hat Bugzilla – Bug 1304004
1.3.2: Installation: After blacklisting nf-conntrack modules, cannot start firewalld
Last modified: 2016-02-08 16:20:37 EST
Description of problem: After blacklisting nf-conntrack modules, cannot start firewalld Version-Release number of selected component (if applicable):1.3.2 How reproducible: always Steps to Reproduce: 1. Follow the instructions to install calamari from doc: http://10.34.3.139:8080/view/Ceph/job/doc-Red_Hat_Ceph_Storage-Installation_Guide_RHEL%20%28html-single%29/lastSuccessfulBuild/artifact/index.html. 2. As part of instructions, setup firewall on cluster nodes 3. Follow the step to blacklist nf-conntrack modules 4. Reboot the node as per instructions in the doc 5. After node comes up, run any firewalld command on it and it fails as firewalld is not running. 6. Try starting the firewalld. It fails too. Actual results: Expected results: Additional info: Log: --- [cephuser@magna092 ~]$ sudo systemctl start firewalld [cephuser@magna092 ~]$ sudo systemctl enable firewalld [cephuser@magna092 ~]$ sudo firewall-cmd --zone=public --add-port=6800-7300/tcp --permanent success [cephuser@magna092 ~]$ sudo firewall-cmd --reload success [cephuser@magna092 ~]$ firewall-cmd --zone=public --list-ports 6800-7300/tcp [cephuser@magna092 ~]$ sudo vi /etc/modprobe.d/conntrack.conf [cephuser@magna092 ~]$ cat /etc/modprobe.d/conntrack.conf blacklist nf_conntrack blacklist nf_conntrack_ipv6 blacklist xt_conntrack blacklist nf_conntrack_ftp blacklist xt_state blacklist iptable_nat blacklist ipt_REDIRECT blacklist nf_nat blacklist nf_conntrack_ipv4 [cephuser@magna092 ~]$ sudo reboot : : Warning: Permanently added 'magna092,10.8.128.92' (ECDSA) to the list of known hosts. cephuser@magna092's password: Last login: Tue Feb 2 15:15:51 2016 from 10.8.128.2 [cephuser@magna092 ~]$ firewall-cmd --zone=public --list-ports FirewallD is not running [cephuser@magna092 ~]$ sudo systemctl start firewalld ^C (this command did not complete. Had to issue ctrl-c) [cephuser@magna092 ~]$ sudo lsmod | grep conntrack nf_conntrack 105737 2 nf_nat,nf_nat_ipv4
Kyle, in bz 1290923 you recommended blacklisting nf_conntrack, but I'm wondering if that is too blunt a hammer. It seems that this breaks firewalld. http://ceph-users.ceph.narkive.com/koTRsuPb/nf-conntrack-overflow-crashes-osds ...this mailing list post indicates that setting the following sysctl values should do the trick: net.netfilter.nf_conntrack_max = 1024000 net.nf_conntrack_max = 1024000 What do you think?
Kyle confirmed that we should pursue the sysctl option. Harish can you please un-do the work to blacklist nf-conntrack, and then set net.netfilter.nf_conntrack_max = 1024000 net.nf_conntrack_max = 1024000 in /etc/sysctl.conf? To make the setting effective you can reboot (or run `sudo sysctl -p`, although you'd probably want to reboot anyway to un-blacklist the nf_conntrack modules). to verify the changes, execute sudo sysctl -a | grep nf_conntrack
Hi Ken and Kyle, Thanks for the clarification. Firewalld runs after setting the values mentioned in comment 3. I am marking this bz as doc bz. The documentation needs to be changed in the "Blacklist the nf-conntrack modules" of http://10.34.3.139:8080/view/Ceph/job/doc-Red_Hat_Ceph_Storage-Installation_Guide_RHEL%20%28html-single%29/lastSuccessfulBuild/artifact/index.html#install-selinux. Can you please check the existing documentation above and provide the right doc text to doc team based on new change? Regards, Harish LOG: ---- [cephuser@magna023 ~]$ cat /etc/sysctl.conf # System default settings live in /usr/lib/sysctl.d/00-system.conf. # To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file # # For more information, see sysctl.conf(5) and sysctl.d(5). net.netfilter.nf_conntrack_max = 1024000 net.nf_conntrack_max = 1024000 [cephuser@magna023 ~]$ sudo sysctl -a | grep conntrack_max net.netfilter.nf_conntrack_max = 1024000 net.nf_conntrack_max = 1024000 [cephuser@magna023 ~]$ sudo service firewalld status Redirecting to /bin/systemctl status firewalld.service ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2016-02-05 09:39:43 UTC; 4min 4s ago Main PID: 1553 (firewalld) CGroup: /system.slice/firewalld.service └─1553 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid Feb 05 09:39:40 magna023 systemd[1]: Starting firewalld - dynamic firewall daemon... Feb 05 09:39:43 magna023 systemd[1]: Started firewalld - dynamic firewall daemon. [cephuser@magna023 ~]$ sudo firewall-cmd --zone=public --list-ports 6800-7300/tcp
Hi Ken, I followed the steps you provided in the gitlab on one of my cluster nodes. Firewalld comes up fine after the reboot and firewall settings are retained. One correction needed to the steps is that all systemctl cmds should be with sudo. Please change the doc accordingly. Regards, Harish
Thanks Harish! I've added sudo to the systemctl commands in my latest version. For simplicity, I'm going to go ahead and close this as a dup of Kyle's original docs bug, 1290923, and we can track progress there. *** This bug has been marked as a duplicate of bug 1290923 ***