Bug 1290923 - Ceph documentation should tell users to increase netfilter conntrack limits to prevent network issues
Ceph documentation should tell users to increase netfilter conntrack limits t...
Status: CLOSED CURRENTRELEASE
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Documentation (Show other bugs)
1.3.0
All Linux
low Severity low
: rc
: 1.3.2
Assigned To: Bara Ancincova
ceph-qe-bugs
: Documentation, ZStream
: 1304004 (view as bug list)
Depends On: 1304004
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-11 17:29 EST by Kyle Squizzato
Modified: 2016-03-01 03:22 EST (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-01 03:22:31 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Kyle Squizzato 2015-12-11 17:29:57 EST
Description of problem:
Recently customers have been encountering a situation where nf_conntrack modules were loading and causing packets to drop with: 

 nf_conntrack: table full, dropping packet

visible in logs.

How reproducible:
Somewhat

Actual results:
Documentation does not suggest blacklisting nf_conntrack modules to prevent this behavior which can result in the above behavior and networking problems down the road for a cluster. 

Expected results:
Documentation should suggest blacklisting the conntrack modules in the Pre-Installation portion of the Installation guide documentation: 

Additional info:

Creating a file named /etc/modprobe.d/conntrack.conf and adding the following lines: 

blacklist nf_conntrack
blacklist nf_conntrack_ipv6
blacklist xt_conntrack
blacklist nf_conntrack_ftp
blacklist xt_state
blacklist iptable_nat
blacklist ipt_REDIRECT
blacklist nf_nat
blacklist nf_conntrack_ipv4

should do the trick, but a reboot will be required.  Also, note that running any form of iptables rules may re-enable the conntrack modules.
Comment 8 Tanay Ganguly 2016-02-02 02:02:55 EST
Hi Bara,

Can you please respond to the query asked in Comment 6 and Comment 7.



Also in Redhat Installation guide:

https://gitlab.cee.redhat.com/red-hat-ceph-storage-documentation/doc-Red_Hat_Ceph_Storage_1.3-Installation_Guide_for_Red_Hat_Enterprise_Linux/commit/2f0c61fa9e54c7444e9099922358b67c09f6d995

I don't see the Note Section:

+NOTE: Running any form of the `iptables` rules can enable the `nf-conntrack`
+modules again. Make sure to blacklist the modules before changing the
+`iptables` rules.

But in Ubuntu its present.
Comment 10 Tanay Ganguly 2016-02-03 04:11:28 EST
Marking this as verified.
Comment 11 Ken Dreyer (Red Hat) 2016-02-03 18:37:40 EST
We might need to change this recommendation; discussion in bug 1304004
Comment 12 Florian Weimer 2016-02-07 13:50:48 EST
What does the Ceph traffic look like which triggers this?  Maybe there is a better way to avoid state table exhaustion rather than disabling the stateful firewall completely.
Comment 13 Ken Dreyer (Red Hat) 2016-02-08 16:20:37 EST
*** Bug 1304004 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.