Bug 1290923 - Ceph documentation should tell users to increase netfilter conntrack limits to prevent network issues
Summary: Ceph documentation should tell users to increase netfilter conntrack limits t...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Documentation
Version: 1.3.0
Hardware: All
OS: Linux
low
low
Target Milestone: rc
: 1.3.2
Assignee: Bara Ancincova
QA Contact: ceph-qe-bugs
URL:
Whiteboard:
: 1304004 (view as bug list)
Depends On: 1304004
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-12-11 22:29 UTC by Kyle Squizzato
Modified: 2019-09-12 09:33 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-03-01 08:22:31 UTC
Embargoed:


Attachments (Terms of Use)

Description Kyle Squizzato 2015-12-11 22:29:57 UTC
Description of problem:
Recently customers have been encountering a situation where nf_conntrack modules were loading and causing packets to drop with: 

 nf_conntrack: table full, dropping packet

visible in logs.

How reproducible:
Somewhat

Actual results:
Documentation does not suggest blacklisting nf_conntrack modules to prevent this behavior which can result in the above behavior and networking problems down the road for a cluster. 

Expected results:
Documentation should suggest blacklisting the conntrack modules in the Pre-Installation portion of the Installation guide documentation: 

Additional info:

Creating a file named /etc/modprobe.d/conntrack.conf and adding the following lines: 

blacklist nf_conntrack
blacklist nf_conntrack_ipv6
blacklist xt_conntrack
blacklist nf_conntrack_ftp
blacklist xt_state
blacklist iptable_nat
blacklist ipt_REDIRECT
blacklist nf_nat
blacklist nf_conntrack_ipv4

should do the trick, but a reboot will be required.  Also, note that running any form of iptables rules may re-enable the conntrack modules.

Comment 8 Tanay Ganguly 2016-02-02 07:02:55 UTC
Hi Bara,

Can you please respond to the query asked in Comment 6 and Comment 7.



Also in Redhat Installation guide:

https://gitlab.cee.redhat.com/red-hat-ceph-storage-documentation/doc-Red_Hat_Ceph_Storage_1.3-Installation_Guide_for_Red_Hat_Enterprise_Linux/commit/2f0c61fa9e54c7444e9099922358b67c09f6d995

I don't see the Note Section:

+NOTE: Running any form of the `iptables` rules can enable the `nf-conntrack`
+modules again. Make sure to blacklist the modules before changing the
+`iptables` rules.

But in Ubuntu its present.

Comment 10 Tanay Ganguly 2016-02-03 09:11:28 UTC
Marking this as verified.

Comment 11 Ken Dreyer (Red Hat) 2016-02-03 23:37:40 UTC
We might need to change this recommendation; discussion in bug 1304004

Comment 12 Florian Weimer 2016-02-07 18:50:48 UTC
What does the Ceph traffic look like which triggers this?  Maybe there is a better way to avoid state table exhaustion rather than disabling the stateful firewall completely.

Comment 13 Ken Dreyer (Red Hat) 2016-02-08 21:20:37 UTC
*** Bug 1304004 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.