Bug 1707714
| Summary: | add rules to iptables takes time longer | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Nakajima Akira <nakajima.akira> |
| Component: | iptables | Assignee: | Phil Sutter <psutter> |
| Status: | CLOSED DUPLICATE | QA Contact: | qe-baseos-daemons |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 8.0 | CC: | iptables-maint-list, rkhan, todoleza |
| Target Milestone: | rc | ||
| Target Release: | 8.0 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-12-05 10:09:32 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hi, (In reply to Nakajima Akira from comment #0) > Description of problem: > > I added 30,000 rules to iptables, > RHEL 6/7, Fedora 28/29 takes 8 minutes. > But RHEL 8 takes more than 40 minutes.(same as RHEL 8 Beta) > What's going on? RHEL8 ships iptables-nft as iptables, a drop-in replacement to the legacy tool. It uses nftables in background, so the code base is largely different (both in user and kernel space). While some differences in performance are to be expected, a factor of 5 when compared to RHEL7 is of course not acceptable. Are you just sanity testing or is there a real use-case involving many repeated calls to iptables? Usually projects requiring this feed suitable input into iptables-restore to eliminate the program startup overhead. Thanks, Phil > Are you just sanity testing This is only just sanity testing. There are no plans to use it in a commercial environment. > it uses nftables in background On Fedora29, it is not reproduced, Does it not use nftables in background on Feodra29? Hi Nakajima-san, (In reply to Nakajima Akira from comment #2) > > Are you just sanity testing > This is only just sanity testing. There are no plans to use it in a > commercial environment. Thanks for clarifying! Note that using iptables-restore instead reduces run-time of your test to near insignificance. For reference, this is the script I used: | #! /bin/bash | | dump=/tmp/ipt_add_rules.dump | | echo '*filter' >$dump | echo ':INPUT ACCEPT [0:0]' >>$dump | for dport in $(seq 60001 60030); do | for sport in $(seq 50001 51000); do | echo "-A INPUT -p tcp --sport ${sport} --dport ${dport} -j ACCEPT" | done | done >>$dump | echo 'COMMIT' >>$dump | | time iptables-restore $dump > > it uses nftables in background > On Fedora29, it is not reproduced, Does it not use nftables in background on > Feodra29? No, Fedora still uses legacy iptables by default. You may switch to using iptables-nft using 'alternatives --set iptables /usr/sbin/iptables-nft'. Cheers, Phil Your script completed in an instant(< 1sec). Thanks for good script. As a test, I did try iptables latest git on Fedora 29. My script ipt.sh took over 40 minutes with git ver /usr/local/sbin/iptables-nft (-> xtables-nft-multi). On the other hand, I did try iptables 1.8.1 on RHEL 8. It took less than 8 minutes (iptables -> xtables-legacy-multi). From these results, is this a specification of iptables and no problems? (In reply to Nakajima Akira from comment #4) > Your script completed in an instant(< 1sec). Thanks for good script. Note that I cheated a bit: My script doesn't account for the dump file setup. But in general, using iptables-restore instead of individual iptables calls is the best thing one can do to improve performance with large rulesets. This is not just because of the obvious benefits (program startup and cache fetching happen just once) but also because upstream focusses more on that in performance considerations. > As a test, I did try iptables latest git on Fedora 29. > My script ipt.sh took over 40 minutes with git ver > /usr/local/sbin/iptables-nft (-> xtables-nft-multi). > On the other hand, I did try iptables 1.8.1 on RHEL 8. It took less than 8 > minutes (iptables -> xtables-legacy-multi). > > From these results, is this a specification of iptables and no problems? A big problem is that cache updates take longer, also the code is overall not as optimized as legacy one. This overhead has drastic effects in a use-case like yours where iptables is called over and over again - each time it fetches the kernel ruleset (which becomes larger and larger due to the added rules). In theory, For the simple task of appending a rule to a chain, iptables-nft wouldn't need a full cache: Making sure the requested table and chain exists is sufficient. I have a PoC which avoids fetching the rules in this case and it performs better than iptables-legacy. Introducing more finer grained cache handling increases complexity though, and upstream wasn't quite happy about that, also because iptables-nft-restore will effectively fetch the cache just once and be done with that. So I'm optimizing a less than optimal use-case and hence feasibility is questionable. My current plan is to simplify cache handling overall a bit and then respin my PoC. So maybe there will be more to come, but in general I wouldn't consider this a bug but rather lack of feature simply because you want to use iptables-restore (either legacy or nft) anyway if you want performance with large rulesets. Cheers, Phil This is a very similar complaint as in Bug 1744634, I'm merging both by marking this one as a duplicate. worth mentioning though, described use-case here is perfectly covered by iptables rebase to 1.8.4 in RHEL8.2. *** This bug has been marked as a duplicate of bug 1744634 *** |
Description of problem: I added 30,000 rules to iptables, RHEL 6/7, Fedora 28/29 takes 8 minutes. But RHEL 8 takes more than 40 minutes.(same as RHEL 8 Beta) What's going on? Actual results: ipt.sh ========================================================================== #!/bin/bash iptables -F INPUT for dport in $(seq 60001 60030) do for sport in $(seq 50001 51000) do iptables -A INPUT -p tcp --sport ${sport} --dport ${dport} -j DROP done done iptables -F INPUT ========================================================================== [root@rhel69 ~]# time ./ipt.sh real 6m39.418s user 1m57.781s sys 4m15.791s [root@rhel69 ~]# uname -a Linux rhel69 2.6.32-696.el6.x86_64 #1 SMP Tue Feb 21 00:53:17 EST 2017 x86_64 x86_64 x86_64 GNU/Linux [root@rhel69 ~]# iptables -V iptables v1.4.7 [root@rhel76 ~]# time ./ipt.sh real 7m28.054s user 2m5.478s sys 5m24.287s [root@rhel76 ~]# uname -a Linux rhel76 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux [root@rhel76 ~]# iptables -V iptables v1.4.21 [root@rhel8 ~]# time ./ipt.sh real 42m31.205s user 16m16.994s sys 25m48.391s [root@rhel8 ~]# uname -a Linux rhel8 4.18.0-80.el8.x86_64 #1 SMP Wed Mar 13 12:02:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux [root@rhel8 ~]# iptables -V iptables v1.8.2 (nf_tables) [root@fedora28~]# time ./ipt.sh real 8m18.185s user 2m48.646s sys 5m5.898s [root@fedora28~]# uname -a Linux fedora28 4.16.3-301.fc28.x86_64 #1 SMP Mon Apr 23 21:59:58 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux [root@fedora28~]# iptables -V iptables v1.6.2 [root@fedora29 ~]# time ./ipt.sh real 8m8.308s user 2m38.883s sys 5m0.625s [root@fedora29 ~]# uname -a Linux fedora29 4.18.16-300.fc29.x86_64 #1 SMP Sat Oct 20 23:24:08 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux [root@fedora29 ~]# iptables -V iptables v1.8.0 (legacy) Additional info: All is running on KVM Guest. (each Guest is allocated MEM 8GB, CPU Xeon E3-1220 V2 3.1 GHz)