Bug 1294415

Summary: Removing the nf_conntrack module hangs at shutdown using 100% CPU
Product: [Fedora] Fedora Reporter: Daniel Miranda <danielkza2>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 32CC: aka.hector, anon-98kbjd, arthur.mauvezin, ben, bstinson, bugzilla.redhat, bugzilla, cajus, carl, chrfedora, dan, danielkza2, dannymrt1, dasgeekchannel, d.engelbarts, djuran, dominik, dylanxmackenzie, enrico.tagliavini, ezwen-redhatbugzilla, fat.lobyte9, fulminemizzega, gansalmon, henrique.ferreiro, igeorgex, itamar, jbirch, jeroen, jonathan, jorijn, kernel-maint, lantw44, link, luca.giuzzi, madhu.chinakonda, mael.lavault, mail, mail, m.alshafay, marcink7440, mark, mchehab, mjg59, nathanmerkley, orion, pabeni, phea.duch, sandys, stefan, tflink, thaytan, tlfalaska, tn, virtuousfox, wpoely86, xupaddy
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-25 14:56:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Outputs of lsmod and lshw
none
conntrack errors in dmesg
none
ftrace module nf_conntrack
none
100% cpu on rmmod nf_conntrack none

Description Daniel Miranda 2015-12-28 06:57:48 UTC
Created attachment 1109959 [details]
Outputs of lsmod and lshw

Description of problem:

firewalld's cleanup code, as enabled in the default Fedora 23 configuration, does `rmmod nf_conntrack`. When done on shutdown, the process hangs indefinitely using 100% CPU and holding up clean unmounting of filesystems. I can't reproduce it reliably, but it seems uptime is correlated somehow: shutting down after a longer time has a larger probability of failure.


Version-Release number of selected component (if applicable):

Fedora 23 x86_64
kernel-4.4.0-0.rc6.git1.2.fc24.x86_64 from the rawhide-kernel-nodebug repository
firewalld 0.3.14.2 (I don't think it matter though: rmmod is the command that hangs)


How reproducible:

Intermittently


Steps to Reproduce:

Hard to know if configuration-dependent, but for me, shutting down the system with firewalld's cleanup enabled. Stopping firewalld manually or calling rmmod manually outside of shutdown does not hang. Disabling the cleanup also does not hang, with seemly no ill effects.


Actual results:

rmmod hangs and holds up shutdown. Setting up a systemd debug console let me see that it hangs using 100% of a single core. I tried attaching GDB to the process but it does not succeed: GDB hangs before I get a prompt.


Expected results:

rmmod does not hang


Additional info:

I tried to reproduce with kernel-4.2.6-301.fc23.x86_64 to no avail. Unfortunately it also changes the hardware setup: the Wi-fi chipset (Broadcom BCM4350) only works in kernel 4.4. It is possible it is involved somehow, but I couldn't tell.

There are no messages on dmesg of any interest, other than the ebtables unload messaging indicating that the rmmod did not hang (as it is unloaded right afterwards).

I tried installing the conntrack-tools package, but I cannot run the `conntrack` command in the debug console: it fails with a bunch of messages about unknown symbols related to netlink.

I attached the output of lsmod with firewalld running, and a hardware description from lshw. I'm using is a Dell XPS 13 laptop (Late 2015) edition.

Comment 1 Daniel Miranda 2015-12-28 07:16:22 UTC
Just had firewalld's restart hang without shutting down. As before, I cannot attach GDB to the process. Running the `conntrack` utility hangs without output (although there seem to be some errors produced in the kernel log). Fortunately I can get a bit of information from /proc.

$ cat /proc/5701/cmdline
/sbin/rmmodnf_conntrack

$ cat /proc/5701/stack
[<ffffffffa0890478>] nf_conntrack_cleanup_net_list+0x48/0x110 [nf_conntrack]
[<ffffffffa0890bb0>] nf_conntrack_pernet_exit+0x70/0x80 [nf_conntrack]
[<ffffffff81677c82>] ops_exit_list.isra.4+0x52/0x60
[<ffffffff81678198>] unregister_pernet_operations+0x78/0xd0
[<ffffffff81678211>] unregister_pernet_subsys+0x21/0x30
[<ffffffffa0897fac>] nf_conntrack_standalone_fini+0x15/0x69 [nf_conntrack]
[<ffffffff811266e5>] SyS_delete_module+0x1b5/0x210
[<ffffffff8179932e>] entry_SYSCALL_64_fastpath+0x12/0x71
[<ffffffffffffffff>] 0xffffffffffffffff

Comment 2 Daniel Miranda 2015-12-28 07:16:53 UTC
Created attachment 1109963 [details]
conntrack errors in dmesg

Comment 3 Gamaliel 2016-02-08 14:08:42 UTC
I am able to reproduce this bug in my Dell XPS 13 Windows Edition (Broadcom 4350 Wireless). It just continues to say "a stop job is running for firewalld", and keeps adding up hang time from 90 seconds upwards.

Comment 4 Ward 2016-02-22 08:47:07 UTC
I'm also seeing this bug on a MacBook Pro 13" (2015) using a Broadcom BCM43602.

It is not consistently reproducible but long uptimes seem to trigger it.

Comment 5 Luca Giuzzi 2016-03-02 08:48:45 UTC
I have the same problem on a Dell XPS 15 where the wireless controller is a broadcom BCM43602. I have not altered the firewall configuration from stock (apart from opening port 22).

Comment 6 Jason Birch 2016-04-24 16:16:08 UTC
I've noticed this twice under Fedora 24 Alpha, on a Dell XPS 13 (9350) with a Broadcom BCM4350 rev08, during normal day-to-day usage. I suspect the `rmmod` is being triggered as part of upgrading packages (last `dnf upgrade` picked up firewalld-0.4.1-1.fc24.noarch). Impact being that instead of my laptop being unable to shutdown, I instead on upgrade of firewalld chew through battery as a core is pegged at 100%.

I can't really do much with the  process. It's unkillable, _trace/gdb hang when attaching to it, and the stack is empty (which I don't know is useful, but someone else has given those details so I thought I would as well):

# cat /proc/25939/stack 
[<ffffffffffffffff>] 0xffffffffffffffff

`dmesg` and `journalctl` are absolutely littered with unknown symbols for nf_nat and xt_conntrack as in comment #2.

# uname -a
Linux sleepygary 4.5.1-300.fc24.x86_64 #1 SMP Tue Apr 12 18:55:06 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Comment 7 Matthew Garrett 2016-05-13 17:47:42 UTC
Is anybody seeing this *not* using the brcmfmac driver?

Comment 8 Ward 2016-05-18 19:14:47 UTC
All the people I know suffering from this problem are using brcmfmac. So yes, it's probably an issue with this driver.

Currently (on my up to dat F23 system), it happens in 99% of the cases if the uptime is long enough (10 minutes is enough).

Comment 9 dylanxmackenzie 2016-05-21 21:25:58 UTC
Also experiencing this problem on a Dell XPS 15 with the brcmfmac driver on both Fedora 23 and Fedora 24 Beta (Kernel 4.5.4-300.fc24.x86_64. Since I never stop firewalld except on shutdown and my laptop isn't exactly mission critical, I removed the two lines which call unload_firewall_modules() from /usr/lib/python3.5/site-packages/firewall/core/fw.py, which prevents firewalld from calling rmmod when it is stopped or restarted. According to https://bugzilla.redhat.com/show_bug.cgi?id=1031102#c22, firewalld removes these modules to provide a small performance boost. This fixes the problem, but is obviously not ideal.

Comment 10 Daniel Miranda 2016-05-21 23:15:50 UTC
dylanxmackenzie: it's not strictly necessary to modify firewalld's code, there is a CleanupOnExit option in /etc/firewalld/firewalld.conf that you can set to `no`. It also has the side effect of not resetting the iptables rules when the daemon exits, but if you're not usually stopping it manually it shouldn't make a difference.

Comment 11 dylanxmackenzie 2016-05-21 23:28:35 UTC
Daniel Miranda (In reply to Daniel Miranda from comment #10)
> dylanxmackenzie: it's not strictly necessary to modify firewalld's code,
> there is a CleanupOnExit option in /etc/firewalld/firewalld.conf that you
> can set to `no`. It also has the side effect of not resetting the iptables
> rules when the daemon exits, but if you're not usually stopping it manually
> it shouldn't make a difference.

Whoops, that makes things much simpler. Thanks.

Comment 12 dylanxmackenzie 2016-05-21 23:29:16 UTC
Daniel Miranda (In reply to Daniel Miranda from comment #10)
> dylanxmackenzie: it's not strictly necessary to modify firewalld's code,
> there is a CleanupOnExit option in /etc/firewalld/firewalld.conf that you
> can set to `no`. It also has the side effect of not resetting the iptables
> rules when the daemon exits, but if you're not usually stopping it manually
> it shouldn't make a difference.

Whoops, that makes things much simpler. Thanks.

Comment 13 Jason Birch 2016-07-06 03:14:50 UTC
Still noticing this one under 4.6.3-300.fc24.x86_64 for kernel/net/netfilter/nf_conntrack.ko.xz. Is there any other additional information that I can provide or things I can try to help diagnose the cause?

Comment 14 d.engelbarts 2016-09-21 22:43:42 UTC
I actually see rmmod nf_conntrack eating 100% CPU even before rebooting, fresh install on a MBP 15" (recent model). Installed a bunch of stuff and did an update, dnf started to slow down and "top" showed rmmod hogging one core. dnf timed out after a long wait. I first reran the dnf command, this second run it performed like expected, rmmod was still hogging the core. I then rebooted which took a long time but it did eventually proceed.

Comment 15 Laura Abbott 2016-09-23 19:09:59 UTC
*********** MASS BUG UPDATE **************
 
We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 24 kernel bugs.
 
Fedora 24 has now been rebased to 4.7.4-200.fc24.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.
 
If you have moved on to Fedora 25, and are still experiencing this issue, please change the version to Fedora 25.
 
If you experience different issues, please open a new bug report for those.

Comment 16 OlliC 2016-10-04 13:32:49 UTC
Bug still appears on my Thinkpad L460 with kernel 4.7.5-200.fc24.

Comment 17 OlliC 2016-10-09 17:49:29 UTC
Also still happens with 4.7.6-200.fc24.

Comment 18 Dan Siemon 2016-10-17 02:12:24 UTC
I still see this with Fedora 25 beta and kernel 4.8.1-1.fc25.x86_64.

Comment 19 fulminemizzega 2016-11-20 19:04:28 UTC
I still se this with F25 RC1.2 and F24 on a dell XPS 15 9550
(same wifi/bt chip as others have already reported:
https://wikidevi.com/wiki/Dell_Wireless_1830_(DW1830) )

Comment 20 Ben Konrath 2016-12-30 12:05:29 UTC
I'm seeing this on a MacBook Pro 13" (2015) in CentOS now that the BCM43602 is supported in 7.3. It wasn't an issue for CentOS 7.2 because wifi card wasn't supported in that version. I realize this bug is filed against Fedora, not RHEL or CentOS but I wanted to add this note in case it's helpful for diagnosing and solving the problem.

Comment 21 Trevor Flynn 2017-01-05 00:46:08 UTC
Hello I also have this issue on a Dell XPS 9550 with a Broadcom Limited BCM43602. I am running Fedora 25 4.8.15-300.fc25.x86_64.

Comment 22 Maël Lavault 2017-01-12 09:15:00 UTC
Same issue here Macbook pro mid 2015 with BCM43602. Haven't been able to do a clean shutdown since i installed fedora (which i use every day).

Comment 23 Enrico Tagliavini 2017-01-12 09:24:31 UTC
(In reply to Maël Lavault from comment #22)
> Same issue here Macbook pro mid 2015 with BCM43602. Haven't been able to do
> a clean shutdown since i installed fedora (which i use every day).

As mentioned in comment #10 you can set CleanupOnExit=no in firewalld.conf as a temporary workaround.

Btw is there an upstream bug for this? That's a kernel bug in the end, should not hang up when removing a module.

Comment 24 Thomas Cameron 2017-01-17 16:15:39 UTC
I'm pretty sure bugs 1294415, 1293041, and 1397274 are all related. I'm having the same issue on my RHEL7 installations (six of them).

Comment 25 fulminemizzega 2017-02-15 19:21:17 UTC
Created attachment 1250707 [details]
ftrace module nf_conntrack

Hello, I've run some more tests, now with F25 + kernel 4.9.9-200.fc25.x86_64 on XPS 9550 (it's using brcmfmac), I've found that if I stop and start firewalld just after boot everything works fine, instead if I stop it after some time I can reproduce the issue. I've tried to dig deeper but I don't really know what I'm doing. This is what I've found, I hope it can help:
firewalld running -> nf_conntrack has used by = 10 (from lsmod)
firewalld stop works -> lsmod | grep nf_ is empty
firewalld failing to stop -> nf_conntrack has used by = -1 (from lsmod)
used by = -1 means that the module is unloading (function module_refcount from kernel/module.c, there's a comment above); the output of lsmod | grep nf_ in this case is:
nf_reject_ipv6         16384  1 ip6t_REJECT
nf_defrag_ipv6         36864  0
nf_defrag_ipv4         16384  0
nf_conntrack          106496  -1

I booted F25 in rescue mode (add 1 to kernel cmdline), then run
 trace-cmd start -e module -f 'name == nf_conntrack'
then resumed boot, then I run systemctl start/stop firewalld a few times, when it failed I stopped ftrace and saved the report, which I've attached here.
Is this bug also filed upstream?

Comment 26 Trevor Flynn 2017-02-15 20:04:56 UTC
There is a bug filed for this for F25 here: bug 1397274

Comment 27 Justin M. Forbes 2017-04-11 14:40:56 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 24 kernel bugs.

Fedora 25 has now been rebased to 4.10.9-100.fc24.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 26, and are still experiencing this issue, please change the version to Fedora 26.

If you experience different issues, please open a new bug report for those.

Comment 28 OlliC 2017-04-14 18:13:23 UTC
Still happens on Fedora 25 with 4.10.8-200.fc25.x86_64.

Comment 29 Cajus Pollmeier 2017-04-24 07:25:44 UTC
Same on 4.10.10-200.fc25.x86_64. Additionally rmmod eats up 99% of my CPU time and I've to reboot.

Comment 30 Jeroen Tietema 2017-04-29 07:50:23 UTC
Still there on 4.10.12-200.fc25.x86_64

I also tested on the Fedora 26 Alpha live usb, and it is present there as well.

I am running on a MacbookPro Mid 2015 15".

Comment 31 sandeep 2017-06-16 09:51:50 UTC
Created attachment 1288289 [details]
100% cpu on rmmod nf_conntrack

I have similar issue on Fedora 25

LSB Version:	:core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID:	Fedora
Description:	Fedora release 25 (Twenty Five)
Release:	25
Codename:	TwentyFive

Linux menzoberranzan 4.11.4-200.fc25.x86_64 #1 SMP Wed Jun 7 18:28:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

my machine hangs sometimes with 100% cpu and overheating. It is always usually an rmmod nf_conntrack. Screenshot attached

Comment 32 Paolo Abeni 2017-06-21 10:29:00 UTC
Hi,

this should be fixed in upstream kernel v4.12-rc1 and later - so the rawhide kernel should be good. Can you please have a run at it?

Thanks,

Paolo

Comment 33 Jan Schmidt 2017-06-26 12:13:12 UTC
Just encountered it on 4.12.0-0.rc2.git2.2.fc27.x86_64

Comment 34 Phea Duch 2017-07-20 10:31:06 UTC
Still encountering this in 4.13.0-0.rc1.git0.1.fc27. 

I am another Dell XPS 9350 user with the Broadcom chip and have been dealing with this issue for close to a year on Rawhide. The machine does eventually poweroff after firewalld shutdown process times out in ~2 minutes.

Comment 35 Paolo Abeni 2017-07-20 11:03:08 UTC
(In reply to Phea Duch from comment #34)
> Still encountering this in 4.13.0-0.rc1.git0.1.fc27. 

This is quite unexpected: 4.13.0-0.rc1.git0.1.fc27 definitely does not contain the code path triggered by the issue, as described e.g. in comment#1.

Can you please provide the output of:

cat /proc/`pidof rmmod`/stack

?
(to be run when the issue occurs, as root)

Thanks,

Paolo

Comment 36 Fedora End Of Life 2017-07-25 19:41:15 UTC
This message is a reminder that Fedora 24 is nearing its end of life.
Approximately 2 (two) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 24. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '24'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 24 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 37 Fedora End Of Life 2017-08-08 12:35:35 UTC
Fedora 24 changed to end-of-life (EOL) status on 2017-08-08. Fedora 24 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 38 Uwe Köcher 2017-08-11 17:10:15 UTC
(In reply to Paolo Abeni from comment #32)
> Hi,
> 
> this should be fixed in upstream kernel v4.12-rc1 and later - so the rawhide
> kernel should be good. Can you please have a run at it?
> 
> Thanks,
> 
> Paolo

Hej Paolo,

I have a XPS 13 (9359 / late 2015 with Broadcom Wifi). The system had the issue on F24 and now also on a freshly installed F26. For testing purpose, I've installed
* kernel-4.12.5-300.fc26.x86_64
* kernel-core-4.12.5-300.fc26.x86_64 
* kernel-modules-4.12.5-300.fc26.x86_64
from koji. But this does not resolve the issue with the non-stopping dynamic firewalld on reboot or shutdown.

Only the workaround, i.e. setting CleanupOnExit=no in /etc/firewalld/firewalld.conf resolves the issue.

Maybe it is not a kernel issue, instead I think it is something around firewalld and the broadcom wifi.

Note: on the current XPS 13 (late 2016 with Killer Wifi) the issue is non-present.

Comment 39 OlliC 2017-12-21 14:07:21 UTC
Still having this issue on Fedora 27 with a Thinkpad L460 and Broadcom Wifi. 

Does it really need to be reopened?

Comment 40 Martin Bříza 2018-02-02 12:02:50 UTC
Still happening consistently with Fedora 27, I guess about the same hardware as everyone else in this bug:

kernel-4.14.14-300.fc27.x86_64

02:00.0 Network controller: Broadcom Limited BCM43602 802.11ac Wireless LAN SoC (rev 01)


[root@localhost ~]# cat /proc/`pidof rmmod`/stack 
[<ffffffffffffffff>] 0xffffffffffffffff

Comment 41 Cajus Pollmeier 2018-05-11 05:14:52 UTC
F28 still has the issue, too.

Comment 42 Marcin Kozyra 2018-06-05 20:02:24 UTC
f28 still has the same issues with bcm43602

Comment 43 Orion Poplawski 2018-06-20 16:07:34 UTC
root       384     1 99 03:38 ?        213504-04:54:54 /sbin/rmmod nf_defrag_ipv6

4.16.15-300.fc28.x86_64

# cat /proc/`pidof rmmod`/stack
[<0>] 0xffffffffffffffff

178943.826225] CPU: 5 PID: 384 Comm: rmmod Not tainted 4.16.15-300.fc28.x86_64 #1
[178943.826226] Hardware name: Dell Inc. Studio 1747/0J509P, BIOS A11 07/13/2010
[178943.826229] RIP: 0010:_raw_spin_lock+0x10/0x20
[178943.826230] RSP: 0018:ffffc36402707dd8 EFLAGS: 00000246
[178943.826232] RAX: 0000000000000000 RBX: ffffffffc0f7c920 RCX: 0000000000000000
[178943.826233] RDX: 0000000000000001 RSI: ffffffffc0f79eb0 RDI: ffffffffc0f79eb8
[178943.826234] RBP: ffffffffc0f7c988 R08: 0001196b401847e2 R09: 0000000000000101
[178943.826239] R10: 0000000000000000 R11: ffffc36402707df8 R12: 0000000000000000
[178943.826240] R13: 0000000000000000 R14: ffffffffc0f79eb0 R15: ffffffffc0f79ec0
[178943.826243] FS:  00007f8a6db4e0c0(0000) GS:ffff9f86bfd40000(0000) knlGS:0000000000000000
[178943.826244] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[178943.826246] CR2: 0000000005c67000 CR3: 00000001ddac8000 CR4: 00000000000026e0
[178943.826247] Call Trace:
[178943.826253]  inet_evict_bucket.isra.10+0x3b/0x110
[178943.826258]  ? inet_frags_exit_net+0x66/0xb0
[178943.826260]  inet_frags_exit_net+0x61/0xb0
[178943.826266]  ops_exit_list.isra.8+0x36/0x70
[178943.826269]  unregister_pernet_operations+0xb8/0x110
[178943.826272]  unregister_pernet_subsys+0x1d/0x30
[178943.826277]  nf_ct_frag6_cleanup+0x11/0x1d [nf_defrag_ipv6]
[178943.826282]  SyS_delete_module+0x186/0x2b0
[178943.826290]  do_syscall_64+0x74/0x180
[178943.826296]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[178943.826300] RIP: 0033:0x7f8a6d04c917
[178943.826303] RSP: 002b:00007ffd78d6b238 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
[178943.826308] RAX: ffffffffffffffda RBX: 0000565080077820 RCX: 00007f8a6d04c917
[178943.826311] RDX: 000000000000000a RSI: 0000000000000800 RDI: 0000565080077888
[178943.826314] RBP: 0000000000000000 R08: 00007ffd78d6a1b1 R09: 0000000000000000
[178943.826316] R10: 00007f8a6d0bce40 R11: 0000000000000206 R12: 00007ffd78d6b460
[178943.826319] R13: 00007ffd78d6cfd6 R14: 0000565080077260 R15: 0000565080077820
[178943.826326] Code: c0 74 03 31 c0 c3 b8 01 00 00 00 c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 31 c0 ba 01 00 00 00 f0 0f b1 17 <85> c0 75 01 c3 89 c6 e9 14 e4 83 ff 0f 1f 40 00 66 66 66 66 90

Comment 44 Gwendal 2018-07-06 20:47:01 UTC
Still happening, Fedora 28, kernel 4.17.3, Macbook Pro

Comment 45 Justin M. Forbes 2018-07-23 14:59:08 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 27 kernel bugs.

Fedora 27 has now been rebased to 4.17.7-100.fc27.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 28, and are still experiencing this issue, please change the version to Fedora 28.

If you experience different issues, please open a new bug report for those.

Comment 46 Ting-Wei Lan 2018-07-31 03:50:03 UTC
It still happens on 4.17.9-200.fc28.x86_64. 'systemctl stop firewalld' hangs and leaves an unkillable 'rmmod' process on the system.

Comment 47 Yann Soubeyrand 2018-09-01 08:43:45 UTC
Hi,

It's still happening on Fedora 28 with kernel 4.17.18-200.fc28.x86_64 on a Macbook Pro.

I cannot change the version of this bug report though.

Regards

Comment 48 Laura Abbott 2018-10-01 21:23:51 UTC
We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 27 kernel bugs.
 
Fedora 27 has now been rebased to 4.18.10-100.fc27.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.
 
If you have moved on to Fedora 28 or Fedora 29, and are still experiencing this issue, please change the version to Fedora 28 or 29.
 
If you experience different issues, please open a new bug report for those.

Comment 49 Yann Soubeyrand 2018-11-10 10:11:56 UTC
Hi,

It's still happening on Fedora 29 with kernel 4.18.16-300.fc29.x86_64 on a Macbook Pro.

I cannot change the version of this bug report though.

Regards

Comment 50 Marcin Kozyra 2018-11-10 16:46:59 UTC
Ive recreated this ticket under version 29 some time ago. 

https://bugzilla.redhat.com/show_bug.cgi?id=1646609

Comment 51 Eric Garver 2018-11-15 20:29:33 UTC
*** Bug 1646609 has been marked as a duplicate of this bug. ***

Comment 52 Jeremy Cline 2018-12-03 17:29:18 UTC
We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 29 kernel bugs.
 
Fedora 29 has now been rebased to 4.19.5-300.fc29.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.
 
If you experience different issues, please open a new bug report for those.

Comment 53 Yann Soubeyrand 2018-12-09 18:14:26 UTC
Hi,

This bug is still present with the new kernel.

Comment 54 Justin M. Forbes 2019-01-29 16:13:08 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 29 kernel bugs.

Fedora 29 has now been rebased to 4.20.5-200.fc29.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you experience different issues, please open a new bug report for those.

Comment 55 Marcin Kozyra 2019-02-06 17:14:33 UTC
Still not working on the newest kernel.

Comment 56 Ryan 2019-05-13 00:24:56 UTC
Same issue occurring with Dell XPS 9350 using Fedora 30. Upon reboot system gets stuck for 3 minute time out on FirewallD start stop job. Then system never gets past Watchdog error. Does not occur on other distros tested.

Comment 57 Ryan 2019-05-13 00:48:39 UTC
(In reply to Ryan from comment #56)
> Same issue occurring with Dell XPS 9350 using Fedora 30. Upon reboot system
> gets stuck for 3 minute time out on FirewallD start stop job. Then system
> never gets past Watchdog error. Does not occur on other distros tested.

Can confirm this work around does stop the system from hanging on reboot:

CleanupOnExit=no in /etc/firewalld/firewalld.conf from Bug 1646609 which was marked as a duplicate of this issue and has been open since 2018-11.

Comment 58 aka.hector 2019-07-31 06:16:10 UTC
This also causes my Dell XPS 9350 to hang on Fedora 30.

Comment 59 Sam Tuke 2019-08-04 19:09:05 UTC
I appear to be seeing this for the first time on my XPS 9530 using Fedora 30 (I have have used all previous Fedora versions on this device since Fedora ~20.

Comment 60 Justin M. Forbes 2019-08-20 17:39:32 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 29 kernel bugs.

Fedora 29 has now been rebased to 5.2.9-100.fc29.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 30, and are still experiencing this issue, please change the version to Fedora 30.

If you experience different issues, please open a new bug report for those.

Comment 61 Justin M. Forbes 2019-09-17 20:02:25 UTC
*********** MASS BUG UPDATE **************
This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 3 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously.

Comment 62 Link Dupont 2019-11-21 17:56:07 UTC
Description:

I'm able to reproduce this on Fedora 31 (Linux thelio 5.3.11-300.fc31.x86_64 #1 SMP Tue Nov 12 19:08:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux).

Steps to Reproduce:

1) Modify the rules by opening a port: firewall-cmd --add-port 5678/tcp
2) Restart firewalld: systemctl restart firewalld

Expected Results:

firewalld restarts successfully

Actual Results:

The firewalld unit does not restart successfully. Instead rmmod nf_conntrack consumes 100% of the CPU.

Notes:

I am able to kill rmmod to recover the system, but my networking stack is unusable until a reboot.

$ lspci | grep Ethernet
09:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)

I appear to be using the 'igb' NIC module.

# firewall-cmd --get-active-zones
FedoraWorkstation
  interfaces: enp9s0
libvirt
  interfaces: virbr0

# firewall-cmd --list-ports --zone=libvirt

# firewall-cmd --list-ports
1025-65535/udp 1025-65535/tcp

Comment 63 Justin M. Forbes 2020-03-03 16:16:55 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 31 kernel bugs.

Fedora 31 has now been rebased to 5.5.7-200.fc31.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 32, and are still experiencing this issue, please change the version to Fedora 32.

If you experience different issues, please open a new bug report for those.

Comment 64 Marcin Kozyra 2020-03-09 08:00:38 UTC
Can we have this bug marked as critical and finally fixed? Today while leaving my office I clicked on shut down and placed laptop in my backpack. It never turned off, it got so hot I've barely managed to get it out without burning my hand.

Comment 65 nathanmerkley 2020-12-11 18:55:16 UTC
This has plagued me for a while with a few versions of Fedora (currently on a fresh install of 33). For me it happens whenever I start specific software for work in a qemu Windows VM. This causes all kinds of software to hang either indefinitely or for very long periods of time. It takes ~30 seconds for a zsh prompt to appear if I open a new terminal. If I force kill the VM the rmmod process finishes and everything becomes responsive again.
However, if I stop firewalld before starting the VM the problem never appears

Comment 66 Fedora Program Management 2021-04-29 15:52:25 UTC
This message is a reminder that Fedora 32 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '32'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 32 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 67 Ben Cotton 2021-05-25 14:56:21 UTC
Fedora 32 changed to end-of-life (EOL) status on 2021-05-25. Fedora 32 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.