Bug 2180423
Summary: | kworker/1:0+cifsiod is using 100% of a CPU core and freezes my server | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | daniel_ulmer | ||||
Component: | cifs-utils | Assignee: | Nobody <nobody> | ||||
Status: | CLOSED MIGRATED | QA Contact: | xiaoli feng <xifeng> | ||||
Severity: | unspecified | Docs Contact: | |||||
Priority: | unspecified | ||||||
Version: | CentOS Stream | CC: | bstinson, bugzilla, Eldon.Rosenberg, fabien.magagnosc, jwboyer, ryan.brothers, xzhou | ||||
Target Milestone: | rc | Keywords: | MigratedToJIRA | ||||
Target Release: | --- | ||||||
Hardware: | x86_64 | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2023-09-23 11:35:38 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Hi there, we got the same issue (CentOS Stream 9), the CPUX is stucks, we can see massive CPU usage on a single kworker+cifsiod And we find that the culpid (after testing multiple CIFS version, even SMB) was the upgrade of the cifs-utils, from 6.14-1.el9 to 7.0-1.el9 which happened before the week end (we didn't saw it then, no load) Here some details, maybe to help : We are mounting a Windows 2012 server, using CIFS 3.0 : //server/directory /mnt/directory cifs nobrl,rw,noperm,user=XXXXX,pass=XXXXX,mfsymlinks,uid=2000,gid=2000,dir_mode=0755,file_mode=0755,vers=3.0 0 0 Here, the latest kernel as per the 22th of March 2023 installed : Linux WWWWW 5.14.0-285.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 7 17:32:48 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux Here, the situation DNF wise (after we locked to the previous cifs-utils) Installed Packages cifs-utils.x86_64 6.14-1.el9 @baseos Available Packages cifs-utils.x86_64 6.11-5.el9 baseos cifs-utils.x86_64 6.14-1.el9 baseos cifs-utils.x86_64 7.0-1.el9 baseos Hello, There is a similar bug bz2177562 and it is fixed from kernel-5.14.0-301.el9. Could you try to test on kernel-5.14.0-301.el9 or later version? Thanks. @xiaoli feng => To be clear, you means update : _ udapte the CIFS utils pacakges _ update the kernel to at least 5.14.0-301.el9 isn't it ? Right now this is what I run perfectly stable : [tecnotree@teamcity-repository9 ~]$ sudo dnf --showduplicates list cifs-utils Last metadata expiration check: 2:10:27 ago on Mon 15 May 2023 08:12:29 AM EEST. Installed Packages cifs-utils.x86_64 6.14-1.el9 @baseos Available Packages cifs-utils.x86_64 6.11-5.el9 baseos cifs-utils.x86_64 6.14-1.el9 baseos cifs-utils.x86_64 7.0-1.el9 baseos [tecnotree@teamcity-repository9 ~]$ uname -a Linux teamcity-repository9.tecnotree.com 5.14.0-307.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed May 3 06:16:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux And it's perfectly stable since we rollbacked the cifs-utils and any kernels I cannot access the bug 2177562 so I'm not even sure this is related. But I'll try to update this week end, and see if the server can sustain the week end (it uses to break after few hours) I am running into this issue too after upgrading to 5.14.0-284.11.1.el9_2. Will a fix for it be released soon? Same here, most of the time when waking up from suspend. Sorry for the late response but after the latest update of CentOS it worked again for me. That's why i didn't response earlier. I left everything the same and didn't change anything and it startet working again. The update was somewhere around early Mai. If that was a kernel update for CentOS 9 stream then I suspect it hasn't been released yet for RHEL (and therefore Alma and Rocky Linux). Is there any timeline of when the fix will be released for RHEL 9? This should be fixed in kernel-5.14.0-284.17.1.el9_2 Thanks, do you know when kernel-5.14.0-284.17.1.el9_2 will be released? Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug. This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there. Due to differences in account names between systems, some fields were not replicated. Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information. To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer. You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like: "Bugzilla Bug" = 1234567 In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information. |
Created attachment 1952401 [details] Its a screenshot of the process which is running wild Description of problem: My server was running on CentOS9 and everything worked perfectly. Until one day the system told me there is an operating system update. I updated and restartet the PC and after running on the new update the pc randomly freezes and i can't use it anymore and have to plug the cable because it wouldn't turn off by itself. And if i look which process is causing trouble its kworker/1:0cifsiod My theory is its a cifs problem because in var/log/messages it shows: Mar 20 11:26:00 ersatzwinserver kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 805s! [kworker/1:1:149] Workqueue: cifsiod smb2_reconnect_server [cifs] Mar 20 11:26:00 ersatzwinserver kernel: RIP: 0010:native_queued_spin_lock_slowpath.part.0+0x42/0x1a0 Mar 20 11:26:00 ersatzwinserver kernel: Code: 0f ba 2a 08 0f 92 c1 8b 02 0f b6 c9 c1 e1 08 30 e4 09 c8 a9 00 01 ff ff 0f 85 f5 00 00 00 85 c0 74 0e 8b 02 84 c0 74 08 f3 90 <8b> 02 84 c0 75 f8 b8 01 00 00 00 66 89 02 c3 cc cc cc cc 8b 37 b8 Mar 20 11:26:00 ersatzwinserver kernel: RSP: 0018:ffffa91e40667d78 EFLAGS: 00000202 Mar 20 11:26:00 ersatzwinserver kernel: RAX: 0000000000000101 RBX: ffff9907437c2800 RCX: 0000000000000000 Mar 20 11:26:00 ersatzwinserver kernel: RDX: ffff9907905aa028 RSI: 0000000000000000 RDI: ffff9907905aa028 Mar 20 11:26:00 ersatzwinserver kernel: RBP: ffff9907905aa028 R08: ffffa91e40667e38 R09: 0000000000000000 Mar 20 11:26:00 ersatzwinserver kernel: R10: ffffa91e40667ce8 R11: 0000000000000018 R12: 0000000000000000 Mar 20 11:26:00 ersatzwinserver kernel: R13: ffff9907905aa000 R14: 000000000000ffff R15: 000000000000ffff Mar 20 11:26:00 ersatzwinserver kernel: FS: 0000000000000000(0000) GS:ffff9916be340000(0000) knlGS:0000000000000000 Mar 20 11:26:00 ersatzwinserver kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Mar 20 11:26:00 ersatzwinserver kernel: CR2: 000055a5b88c2000 CR3: 0000000fe4410004 CR4: 00000000003706e0 Mar 20 11:26:00 ersatzwinserver kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Mar 20 11:26:00 ersatzwinserver kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400