Bug 2210435

Summary:	ath10k_pci firmware crashed!
Product:	[Fedora] Fedora	Reporter:	Garry T. Williams <gtwilliams>
Component:	kernel	Assignee:	Kernel Maintainer List <kernel-maint>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	38	CC:	acaringi, adscvr, airlied, alciregi, bskeggs, hdegoede, hpa, jarodwilson, josef, kernel-maint, lgoncalv, linville, ludovico.cavedon, masami256, mchehab, ptalbert, steved
Target Milestone:	---	Keywords:	Regression, Reopened
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-07-08 22:39:50 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Garry T. Williams 2023-05-27 14:53:53 UTC

kernel 6.2.15-300.fc38.x86_64 and 6.3.3-200.fc38.x86_64 introduced this error:

May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: firmware crashed! (guid 334f45e1-6fbc-4594-a04d-5c6f25f0ba2c)
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: qca6174 hw3.2 target 0x05030000 chip_id 0x00340aff sub 1a56:143a
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 0 testmode 0
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: firmware ver WLAN.RM.4.4.1-00288- api 6 features wowlan,ignore-otp,mfp crc32 bf907c7c
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: board_file api 2 bmi_id N/A crc32 d2863f91
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: htt-ver 3.87 wmi-op 4 htt-op 3 cal otp max-sta 32 raw 0 hwcrypto 1
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: failed to get memcpy hi address for firmware address 4: -16
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: failed to read firmware dump area: -16
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: Copy Engine register dump:
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: [00]: 0x00034400  12  12   3   3
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: [01]: 0x00034800   3   2 175 176
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: [02]: 0x00034c00  20  19  18  19
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: [03]: 0x00035000  18  17  18  16
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: [04]: 0x00035400 3531 3529  37 229
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: [05]: 0x00035800   0   0  64   0
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: [06]: 0x00035c00   8   6   8   6
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: [07]: 0x00036000   1   0   1   0
May 25 19:20:03 kernel: ath10k_pci 0000:02:00.0: failed to read hi_board_data address: -28
May 25 19:20:03 kernel: ieee80211 phy0: Hardware restart was requested
May 25 19:20:04 kernel: ath10k_pci 0000:02:00.0: timed out waiting peer stats info
May 25 19:20:04 kernel: ath10k_pci 0000:02:00.0: could not request stats (-108)
May 25 19:20:04 kernel: ath10k_pci 0000:02:00.0: device successfully recovered

which kills the wifi connection.  When I disconnect and connect, the connection is recovered, but the error eventually happens again repeatedly.

I booted kernel 6.2.14-300.fc38.x86_64 and the error will not recur.

Reproducible: Always

Steps to Reproduce:
1. Boot recent kernel
2.
3.
Actual Results:  
Above error happens after some few hours and recurs after resetting wifi connection.

Expected Results:  
Stuff works

Comment 1 Garry T. Williams 2023-06-01 20:45:13 UTC

Just tested kernel 6.3.5-200.fc38.x86_64 and the bug persists.  Last good kernel is 6.2.14-300.fc38.x86_64.

Comment 2 Garry T. Williams 2023-06-12 03:01:23 UTC

kernel 6.3.7-200.fc38.x86_64 fixes this error.

Comment 3 Garry T. Williams 2023-06-13 20:10:37 UTC

Oops.  Bug is still in this most recent kernel (6.3.5-200.fc38).  I just didn't wait long enough -- it took two days to finally trigger again.  Now it triggers repeatedly every few hours again.  Last known good kernel is still 6.2.14-300.fc38.

Comment 4 Garry T. Williams 2023-06-20 03:22:53 UTC

If I build 6.3.3 from kernel sources, there is no bug.  Once I apply patch-6.3-redhat.patch, the bug appears.  I have closed this bug upstream (https://bugzilla.kernel.org/show_bug.cgi?id=217549) since it is a Fedora patch that introduced it.  I do not know any more specifics other than I could not reproduce this error when I built 6.2.15 from the kernel sources.  So I believe that the patch in that kernel from Fedora introduced this error.

Comment 5 Garry T. Williams 2023-07-08 22:39:50 UTC

It looks like 6.3.11 fixed this.  It may be that 6.3.10 did, but for sure, I cannot reproduce with 6.3.11.