Bug 2284279

Summary: RIP: 0010:gss_free_in_token_pages+0x2b/0xc0 [auth_rpcgss]
Product: [Fedora] Fedora Reporter: Anthony Messina <amessina>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 40CC: acaringi, adscvr, airlied, alciregi, bojan, bskeggs, dominik, hdegoede, hpa, josef, kernel-maint, linville, masami256, mchehab, ptalbert, robatino, steved, suraj.ghimire7
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
URL: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/commit/net/sunrpc?h=queue/6.9&id=2e851b6d4a1e23e92f7f3d02f2d1b31cbce5f37b
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-06-13 12:58:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Anthony Messina 2024-06-02 15:36:42 UTC
With kernel-6.8.12-300.fc40.x86_64, the NFS server fails:

[Sun Jun  2 10:27:07 2024] general protection fault, probably for non-canonical address 0x185e37a322f016bd: 0000 [#1] PREEMPT SMP PTI
[Sun Jun  2 10:27:07 2024] CPU: 7 PID: 1523 Comm: nfsd Not tainted 6.8.12-300.fc40.x86_64 #1
[Sun Jun  2 10:27:07 2024] Hardware name: Dell Inc. PowerEdge R230/0FRVY0, BIOS 2.13.0 01/18/2022
[Sun Jun  2 10:27:07 2024] RIP: 0010:gss_free_in_token_pages+0x2b/0xc0 [auth_rpcgss]
[Sun Jun  2 10:27:07 2024] Code: 1f 44 00 00 41 54 49 89 fc 55 53 48 8b 3f 48 8b 1f 48 85 db 74 41 31 ed eb 10 49 8b 3c 24 48 63 c5 48 8b 1c c7 48 85 db 74 2d <48> 8b 43 08 83 c5 01 a8 01 75 79 66 90 66 90 f0 ff 4b 34 75 db 48
[Sun Jun  2 10:27:07 2024] RSP: 0018:ffffa3ac40cebca0 EFLAGS: 00010202
[Sun Jun  2 10:27:07 2024] RAX: 0000000000000001 RBX: 185e37a322f016b5 RCX: 000000000000020e
[Sun Jun  2 10:27:07 2024] RDX: 0000000000000000 RSI: 000000000000182b RDI: ffff902c25ceb728
[Sun Jun  2 10:27:07 2024] RBP: 0000000000000001 R08: 00000000000007e0 R09: 0000000000000000
[Sun Jun  2 10:27:07 2024] R10: 06c6646c1514941a R11: 67c5d40effe777af R12: ffffa3ac40cebd00
[Sun Jun  2 10:27:07 2024] R13: 000000000000000a R14: ffff902c25ceb728 R15: 00000000000003c7
[Sun Jun  2 10:27:07 2024] FS:  0000000000000000(0000) GS:ffff90330fd80000(0000) knlGS:0000000000000000
[Sun Jun  2 10:27:07 2024] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Sun Jun  2 10:27:07 2024] CR2: 00007f466b58fd18 CR3: 000000012d678001 CR4: 00000000003706f0
[Sun Jun  2 10:27:07 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Sun Jun  2 10:27:07 2024] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[Sun Jun  2 10:27:07 2024] Call Trace:
[Sun Jun  2 10:27:07 2024]  <TASK>
[Sun Jun  2 10:27:07 2024]  ? die_addr+0x36/0x90
[Sun Jun  2 10:27:07 2024]  ? exc_general_protection+0x17c/0x450
[Sun Jun  2 10:27:07 2024]  ? asm_exc_general_protection+0x26/0x30
[Sun Jun  2 10:27:07 2024]  ? gss_free_in_token_pages+0x2b/0xc0 [auth_rpcgss]
[Sun Jun  2 10:27:07 2024]  svcauth_gss_proxy_init+0x3f6/0x4c0 [auth_rpcgss]
[Sun Jun  2 10:27:07 2024]  svcauth_gss_proc_init+0xc5/0x170 [auth_rpcgss]
[Sun Jun  2 10:27:07 2024]  svc_process_common+0x225/0x6d0 [sunrpc]
[Sun Jun  2 10:27:07 2024]  svc_process+0x131/0x180 [sunrpc]
[Sun Jun  2 10:27:07 2024]  svc_recv+0x857/0xa50 [sunrpc]
[Sun Jun  2 10:27:07 2024]  ? __pfx_nfsd+0x10/0x10 [nfsd]
[Sun Jun  2 10:27:07 2024]  nfsd+0x84/0xf0 [nfsd]
[Sun Jun  2 10:27:07 2024]  kthread+0xe5/0x120
[Sun Jun  2 10:27:07 2024]  ? __pfx_kthread+0x10/0x10
[Sun Jun  2 10:27:07 2024]  ret_from_fork+0x31/0x50
[Sun Jun  2 10:27:07 2024]  ? __pfx_kthread+0x10/0x10
[Sun Jun  2 10:27:07 2024]  ret_from_fork_asm+0x1b/0x30
[Sun Jun  2 10:27:07 2024]  </TASK>
[Sun Jun  2 10:27:07 2024] Modules linked in: rpcsec_gss_krb5 tls macvlan cfg80211 nf_tables vfat fat intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_pmc_core_pltdrv intel_pmc_core intel_vsec pmt_telemetry pmt_class intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp ipmi_ssif pktcdvd dell_wmi dell_smbios dell_wmi_descriptor ledtrig_audio sparse_keymap rfkill jc42 coretemp rapl tg3 dcdbas ee1004 intel_cstate mei_me intel_pch_thermal intel_uncore intel_wmi_thunderbolt mei i2c_i801 ie31200_edac i2c_smbus ipmi_si ipmi_devintf acpi_power_meter ipmi_msghandler nfsd nfs_acl lockd grace auth_rpcgss fuse loop sunrpc tun dm_multipath nfnetlink zram xfs crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 sha256_ssse3 megaraid_sas mgag200 wdat_wdt i2c_algo_bit sha1_ssse3 video wmi scsi_dh_rdac scsi_dh_emc scsi_dh_alua
[Sun Jun  2 10:27:07 2024] ---[ end trace 0000000000000000 ]---
[Sun Jun  2 10:27:07 2024] RIP: 0010:gss_free_in_token_pages+0x2b/0xc0 [auth_rpcgss]
[Sun Jun  2 10:27:07 2024] Code: 1f 44 00 00 41 54 49 89 fc 55 53 48 8b 3f 48 8b 1f 48 85 db 74 41 31 ed eb 10 49 8b 3c 24 48 63 c5 48 8b 1c c7 48 85 db 74 2d <48> 8b 43 08 83 c5 01 a8 01 75 79 66 90 66 90 f0 ff 4b 34 75 db 48
[Sun Jun  2 10:27:07 2024] RSP: 0018:ffffa3ac40cebca0 EFLAGS: 00010202
[Sun Jun  2 10:27:07 2024] RAX: 0000000000000001 RBX: 185e37a322f016b5 RCX: 000000000000020e
[Sun Jun  2 10:27:07 2024] RDX: 0000000000000000 RSI: 000000000000182b RDI: ffff902c25ceb728
[Sun Jun  2 10:27:07 2024] RBP: 0000000000000001 R08: 00000000000007e0 R09: 0000000000000000
[Sun Jun  2 10:27:07 2024] R10: 06c6646c1514941a R11: 67c5d40effe777af R12: ffffa3ac40cebd00
[Sun Jun  2 10:27:07 2024] R13: 000000000000000a R14: ffff902c25ceb728 R15: 00000000000003c7
[Sun Jun  2 10:27:07 2024] FS:  0000000000000000(0000) GS:ffff90330fd80000(0000) knlGS:0000000000000000
[Sun Jun  2 10:27:07 2024] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Sun Jun  2 10:27:07 2024] CR2: 00007f466b58fd18 CR3: 000000012d678001 CR4: 00000000003706f0
[Sun Jun  2 10:27:07 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Sun Jun  2 10:27:07 2024] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[Sun Jun  2 10:27:30 2024] systemd-journald[569]: /var/log/journal/e71248e5a94b42069b4603dde95c99b4/user-1136600007.journal: Journal file uses a different sequence number ID, rotating.
[Sun Jun  2 10:27:32 2024] general protection fault, probably for non-canonical address 0xf096996ced5ed2b5: 0000 [#2] PREEMPT SMP PTI
[Sun Jun  2 10:27:32 2024] CPU: 7 PID: 1524 Comm: nfsd Tainted: G      D            6.8.12-300.fc40.x86_64 #1
[Sun Jun  2 10:27:32 2024] Hardware name: Dell Inc. PowerEdge R230/0FRVY0, BIOS 2.13.0 01/18/2022
[Sun Jun  2 10:27:32 2024] RIP: 0010:gss_free_in_token_pages+0x2b/0xc0 [auth_rpcgss]
[Sun Jun  2 10:27:32 2024] Code: 1f 44 00 00 41 54 49 89 fc 55 53 48 8b 3f 48 8b 1f 48 85 db 74 41 31 ed eb 10 49 8b 3c 24 48 63 c5 48 8b 1c c7 48 85 db 74 2d <48> 8b 43 08 83 c5 01 a8 01 75 79 66 90 66 90 f0 ff 4b 34 75 db 48
[Sun Jun  2 10:27:32 2024] RSP: 0018:ffffa3ac40cf3ca0 EFLAGS: 00010282
[Sun Jun  2 10:27:32 2024] RAX: 0000000000000001 RBX: f096996ced5ed2ad RCX: 0000000000000e77
[Sun Jun  2 10:27:32 2024] RDX: 0000000000000000 RSI: 000000000001d247 RDI: ffff902bed607fd0
[Sun Jun  2 10:27:32 2024] RBP: 0000000000000001 R08: 00000000000007e0 R09: 0000000000000000
[Sun Jun  2 10:27:32 2024] R10: fef6d628a9960e07 R11: 4de845649be40c76 R12: ffffa3ac40cf3d00
[Sun Jun  2 10:27:32 2024] R13: 000000000000000a R14: ffff902bed607fd0 R15: 00000000000003c7
[Sun Jun  2 10:27:32 2024] FS:  0000000000000000(0000) GS:ffff90330fd80000(0000) knlGS:0000000000000000
[Sun Jun  2 10:27:32 2024] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Sun Jun  2 10:27:32 2024] CR2: 00005633d2103a10 CR3: 000000060c428006 CR4: 00000000003706f0
[Sun Jun  2 10:27:32 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Sun Jun  2 10:27:32 2024] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[Sun Jun  2 10:27:32 2024] Call Trace:
[Sun Jun  2 10:27:32 2024]  <TASK>
[Sun Jun  2 10:27:32 2024]  ? die_addr+0x36/0x90
[Sun Jun  2 10:27:32 2024]  ? exc_general_protection+0x17c/0x450
[Sun Jun  2 10:27:32 2024]  ? asm_exc_general_protection+0x26/0x30
[Sun Jun  2 10:27:32 2024]  ? gss_free_in_token_pages+0x2b/0xc0 [auth_rpcgss]
[Sun Jun  2 10:27:32 2024]  svcauth_gss_proxy_init+0x3f6/0x4c0 [auth_rpcgss]
[Sun Jun  2 10:27:32 2024]  svcauth_gss_proc_init+0xc5/0x170 [auth_rpcgss]
[Sun Jun  2 10:27:32 2024]  svc_process_common+0x225/0x6d0 [sunrpc]
[Sun Jun  2 10:27:32 2024]  svc_process+0x131/0x180 [sunrpc]
[Sun Jun  2 10:27:32 2024]  svc_recv+0x857/0xa50 [sunrpc]
[Sun Jun  2 10:27:32 2024]  ? __pfx_nfsd+0x10/0x10 [nfsd]
[Sun Jun  2 10:27:32 2024]  nfsd+0x84/0xf0 [nfsd]
[Sun Jun  2 10:27:32 2024]  kthread+0xe5/0x120
[Sun Jun  2 10:27:32 2024]  ? __pfx_kthread+0x10/0x10
[Sun Jun  2 10:27:32 2024]  ret_from_fork+0x31/0x50
[Sun Jun  2 10:27:32 2024]  ? __pfx_kthread+0x10/0x10
[Sun Jun  2 10:27:32 2024]  ret_from_fork_asm+0x1b/0x30
[Sun Jun  2 10:27:32 2024]  </TASK>
[Sun Jun  2 10:27:32 2024] Modules linked in: nf_log_syslog nft_log nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set rpcsec_gss_krb5 tls macvlan cfg80211 nf_tables vfat fat intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_pmc_core_pltdrv intel_pmc_core intel_vsec pmt_telemetry pmt_class intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp ipmi_ssif pktcdvd dell_wmi dell_smbios dell_wmi_descriptor ledtrig_audio sparse_keymap rfkill jc42 coretemp rapl tg3 dcdbas ee1004 intel_cstate mei_me intel_pch_thermal intel_uncore intel_wmi_thunderbolt mei i2c_i801 ie31200_edac i2c_smbus ipmi_si ipmi_devintf acpi_power_meter ipmi_msghandler nfsd nfs_acl lockd grace auth_rpcgss fuse loop sunrpc tun dm_multipath nfnetlink zram xfs crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 sha256_ssse3 megaraid_sas mgag200
[Sun Jun  2 10:27:32 2024]  wdat_wdt i2c_algo_bit sha1_ssse3 video wmi scsi_dh_rdac scsi_dh_emc scsi_dh_alua
[Sun Jun  2 10:27:32 2024] ---[ end trace 0000000000000000 ]---
[Sun Jun  2 10:27:32 2024] RIP: 0010:gss_free_in_token_pages+0x2b/0xc0 [auth_rpcgss]
[Sun Jun  2 10:27:32 2024] Code: 1f 44 00 00 41 54 49 89 fc 55 53 48 8b 3f 48 8b 1f 48 85 db 74 41 31 ed eb 10 49 8b 3c 24 48 63 c5 48 8b 1c c7 48 85 db 74 2d <48> 8b 43 08 83 c5 01 a8 01 75 79 66 90 66 90 f0 ff 4b 34 75 db 48
[Sun Jun  2 10:27:32 2024] RSP: 0018:ffffa3ac40cebca0 EFLAGS: 00010202
[Sun Jun  2 10:27:32 2024] RAX: 0000000000000001 RBX: 185e37a322f016b5 RCX: 000000000000020e
[Sun Jun  2 10:27:32 2024] RDX: 0000000000000000 RSI: 000000000000182b RDI: ffff902c25ceb728
[Sun Jun  2 10:27:32 2024] RBP: 0000000000000001 R08: 00000000000007e0 R09: 0000000000000000
[Sun Jun  2 10:27:32 2024] R10: 06c6646c1514941a R11: 67c5d40effe777af R12: ffffa3ac40cebd00
[Sun Jun  2 10:27:32 2024] R13: 000000000000000a R14: ffff902c25ceb728 R15: 00000000000003c7
[Sun Jun  2 10:27:32 2024] FS:  0000000000000000(0000) GS:ffff90330fd80000(0000) knlGS:0000000000000000
[Sun Jun  2 10:27:32 2024] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Sun Jun  2 10:27:32 2024] CR2: 00005633d2103a10 CR3: 000000060c428006 CR4: 00000000003706f0
[Sun Jun  2 10:27:32 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Sun Jun  2 10:27:32 2024] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

Reproducible: Always

Comment 1 Dominik 'Rathann' Mierzejewski 2024-06-06 23:10:38 UTC
@amessina could you share more details about your hardware?
I'm getting a similar general protection fault trace, triggered reliably by running systemctl start sysstat-collect.service .

Comment 2 Anthony Messina 2024-06-07 00:44:48 UTC
(In reply to Dominik 'Rathann' Mierzejewski from comment #1)
> @amessina could you share more details about your hardware?
> I'm getting a similar general protection fault trace, triggered reliably by
> running systemctl start sysstat-collect.service .

My hardware is listed in the trace above:

Hardware name: Dell Inc. PowerEdge R230/0FRVY0, BIOS 2.13.0 01/18/2022

Running x86_64.

For others concerned, this does not happen with kernel-6.8.11

Comment 3 Dominik 'Rathann' Mierzejewski 2024-06-07 16:04:35 UTC
(In reply to Anthony Messina from comment #2)
> (In reply to Dominik 'Rathann' Mierzejewski from comment #1)
> > @amessina could you share more details about your hardware?
> > I'm getting a similar general protection fault trace, triggered reliably by
> > running systemctl start sysstat-collect.service .
> 
> My hardware is listed in the trace above:
> 
> Hardware name: Dell Inc. PowerEdge R230/0FRVY0, BIOS 2.13.0 01/18/2022
> 
> Running x86_64.
> 
> For others concerned, this does not happen with kernel-6.8.11

Thanks. The one I'm getting happens on 6.8.10 and 6.8.11.
6.9.3 also doesn't have it.

I opened bug 2290881 for the record.

Comment 4 Anthony Messina 2024-06-13 12:58:01 UTC
Resolved in https://bodhi.fedoraproject.org/updates/FEDORA-2024-8c4744962d