Bug 1619508

Summary: usb error and kernel soft lockup
Product: [Fedora] Fedora Reporter: Chunyu Hu <chuhu>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 29CC: airlied, bskeggs, chuhu, ewk, hdegoede, ichavero, itamar, jarodwilson, jglisse, john.j5live, jonathan, josef, kernel-maint, labbott, linville, mchehab, mjg59, nobody+liwang, pifang, steved, sztsian
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-09 19:52:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
usb error and soft lockup console output none

Description Chunyu Hu 2018-08-21 05:41:18 UTC
Description of problem:

I tried to test 4.18.1-300.fc29.x86_64, and found kernel soft lockup happened after booting, and it happened after the USB error message. And looks like it's dead looping in chrdev_show. 

sgi-uv2-01 login: [-- MARK -- Mon Aug 20 11:45:00 2018]
[   50.210051] usb 1-1.3: device descriptor read/64, error -110
[   50.319222] usb 1-1-port3: attempt power cycle
[   50.899065] usb 1-1.3: new high-speed USB device number 5 using ehci-pci
[   56.290209] usb 1-1.3: device descriptor read/8, error -110
[   61.922090] usb 1-1.3: device descriptor read/8, error -110
[   62.095039] usb 1-1.3: new high-speed USB device number 6 using ehci-pci
[   67.554211] usb 1-1.3: device descriptor read/8, error -110
[   72.301000] watchdog: BUG: soft lockup - CPU#15 stuck for 23s! [lvm:1903]
[   72.308592] Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl sunrpc x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm vfat irqbypass crct10dif_pclmul fat crc32_pclmul ghash_clmulni_intel intel_cstate intel_uncore mgag200 ttm intel_rapl_perf drm_kms_helper iTCO_wdt drm iTCO_vendor_support isci ipmi_si(+) libsas ipmi_devintf ipmi_msghandler scsi_transport_sas i2c_i801 lpc_ich ioatdma pcc_cpufreq binfmt_misc xfs libcrc32c crc32c_intel igb i2c_algo_bit dca
[   72.359484] CPU: 15 PID: 1903 Comm: lvm Kdump: loaded Not tainted 4.18.1-300.fc29.x86_64 #1
[   72.368806] Hardware name: SGI UV2000/ROMLEY, BIOS SGI UV 2000/3000 series BIOS 01/15/2013
[   72.378071] RIP: 0010:chrdev_show+0x5b/0x90
[   72.382737] Code: e2 08 29 c2 29 d6 48 63 c6 48 8b 1c c5 00 df a5 90 48 85 db 74 33 49 c7 c5 75 28 0e 90 eb 08 48 8b 1b 48 85 db 74 22 8b 43 08 <48> 89 c2 48 39 e8 75 ed 48 8d 4b 14 4c 89 ee 4c 89 e7 e8 be 1e 02
[   72.403813] RSP: 0018:ffffb15f0ad37d90 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
[   72.412263] RAX: 000000005d661000 RBX: ffff8bf15d665000 RCX: 0000000000000001
[   72.420228] RDX: 000000005d658000 RSI: 0000000000000000 RDI: ffff8be15c09d0eb
[   72.428191] RBP: 00000000000000e2 R08: 0000000000001000 R09: 00000000a4143f8c
[   72.436154] R10: ffff8be15c09e000 R11: ffff8be15c09d0e6 R12: ffff8be15b4dc200
[   72.444119] R13: ffffffff900e2875 R14: ffffb15f0ad37df8 R15: ffff8be15b4dc200
[   72.452085] FS:  00007f2d74cbf6c0(0000) GS:ffff8be15fbc0000(0000) knlGS:0000000000000000
[   72.461115] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   72.467529] CR2: 000056528a5d6000 CR3: 000000179cfcc002 CR4: 00000000000606e0
[   72.475495] Call Trace:
[   72.478263]  devinfo_show+0x24/0x60
[   72.482173]  seq_read+0x360/0x460
[   72.485892]  proc_reg_read+0x3c/0x60
[   72.489887]  __vfs_read+0x36/0x180
[   72.493684]  vfs_read+0x8a/0x140
[   72.497286]  ksys_read+0x4f/0xb0
[   72.500911]  do_syscall_64+0x5b/0x160
[   72.505020]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   72.510666] RIP: 0033:0x7f2d7381b091
[   72.514652] Code: fe ff ff 50 48 8d 3d b6 b6 09 00 e8 59 05 02 00 66 0f 1f 84 00 00 00 00 00 48 8d 05 51 39 2d 00 8b 00 85 c0 75 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 41 54 49 89 d4 55 48
[   72.535736] RSP: 002b:00007fffdfb3ef78 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[   72.544185] RAX: ffffffffffffffda RBX: 000056528a5c1230 RCX: 00007f2d7381b091
[   72.552150] RDX: 0000000000000400 RSI: 000056528a5cdb90 RDI: 0000000000000003
[   72.560115] RBP: 0000000000000d68 R08: 0000000000000001 R09: 0000000000000000
[   72.568078] R10: 00007f2d74cbf6c0 R11: 0000000000000246 R12: 00007f2d73ae5760
[   72.576042] R13: 00007f2d73ae62a0 R14: 000000000000004f R15: 000056528a5c1230
[   73.186215] usb 1-1.3: device descriptor read/8, error -110
[   73.295098] usb 1-1-port3: unable to enumerate USB device
[  100.300981] watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [lvm:1903]
[  100.308562] Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache intel_rapl sunrpc x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm vfat irqbypass crct10dif_pclmul fat crc32_pclmul ghash_clmulni_intel intel_cstate intel_uncore mgag200 ttm intel_rapl_perf drm_kms_helper iTCO_wdt drm iTCO_vendor_support isci ipmi_si(+) libsas ipmi_devintf ipmi_msghandler scsi_transport_sas i2c_i801 lpc_ich ioatdma pcc_cpufreq binfmt_misc xfs libcrc32c crc32c_intel igb i2c_algo_bit dca
[  100.359414] CPU: 15 PID: 1903 Comm: lvm Kdump: loaded Tainted: G             L    4.18.1-300.fc29.x86_64 #1
[  100.370287] Hardware name: SGI UV2000/ROMLEY, BIOS SGI UV 2000/3000 series BIOS 01/15/2013
[  100.379515] RIP: 0010:chrdev_show+0x5b/0x90
[  100.384174] Code: e2 08 29 c2 29 d6 48 63 c6 48 8b 1c c5 00 df a5 90 48 85 db 74 33 49 c7 c5 75 28 0e 90 eb 08 48 8b 1b 48 85 db 74 22 8b 43 08 <48> 89 c2 48 39 e8 75 ed 48 8d 4b 14 4c 89 ee 4c 89 e7 e8 be 1e 02
[  100.405258] RSP: 0018:ffffb15f0ad37d90 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
[  100.413707] RAX: 000000005d67d000 RBX: ffff8bf15d678000 RCX: 0000000000000001
[  100.421672] RDX: 000000005d679000 RSI: 0000000000000000 RDI: ffff8be15c09d0eb
[  100.429639] RBP: 00000000000000e2 R08: 0000000000001000 R09: 00000000a4143f8c
[  100.437602] R10: ffff8be15c09e000 R11: ffff8be15c09d0e6 R12: ffff8be15b4dc200
[  100.445568] R13: ffffffff900e2875 R14: ffffb15f0ad37df8 R15: ffff8be15b4dc200
[  100.453533] FS:  00007f2d74cbf6c0(0000) GS:ffff8be15fbc0000(0000) knlGS:0000000000000000
[  100.462564] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  100.468978] CR2: 000056528a5d6000 CR3: 000000179cfcc002 CR4: 00000000000606e0
[  100.476942] Call Trace:
[  100.479675]  devinfo_show+0x24/0x60
[  100.483567]  seq_read+0x360/0x460



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

crash> dis -l chrdev_show -x
/usr/src/debug/kernel-4.18.fc29/linux-4.18.1-300.fc29.x86_64/fs/char_dev.c: 52
/usr/src/debug/kernel-4.18.fc29/linux-4.18.1-300.fc29.x86_64/fs/char_dev.c: 52
0xffffffff812aabe0 <chrdev_show>:       nopl   0x0(%rax,%rax,1) [FTRACE NOP]
/usr/src/debug/kernel-4.18.fc29/linux-4.18.1-300.fc29.x86_64/fs/char_dev.c: 55
0xffffffff812aabe5 <chrdev_show+0x5>:   push   %r13
0xffffffff812aabe7 <chrdev_show+0x7>:   push   %r12
0xffffffff812aabe9 <chrdev_show+0x9>:   mov    %rdi,%r12
0xffffffff812aabec <chrdev_show+0xc>:   mov    $0xffffffff822976a0,%rdi
0xffffffff812aabf3 <chrdev_show+0x13>:  push   %rbp
0xffffffff812aabf4 <chrdev_show+0x14>:  mov    %rsi,%rbp
0xffffffff812aabf7 <chrdev_show+0x17>:  push   %rbx
0xffffffff812aabf8 <chrdev_show+0x18>:  callq  0xffffffff8190bf40 <mutex_lock>
/usr/src/debug/kernel-4.18.fc29/linux-4.18.1-300.fc29.x86_64/fs/char_dev.c: 56
0xffffffff812aabfd <chrdev_show+0x1d>:  mov    $0x80808081,%edx
0xffffffff812aac02 <chrdev_show+0x22>:  mov    %ebp,%eax
0xffffffff812aac04 <chrdev_show+0x24>:  mov    %ebp,%esi
0xffffffff812aac06 <chrdev_show+0x26>:  imul   %rdx,%rax
0xffffffff812aac0a <chrdev_show+0x2a>:  shr    $0x27,%rax
0xffffffff812aac0e <chrdev_show+0x2e>:  mov    %eax,%edx
0xffffffff812aac10 <chrdev_show+0x30>:  shl    $0x8,%edx
0xffffffff812aac13 <chrdev_show+0x33>:  sub    %eax,%edx
0xffffffff812aac15 <chrdev_show+0x35>:  sub    %edx,%esi
0xffffffff812aac17 <chrdev_show+0x37>:  movslq %esi,%rax
0xffffffff812aac1a <chrdev_show+0x3a>:  mov    -0x7d5a2100(,%rax,8),%rbx
0xffffffff812aac22 <chrdev_show+0x42>:  test   %rbx,%rbx
0xffffffff812aac25 <chrdev_show+0x45>:  je     0xffffffff812aac5a <chrdev_show+0x7a>
0xffffffff812aac27 <chrdev_show+0x47>:  mov    $0xffffffff820e2875,%r13
0xffffffff812aac2e <chrdev_show+0x4e>:  jmp    0xffffffff812aac38 <chrdev_show+0x58>
0xffffffff812aac30 <chrdev_show+0x50>:  mov    (%rbx),%rbx
0xffffffff812aac33 <chrdev_show+0x53>:  test   %rbx,%rbx
0xffffffff812aac36 <chrdev_show+0x56>:  je     0xffffffff812aac5a <chrdev_show+0x7a>
/usr/src/debug/kernel-4.18.fc29/linux-4.18.1-300.fc29.x86_64/fs/char_dev.c: 57
0xffffffff812aac38 <chrdev_show+0x58>:  mov    0x8(%rbx),%eax
0xffffffff812aac3b <chrdev_show+0x5b>:  mov    %rax,%rdx
0xffffffff812aac3e <chrdev_show+0x5e>:  cmp    %rbp,%rax
0xffffffff812aac41 <chrdev_show+0x61>:  jne    0xffffffff812aac30 <chrdev_show+0x50>
/usr/src/debug/kernel-4.18.fc29/linux-4.18.1-300.fc29.x86_64/fs/char_dev.c: 58
0xffffffff812aac43 <chrdev_show+0x63>:  lea    0x14(%rbx),%rcx
0xffffffff812aac47 <chrdev_show+0x67>:  mov    %r13,%rsi
0xffffffff812aac4a <chrdev_show+0x6a>:  mov    %r12,%rdi
0xffffffff812aac4d <chrdev_show+0x6d>:  callq  0xffffffff812ccb10 <seq_printf>
0xffffffff812aac52 <chrdev_show+0x72>:  mov    (%rbx),%rbx
0xffffffff812aac55 <chrdev_show+0x75>:  test   %rbx,%rbx
0xffffffff812aac58 <chrdev_show+0x78>:  jne    0xffffffff812aac38 <chrdev_show+0x58>
/usr/src/debug/kernel-4.18.fc29/linux-4.18.1-300.fc29.x86_64/fs/char_dev.c: 60
0xffffffff812aac5a <chrdev_show+0x7a>:  pop    %rbx
0xffffffff812aac5b <chrdev_show+0x7b>:  mov    $0xffffffff822976a0,%rdi
0xffffffff812aac62 <chrdev_show+0x82>:  pop    %rbp
0xffffffff812aac63 <chrdev_show+0x83>:  pop    %r12
0xffffffff812aac65 <chrdev_show+0x85>:  pop    %r13
0xffffffff812aac67 <chrdev_show+0x87>:  jmpq   0xffffffff8190b9a0 <mutex_unlock>




 51 void chrdev_show(struct seq_file *f, off_t offset)
 52 {
 53     struct char_device_struct *cd;
 54 
 55     mutex_lock(&chrdevs_lock);
 56     for (cd = chrdevs[major_to_index(offset)]; cd; cd = cd->next) {
 57         if (cd->major == offset)
 58             seq_printf(f, "%3d %s\n", cd->major, cd->name);
 59     }
 60     mutex_unlock(&chrdevs_lock);
 61 }

Comment 2 Chunyu Hu 2018-08-21 05:49:29 UTC
Created attachment 1477411 [details]
usb error and soft lockup console output

Comment 3 Jeremy Cline 2018-12-03 17:33:51 UTC
We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 29 kernel bugs.
 
Fedora 29 has now been rebased to 4.19.5-300.fc29.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.
 
If you experience different issues, please open a new bug report for those.

Comment 4 Laura Abbott 2019-04-09 19:52:07 UTC
Since there wasn't an update on the last needinfo, I'm going to close this bug. Please re-test on a newer kernel and reopen if the problem still exists.