Created attachment 774819 [details] Backtrace Description of problem: Since upgrading to libusbx-1.0.16-1.fc20.x86_64, upowerd occasionally hangs (leading for instance to most KDE applications suffering a 25sec startup delay, until the dbus call to upowerd times out). Retrieving a backtrace from upowerd shows that the culprit appears to be libusbx. Backtrace is attached. Version-Release number of selected component (if applicable): libusbx-1.0.16-1.fc20.x86_64 How reproducible: Occasionally Steps to Reproduce: 1. Difficult, appears more or less daily, possibly related to events such as suspend/resume.
Hi, Thanks for the bug report! I'm going on vacation for a week starting tomorrow so I don't have time to look into this myself atm. I've forwarded this bug to the upstream libusbx-devel mailinglist. If upstream does not figure things out while I'm away I'll look into this myself when I'm back. Regards, Hans
Ok, thanks! Have a nice vacation.
Created attachment 775695 [details] Patch fixing the deadlock Hi, So sending this bugreport upstream helped, someone pointed out the cause, and this morning I had some inspiration how to fix this. This patch I'm attaching is the result of this, and should fix your issue. I wanted to do a new build for you with the fix in, but I cannot due to buildsys maintenance: https://fedorahosted.org/fedora-infrastructure/ticket/3882 So if you've some experience in building packages yourself it would be great if you could build it yourself, the changes are in pkgs.fedoraproject.org libusbx module master branch, so you just need to do: As root: yum install fedpkg As user: fedpkg clone --anonymous libusbx cd libusbx fedpkg local And then you should get an x86_64 dir under the libusbx dir with new packages. And now I'm really really really leaving for vacation (we depart in about an hour), see you in a week! Regards, Hans
Hi Hans, Thanks a lot! I'm familiar with moch & fedpkg & co., I'll test this right away and then report back in a few days. Enjoy!
Looks good, I haven't encountered the issue again since applying the patch. Thanks again!
Created attachment 778202 [details] backtrace Unfortunately there is now a new deadlock, I've attached a new backtrace. (Maybe I'll also find a minute myself to look at the code).
Without knowing the udev/libusbx internals, it appears that udev_monitor_receive_device may end up calling linux_udev_scan_devices linux_hotplug_lock is locked before udev_monitor_receive_device is called, and linux_udev_scan_devices also tries to lock that mutex, hence the deadlock.
Just for info: I've been running for the past two days a patched version with the usbi_mutex_static_lock(&linux_hotplug_lock); and usbi_mutex_static_unlock(&linux_hotplug_lock); calls commented out in linux_udev.c@linux_udev_event_thread_main, and things seem to work. Whether that is a proper fix however is another question.
Hi, Thanks for all the testing and the new backtrace. There indeed was another deadlock. I've done a new build which should fix both, you can find it here: http://koji.fedoraproject.org/koji/buildinfo?buildID=439434 Please give this version a try, it should solve all the issues you are seeing. Thanks, Hans
Thanks! Will report back in a few days.
Looking good! I'll close this. Thanks a lot!
libusbx-1.0.16-3.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/libusbx-1.0.16-3.fc19
I can reliably reproduce this by undocking and re-docking my Lenovo T420 laptop. The patch does not help (I am on Arch Linux but that probably doesn't matter). When upower is deadlocked: #0 0x00007fa24fd3c00c in __lll_lock_wait () from /usr/lib/libpthread.so.0 #1 0x00007fa24fd37e86 in _L_lock_507 () from /usr/lib/libpthread.so.0 #2 0x00007fa24fd37cda in pthread_mutex_lock () from /usr/lib/libpthread.so.0 #3 0x00007fa2517f1816 in linux_udev_event_thread_main () from /usr/lib/libusb-1.0.so.0 #4 0x00007fa24fd35dd2 in start_thread () from /usr/lib/libpthread.so.0 #5 0x00007fa25079acdd in clone () from /usr/lib/libc.so.6 Interestingly, attaching an strace to upower before it's deadlocked makes the deadlock go away.
Nevermind: I figured that the patch attached to the issue is just one patch but there needs to be two and I have extracted both 0001-linux-Use-a-separate-lock-to-serialize-start-stop-vs.patch 0002-hotplug-Remove-use-of-pthread_cancel-from-linux_udev.patch and can confirm it fixes the problem!
libusbx-1.0.16-3.fc19 has been pushed to the Fedora 19 stable repository. If problems still persist, please make note of it in this bug report.