AFAICS python-nss-0.8 does not manipulate the Python global interpreter lock, making it impossible to perform more than one I/O operation through NSS using threads, or in fact to perform _any_ operation in a thread if another thread is blocked in a NSS operation. See http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock . This is affecting my attempts to speed up the Fedora signing server.
Yes, you're correct, python-nss is not releasing the GIL during blocking IO operations. I'm in the process of adding a bunch of new features to python-nss which should be complete in about a week. I'll make sure to add the thread support in the next release.
Created attachment 413799 [details] Release the Python GIL for operations that might block This patch against python-nss-0.8 seems to work fine for me.
Thank you for the patch, I truly appreciate it, but you really should have checked with me first before doing this work. It's already been implemented so I probably won't take advantage of the patch, plus I doubt it would apply cleanly because there has been a lot of updates to the code. Once I have the fixes in for the cyclic garbage collector I'll give you a pointer to a test package.
Created attachment 413861 [details] A few more GIL changes This is the result of merging the above patch against current CVS, releasing the lock in a few places, and - more importantly - acquiring the lock in callbacks from C to Python.
Thank you for the patch Miloslav. I appreciate it. Looks good. Thank you for addressing the issue with enusring the GIL state in the callback, that was a critical component I had missed. The patch has been applied. I'll ping you with a pointer to a test build early next week.
I have prepared a new python-nss package which includes your suggested fixes as well as fixes for reference counting, thread local storage, cyclic garbage collection, and a host of new features. The release is 0.9. I have built it in rawhide (http://koji.fedoraproject.org/koji/buildinfo?buildID=175538). I have extensively tested the new version with valgrind, with a special python interpreter build which adds reference counting assertions, and using a script I wrote to verify there are exact matching deallocs for each new object created. There are no objects uncollected, no excessive decref's, no memory accessed when it shouldn't be. Of course I may have missed something. I would appreciate if you could both test the package and do a source code review and provide feedback. Thank you!
python-nss-0.9-8.fc13 has been submitted as an update for Fedora 13. http://admin.fedoraproject.org/updates/python-nss-0.9-8.fc13
python-nss-0.9-8.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/python-nss-0.9-8.fc12
python-nss-0.9-8.fc13 has been pushed to the Fedora 13 stable repository. If problems still persist, please make note of it in this bug report.
python-nss-0.9-8.fc12 has been pushed to the Fedora 12 stable repository. If problems still persist, please make note of it in this bug report.