Description of problem: I'm getting 'Resource temporarily unavailable' when starting processes as a user often with the new kernel. I have the stock user process limit (1024), and there's only about 700-750 *total* processes on the system when this happens. Usual symptom is: -bash: fork: retry: Resource temporarily unavailable root is able to start new processes. If I try 'su notting -c ls', I get a similar EAGAIN failure: 31445 open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3 31445 fstat(3, {st_mode=S_IFREG|0644, st_size=2337, ...}) = 0 31445 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdaab9d2000 31445 read(3, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 2337 31445 close(3) = 0 31445 munmap(0x7fdaab9d2000, 4096) = 0 31445 getuid() = 0 31445 getgid() = 0 31445 setregid(2167, 4294967295) = 0 31445 setreuid(2166, 4294967295) = -1 EAGAIN (Resource temporarily unavailable) Starting off against kernel... may be libc, may be something else. I can fix it by (as root) killing off some number of processes. Version-Release number of selected component (if applicable): kernel-2.6.35.2-9.fc14.x86_64 glibc-2.12.90-8.x86_64 systemd-7-3.fc14.x86_64 How reproducible: 100% (after a while)
Aha, wasn't counting the processes right. I am running against the rlimit for number of active threads (1024). Assigning to the offender.
Created attachment 441324 [details] un-wtf gwibber service gwibber-service was creating a new worker pool of threads/processes for each async request that it decided to do. This fixes that.
Created attachment 441326 [details] patch, v2 Patch v2 cleans up some cruft that's always the same.
That being said, the gwibber code has been this way for a while. I wonder why it's suddenly a problem now.
Upstream claims that the code is supposed to destroy the pool after the operation finishes. Perhaps this is a python issue? I'm going to push an update with this patch anyways, since it isn't necessarily wrong. Also opened a ticket with upstream.
CC'ing David re: comment #5.
Looking briefly at MapAsync.run(), I see that it creates a multiprocessing.Pool() and assigns it to local "pool"; by default it uses the CPU count for the number of child processes. Looking briefly over multiprocessing.pool.Pool, it doesn't seem to have an explicit __del__ method, but it does have methods like terminate(). So when "pool" goes out of scope, the pool will get cleaned up at some future point (it may well have cyclic references to it by other object, so may have to wait for the garbage collector). It looks like it may help to add an explicit: finally: pool.terminate() or somesuch, to explicitly tell the code that it really ought to try to clean up those threads, rather than waiting for implicit cleanup during garbage collection.
(In reply to comment #7) > up those threads, rather than waiting for implicit cleanup during garbage s/threads/processes/
Note that in my patch, the t.join() is superfluous (and actually makes it more blocking.)
*** Bug 628418 has been marked as a duplicate of this bug. ***
gwibber-2.33.0-7.867bzr.fc13 has been submitted as an update for Fedora 13. https://admin.fedoraproject.org/updates/gwibber-2.33.0-7.867bzr.fc13
gwibber-2.33.0-7.867bzr.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/gwibber-2.33.0-7.867bzr.fc14
gwibber-2.33.0-7.867bzr.fc12 has been submitted as an update for Fedora 12. https://admin.fedoraproject.org/updates/gwibber-2.33.0-7.867bzr.fc12
gwibber-2.33.0-7.867bzr.fc14 has been pushed to the Fedora 14 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update gwibber'. You can provide feedback for this update here: https://admin.fedoraproject.org/updates/gwibber-2.33.0-7.867bzr.fc14
gwibber-2.33.0-7.867bzr.fc14 has been pushed to the Fedora 14 stable repository. If problems still persist, please make note of it in this bug report.
gwibber-2.33.0-7.867bzr.fc13 has been pushed to the Fedora 13 stable repository. If problems still persist, please make note of it in this bug report.
gwibber-2.33.0-7.867bzr.fc12 has been pushed to the Fedora 12 stable repository. If problems still persist, please make note of it in this bug report.