Measurements of the speed of vmalloc show that it appears to be slower than it ought to be by about 10x based on a set of measurements on recent kernels. See below for details of the test and results.
Created attachment 405928 [details] Test module used to measure vmalloc speed This test module does one million loops of vmalloc two pages, touch memory, vfree and then prints out the time taken. The results from a recent upstream kernel (2.6.34-rc2) look like this: vmalloc took 148798983 us vmalloc took 151664529 us vmalloc took 152416398 us vmalloc took 151837733 us
Created attachment 405929 [details] Test patch to help track down vmalloc issue With the attached patch applied, the new results from the test module look like this: vmalloc took 15363634 us vmalloc took 15358026 us vmalloc took 15240955 us vmalloc took 15402302 us thats a 10x speed up...
It looks like the issue is that the lazy TLB code is causing lots of old vmalloc'ed areas to stay around, making new allocations slow. I wonder if its possible to make the caching of older areas less aggressive without reducing the beneficial effects of fewer TLB flushes?
There is a patch in -mm that seems to fix this, but it doesn't seem to be going anywhere: http://userweb.kernel.org/~akpm/mmotm/broken-out/mm-vmap-area-cache.patch
This bug appears to have been reported against 'rawhide' during the Fedora 14 development cycle. Changing version to '14'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Might as well close this since the upstream resolution is that we shouldn't be using vmalloc for any allocation where performance matters.