Red Hat Bugzilla – Bug 765336
Streamline translator-local structure allocation
Last modified: 2013-12-18 19:06:55 EST
Right now, the translator API includes a "local" field in the
call_frame_t structure, which many translators use to hold their own
state across asynchronous calls/callbacks. Typically, they GF_CALLOC a
translator-specific structure and stick the pointer into frame->local.
This does get freed automatically when the frame itself is freed, which
is ever so slightly convenient, but otherwise we have a situation in
which every single translator that touches a request incurs a whole
system-level malloc/free pair - not even benefiting from the pool
allocation used for the frames themselves. Besides the cycles
consumed, this also has implications for page and cache warmth.
What I'm proposing is that each translator can provide an *optional*
new entry point which the translator infrastructure can use to query
the size of the structure that the translator will be using "local" to
hold. That infrastructure can then walk the translator graph to
determine the total size of all such structures in the longest path, and
use a single malloc/free for all at once. This might in some cases
result in allocating space for a structure that won't be needed, but in
the great majority of cases it would be more efficient than what we have
now. It would also be more convenient, since translators that
implement the new query function wouldn't have to manage the
local-structure life cycle themselves. Other translators could
continue to use the current method indefinitely.
I'd be willing to write the infrastructure code for this, and modify
some of the most commonly used translators to use it as examples. Any
We do had plans of bringing in mempools for the local allocations. Currently Shehjar started that effort (ref: http://review.gluster.com/#dashboard,1000032 )
We also want to bring in more important logs in statedump about how many times a given type of structure is allocated (so we can decide if it is high %, make it mempool or variable sized iobuf). (ref: http://review.gluster.com/376)
Considering this, I think starting on this effort right now would be bit of duplicate work. Let me know your thoughts.
CHANGE: http://review.gluster.com/2772 (core: utilize mempool for frame->local allocations) merged in master by Anand Avati (email@example.com)
CHANGE: http://review.gluster.com/2784 (mempool: adjustments in pool sizes) merged in master by Vijay Bellur (firstname.lastname@example.org)
Now the local structure for the xlators are allocated from the mempool instead of doing malloc/free.