Description of problem:

Performance regression between patches ipc-5 and ipc-14. The change doesn't show  up much in evsbench, which does not use the flow control API. It shows up in cpgbench which does, and is much worse for smaller message sizes - consistent with the theory that flow control is adding  a fixed overhead per message.

Here are results from cpgbench (slightly modified to test range of e sizes from 256 - 64k bytes)

ipc-14:

24879 messages received   512 bytes per write   5.001 Seconds runtime  4975.090 TP/s   2.547 MB/s.
29775 messages received  1024 bytes per write   5.001 Seconds runtime  5953.890 TP/s   6.097 MB/s.
55775 messages received  2048 bytes per write   5.002 Seconds runtime 11151.547 TP/s  22.838 MB/s.
101660 messages received  4096 bytes per write   5.001 Seconds runtime 20328.788 TP/s  83.267 MB/s.
96930 messages received  8192 bytes per write   5.001 Seconds runtime 19382.333 TP/s 158.780 MB/s.
46482 messages received 16384 bytes per write   5.001 Seconds runtime  9294.894 TP/s 152.288 MB/s.
45756 messages received 32768 bytes per write   5.001 Seconds runtime  9150.170 TP/s 299.833 MB/s.
26071 messages received 65536 bytes per write   5.001 Seconds runtime  5213.320 TP/s 341.660 MB/s.


ipc-5: 
387600 messages received   512 bytes per write   5.002 Seconds runtime 77496.472 TP/s  39.678 MB/s.
361600 messages received  1024 bytes per write   5.004 Seconds runtime 72264.414 TP/s  73.999 MB/s.
301600 messages received  2048 bytes per write   5.004 Seconds runtime 60265.929 TP/s 123.425 MB/s.
312290 messages received  4096 bytes per write   5.005 Seconds runtime 62391.565 TP/s 255.556 MB/s.
211989 messages received  8192 bytes per write   5.001 Seconds runtime 42387.178 TP/s 347.236 MB/s.
117914 messages received 16384 bytes per write   5.002 Seconds runtime 23573.564 TP/s 386.229 MB/s.
61571 messages received 32768 bytes per write   5.002 Seconds runtime 12309.692 TP/s 403.364 MB/s.
29702 messages received 65536 bytes per write   5.002 Seconds runtime  5938.610 TP/s 389.193 MB/s.