Bug 459152

Summary: [RFE] add asynchronous version of mcast_joined to CPG
Product: Red Hat Enterprise Linux 6 Reporter: Alan Conway <aconway>
Component: corosyncAssignee: Angus Salkeld <asalkeld>
Status: CLOSED NEXTRELEASE QA Contact: Cluster QE <mspqa-list>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.0CC: bstevens, ccaulfie, cluster-maint, iannis, sdake, snagar
Target Milestone: betaKeywords: FutureFeature, Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-02-15 16:58:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
modified cpg.c to make mcast asynchrnous. none

Description Alan Conway 2008-08-14 18:44:33 UTC
Created attachment 314341 [details]
modified cpg.c to make mcast asynchrnous.

Description of problem:

cpg_mcast_joined is a synchronous call, i.e. the caller has to wait for the IPC round-trip to the aisexec daemon before proceeding. An asynchronous call from the client library would give much better throughput. This is to support the MRG project in order to use CPG as its replication protocol. Further work needed to determine the extent of the benefit, but inital experiment below showed a threefold speedup.

I hacked lib/cpg.c (attached) to ignore the response in mcast_joined and
instead retrieve and discard them in the dispatch poll. This almost
triples throughput of the cpgbench test on my laptop.

I'm not proposing this hack as an implementation, it was just to get an
idea of possible performance impact. At least the following things are
wrong with it:
 - async_mcast should be a new API, existing sync mcast should remain.
 - wrong locking around response_fd
 - polling 2 fd's in dispatch so cpg_fd_get() is no longer sufficient. 
 - cpgbench goes into flow control quickly (not surprising) but then
never gets out, I think I missed something about resetting flow control.

A neater solution might be to drop the responses at the daemon for
successful mcasts and send an error response to the dispatch_fd for
failed mcasts so there's only 1 fd that needs to be polled. Flow control changes as a result of async mcasts would also need to be sent to dispatch_fd.

Version-Release number of selected component (if applicable):

openais-0.80.3-17.fc9.i386

Steps to Reproduce:

Run cpgbench against normal AIS and with the attached changes to see the throughput difference. I saw almost a factor of 3 improvement.


Additional info:

Comment 1 Christine Caulfield 2009-01-19 15:39:32 UTC
Forward to Steve as he is working on the new IPC layer that is needed to support this.

Comment 2 Steven Dake 2009-03-18 21:36:50 UTC
reassigning to rhel6 since that is the new upstream version for corosync.  If you would like this in a rhel5 release please open a separate bz.

thanks

Comment 3 RHEL Program Management 2009-06-15 20:56:57 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 6 Alan Conway 2010-01-13 15:31:55 UTC
It would mean a performance improvement for MRG but it's not a critical requirement in the short term.

Comment 8 Steven Dake 2010-03-01 09:13:02 UTC
Honza asked to implement this feature.

reassigned.

Comment 10 RHEL Program Management 2010-03-01 09:22:29 UTC
Development Management has reviewed and declined this request.  You may appeal
this decision by reopening this request.