Bug 796965

Summary: RFE: Implement Salesforce Bulk API for inserts/updates
Product: [JBoss] JBoss Data Virtualization 6 Reporter: Marc Shirley <mshirley>
Component: TeiidAssignee: Van Halbert <vhalbert>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.0.0CC: felias, ldimaggi, soa-p-jira, vhalbert
Target Milestone: ER1Keywords: QA-Closed
Target Release: 6.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Previously, JBoss Data Virtualization has not supported bulk inserts. JBoss Data Virtualization 6 now utilizes the Salesforce Bulk API to support this feature. As a result, users can submit multiple inserts to Salesforce in one go, rather than submitting them one at a time. This improves performance by reducing the number of trips between the client application and the server.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-10 08:51:44 UTC Type: Feature Request
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Marc Shirley 2012-02-23 23:14:44 UTC
Description of problem:
Request to add the Salesforce Bulk API functionality to EDS.  The use case would be to improve the performance of writing/updating million record result sets back to salesforce.com.

Version-Release number of selected component (if applicable):
EDS 5.2

Comment 1 Van Halbert 2012-03-05 17:46:19 UTC
Marc,

If the client wants this, can you create a jira under PRODMGT so that the PM's can triage it.   Otherwise, a Teiid jira could be created for the community to work it.  

Thanks.

Van

Comment 2 JBoss JIRA Server 2012-10-11 20:02:20 UTC
Steven Hawkins <shawkins> made a comment on jira TEIID-2248

We'll take a look at this for 8.3, but if turns out to be too large it will get pushed.

Comment 3 JBoss JIRA Server 2012-12-18 15:53:23 UTC
Steven Hawkins <shawkins> made a comment on jira TEIID-2248

This does look fairly concise: http://code.google.com/p/sfdc-bulkapi-demo/source/browse/trunk/demo/src/com/jeffdouglas/BulkLoader.java

The sample uses additional sf libraries and another only minor issue is the translation to csv.  Perhaps the textagg logic should be generalized for use outside of the engine.

Comment 4 JBoss JIRA Server 2013-02-05 23:00:40 UTC
Ramesh Reddy <rareddy> made a comment on jira TEIID-2248

The same example also from http://www.salesforce.com/us/developer/docs/api_asynch/index.htm

Looks like this uses the "invokeHttp" type calls, not SOAP, but does use SOAP for fetching the sessionid. Should I look into updating SalesForceExecutionFactory to extend WSExecutionfactory? If I do then we get into issue with connector side of the connection, as Teiid has separate connectors for WS and SF.

Looks like we can avoid the http://code.google.com/p/sfdc-wsc/ library that is needed since Teiid do not need the extensive parsing of result returned.

Comment 5 JBoss JIRA Server 2013-02-11 21:00:50 UTC
Ramesh Reddy <rareddy> updated the status of jira TEIID-2248 to Resolved

Comment 6 JBoss JIRA Server 2013-02-11 21:00:50 UTC
Ramesh Reddy <rareddy> made a comment on jira TEIID-2248

I ended up using the http://code.google.com/p/sfdc-wsc/ library after all, as it encapsulated lot of call and parsing details. I removed it's dependencies as Teiid does not use them.

The Bulk insert is only applicable for INSERT. There is code already for UPDATE and DELETE based upon multiple IDs. Also Bulk API does not support UPDATE/DELETE.

If there is an error inserting a row into SF, since the communication is ASYNC, there is no proper way to cancel the bulk job. So, the job continues, and reports which rows that have been not inserted. All these errors are currently exposed back to client as SQLWarning. Teiid can throw SQlException but that does not indicate whole job has been aborted, so I choose to implement as SQLWarning.

Comment 7 JBoss JIRA Server 2013-03-25 16:10:13 UTC
Steven Hawkins <shawkins> updated the status of jira TEIID-2248 to Closed

Comment 11 belong 2013-12-10 03:47:18 UTC
Documented in Release Notes as a New Feature.