Bug 780816 (SOA-3262) - JOIN between two text file data sources extremely slow
Summary: JOIN between two text file data sources extremely slow
Keywords:
Status: CLOSED NEXTRELEASE
Alias: SOA-3262
Product: JBoss Enterprise SOA Platform 5
Classification: JBoss
Component: EDS
Version: 5.1.0 GA,5.2.0 ER1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 5.2.0 GA,5.2.0.ER2
Assignee: Van Halbert
QA Contact:
URL: http://jira.jboss.org/jira/browse/SOA...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-08-11 14:56 UTC by Marc Shirley
Modified: 2011-10-07 16:01 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-10-07 16:01:08 UTC
Type: Bug

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker SOA-3262 0 None None None Never
Red Hat Issue Tracker TEIID-1706 0 None None None Never

Description Marc Shirley 2011-08-11 14:56:57 UTC 
Help Desk Ticket Reference: https://c.na7.visual.force.com/apex/Case_View?id=500A0000007pz3y
project_key: SOA

JOIN between two text files sources is extremely slow.  Testing with 5MB files with around 35000 rows in each file resulted in a result set generation rate of about 100 rows/minute.
Comment 1 Marc Shirley 2011-08-11 15:01:04 UTC 
Help Desk Ticket Reference: Added: https://c.na7.visual.force.com/apex/Case_View?id=500A0000007pz3y
Comment 2 Marc Shirley 2011-08-11 15:20:11 UTC 
Link: Added: This issue Cloned to TEIID-1706
Comment 4 David Le Sage 2011-08-25 23:38:57 UTC 
Release Notes Docs Status: Added: Documented as Resolved Issue
Release Notes Text: Added: https://issues.jboss.org/browse/TEIID-1706

Running a Join operation against two text files was extremely slow. This was because the join planning converted the bushy tree into a linear tree with a nested table join at the top which, in turn, forced a re-evaluation of one of the nested tables for each row.  This has now been changed so that planning is more selective.  As a result, running a Join operation against text files is now faster.
Comment 5 David Le Sage 2011-08-25 23:40:07 UTC 
Writer: Added: dlesage
Comment 6 Paul Nittel 2011-10-07 16:01:08 UTC 
Tested with two 7.2MB (170K rows) text sources. Closed.
Note You need to log in before you can comment on or make changes to this bug.