Bug 1010984 - Hive Metadata load incorrectly resolves data types
Summary: Hive Metadata load incorrectly resolves data types
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Data Virtualization 6
Classification: JBoss
Component: Teiid
Version: 6.0.0
Hardware: All
OS: Unspecified
unspecified
urgent
Target Milestone: ER2
: 6.0
Assignee: Van Halbert
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-23 12:47 UTC by Filip Nguyen
Modified: 2014-07-09 05:02 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2014-02-06 15:37:07 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker TEIID-2671 0 Blocker Closed Hive Metadata load incorrectly resolves data types 2014-04-22 18:24:06 UTC

Description Filip Nguyen 2013-09-23 12:47:33 UTC
The metadata load uses DESCRIBE keyword which tires to retrieve type name in String runtimeType = getRuntimeType(type);

However, with Hive 0.11 the type name is returned with padding spaces. Hence all the data types are resolved to default java.lang.String for Hive Translator.

This relates to

Comment 1 Filip Nguyen 2013-09-23 12:48:37 UTC
The related JIRA is https://issues.jboss.org/browse/TEIID-2671

Comment 3 JBoss JIRA Server 2013-09-23 18:23:12 UTC
Ramesh Reddy <rareddy> made a comment on jira TEIID-2671

Ok, this sounded familiar, but after loading sandbox, I saw https://issues.jboss.org/browse/TEIID-2524 which was logged before and resolved.  Since we are recommending 0.11 version of the driver do the "trim" always instead of requiring a translator override?

Comment 4 JBoss JIRA Server 2013-09-23 18:29:05 UTC
Steven Hawkins <shawkins> made a comment on jira TEIID-2671

If we are going to require .11, then yes we can remove the override.  If not, the override should stay and also apply to the column types.  We can also look at adding auto-version detection logic like the other jdbc translators so that the user doesn't have to explicitly set to trim.

Comment 5 JBoss JIRA Server 2013-09-23 18:53:11 UTC
Ramesh Reddy <rareddy> made a comment on jira TEIID-2671

Sorry I confused myself

- Looking more closely TEIID-2524, was about column names, not column types, so that got fixed in 0.11, so one may never need to define a "importer.trimColumNames" importer property. Only working with 0.10 one may need this.

- This JIRA was talking about the data types on the same columns, but while testing I have not seen the behavior indicated.

Filip, do have a sample data file/vdb that I can use to duplicate the issue?

Comment 6 Filip Nguyen 2013-09-24 05:41:08 UTC
Ramesh, I will send you an e-mail and provide you with the vdb and connection information so that you can connect to the Hive Server that is exhibiting the issue.

Comment 7 JBoss JIRA Server 2013-09-24 21:12:01 UTC
Ramesh Reddy <rareddy> updated the status of jira TEIID-2671 to Resolved

Comment 8 JBoss JIRA Server 2013-09-24 21:12:01 UTC
Ramesh Reddy <rareddy> made a comment on jira TEIID-2671

1) Hive 0.11 driver can return the types names with spaces appended to it, to trimming them before the checking for defined type is required to detect correct runtime type
2) 0.71 driver when originally the translator developed it looks like in resultset.getTimestamp(index, cal) method cal parameter is ignored, and in 0.11 this method is written as not supported
3) correction for above was required to retrive the timestamp value correctly
4) Since we are recommending 0.11, updated the module.xml with correct version numbers in the documents section
5) Hive also released Hive2 driver where the driver class is org.apache.hive.jdbc.HiveDriver, provided ds example for this driver in docs section
6) tested with both drivers against the local and gss database

Comment 13 JBoss JIRA Server 2014-04-22 18:24:06 UTC
Steven Hawkins <shawkins> updated the status of jira TEIID-2671 to Closed


Note You need to log in before you can comment on or make changes to this bug.