Bug 1146522 - Vanilla Hive sorting
Summary: Vanilla Hive sorting
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Data Virtualization 6
Classification: JBoss
Component: Teiid
Version: 6.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ER4
: 6.1.0
Assignee: Van Halbert
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-09-25 12:00 UTC by Filip Nguyen
Modified: 2016-02-10 08:58 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Apache Hive 0.13 sorted string numbers in a way different from other Hive implementations like Cloudera Impala. To fix this proble, the translator property collationLocale was added. if there is a difference between this and the system property org.teiid.collationLocale it will inhibit the pushdown of order by string values for join processing and with the system property org.teiid.requireTeiidCollation set will inhibit any pushdown of a string ordering where the locales do not match. As a result, strings are now sorted as expected.
Clone Of:
Environment:
Last Closed: 2016-02-10 08:58:22 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker TEIID-3142 0 Major Closed Vanilla Hive sorting issue 2016-01-26 18:07:37 UTC
Red Hat Issue Tracker TEIID-3156 0 Major Closed Provide a mechanism to turn off string sorting 2016-01-26 18:07:37 UTC

Description Filip Nguyen 2014-09-25 12:00:32 UTC 
It seems that Vanilla Apache Hive 0.13 sorts in a yet another different way than what we expect and also differently from Cloudera Imapla (another Hive flavor).

SELECT BQT1.SmallA.StringNum FROM BQT1.SmallA order by BQT1.SmallA.StringNum

1: -1
2: -10
3: -11
4: -12
5: -13
6: -14
7: -15
8: -16
9: -17
10: -18
11: -19
12: -2
13: -20
14: -21
15: -22
16: -24
17: -3
18: -4
19: -5
20: -6
21: -8
22: -9
23: 0
24: 1
25: 10
26: 11
27: 12
28: 13
29: 14
30: 15
31: 16
32: 17
33: 18
34: 19
35: 2
36: 20
37: 21
38: 22
39: 23
40: 24
41: 3
42: 4
43: 5
44: 6
45: 7
46: 8
47: null
48: null
49: null
50: null
Comment 2 Van Halbert 2014-10-09 21:00:16 UTC 
Upstream feature was added to help with sorting options: https://issues.jboss.org/browse/TEIID-3156

Since we are moving more into European / Asian territories where different locales will occur, this is something that probably should back port, right?
Comment 3 Filip Elias 2014-10-13 15:45:29 UTC 
Yes, it should be back ported. It is better to use TEIID-3156 than disable the sorting completely. 

Otherwise it would be in some rollup patch probably so it is better to back port it now.
Comment 4 JBoss JIRA Server 2014-10-21 17:33:34 UTC 
Steven Hawkins <shawkins> updated the status of jira TEIID-3142 to Resolved
Comment 5 JBoss JIRA Server 2015-03-05 15:41:01 UTC 
Steven Hawkins <shawkins> updated the status of jira TEIID-3142 to Closed
Comment 6 JBoss JIRA Server 2016-01-26 18:00:55 UTC 
Steven Hawkins <shawkins> updated the status of jira TEIID-3156 to Closed
Note You need to log in before you can comment on or make changes to this bug.