Bug 1176718 - Crash in String.getBytes when handling unusual fonts
Summary: Crash in String.getBytes when handling unusual fonts
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: java-1.7.0-openjdk
Version: 6.5
Hardware: Unspecified
OS: Unspecified
Target Milestone: rc
: ---
Assignee: Andrew John Hughes
QA Contact: Lukas Zachar
Depends On:
TreeView+ depends on / blocked
Reported: 2014-12-22 22:55 UTC by James Livingston
Modified: 2019-08-15 04:09 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Prior to this update, the Java Native Interface (JNI) code handling fonts used an incorrect function parameter when setting context. Consequently, Java Virtual Machine (JVM) could terminate unexpectedly when disposing of certain fonts. The use of function parameters by the JNI code has been amended, and JVM no longer crashes in this situation.
Clone Of:
Last Closed: 2015-07-22 05:54:25 UTC

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
openjdk bug system JDK-8020190 None None None Never
Icedtea Bugzilla 1326 None None None 2019-01-28 21:01:03 UTC
Icedtea Bugzilla 2160 None None None 2019-01-28 21:01:02 UTC
Icedtea Bugzilla 2509 None None None 2019-01-28 21:01:02 UTC
Icedtea Bugzilla 2512 None None None 2019-01-28 21:01:02 UTC
Red Hat Knowledge Base (Solution) 369343 None None None Never
Red Hat Product Errata RHEA-2015:1245 normal SHIPPED_LIVE java-1.7.0-openjdk bug fix and enhancement update 2015-07-21 14:15:13 UTC

Description James Livingston 2014-12-22 22:55:00 UTC
Java_sun_font_FreetypeFontScaler_disposeNativeScaler() passes the scaler rather than the font as the second argument to setupFTContext, which will cause JVM crashes in some circumstances, with stacks which look similar to

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J 8977 C2 java.lang.String.getBytes(Ljava/lang/String;)[B (27 bytes) @ 0x00007fa82ce0ce14 [0x00007fa82ce0cde0+0x34]
v  ~StubRoutines::call_stub
V  [libjvm.so+0x5fba7e]
V  [libjvm.so+0x5fa648]
V  [libjvm.so+0x6093d2]
V  [libjvm.so+0x61a0f5]
C  [libjava.so+0x15b0f]  JNU_GetStringPlatformChars+0x63f
C  [libfontmanager.so+0x9e41]

This is JDK bug https://bugs.openjdk.java.net/browse/JDK-8020190, which is resolved in JDK 8.

Comment 1 James Livingston 2014-12-22 23:01:12 UTC
The code change is at http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/31b8d4931a092cac4f

Comment 3 Andrew John Hughes 2015-01-09 17:23:10 UTC
This has been backported and will appear in the upcoming 2.5.4 update:


Comment 7 James Livingston 2015-04-16 03:55:58 UTC
FYI: with the fix for this, you may then run into bug 1212268.

Comment 9 Andrew John Hughes 2015-07-10 16:24:45 UTC
The reproducer from JDK-8020190 still fails when both this and bug 1212268 are fixed.

Stack: [0x0000033fc2713000,0x0000033fc2814000],  sp=0x0000033fc2811d48,  free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libfontmanager.so+0x332dc]  Java_sun_font_SunLayoutEngine_nativeLayout+0x22c
j  sun.font.SunLayoutEngine.nativeLayout(Lsun/font/Font2D;Lsun/font/FontStrike;[FII[CIIIIIIILjava/awt/geom/Point2D$Float;Lsun/f
j  sun.font.SunLayoutEngine.layout(Lsun/font/FontStrikeDesc;[FIILsun/font/TextRecord;ILjava/awt/geom/Point2D$Float;Lsun/font/Gl
j  sun.font.GlyphLayout$EngineRecord.layout()V+95
j  sun.font.GlyphLayout.layout(Ljava/awt/Font;Ljava/awt/font/FontRenderContext;[CIIILsun/font/StandardGlyphVector;)Lsun/font/St
j  sun.font.ExtendedTextSourceLabel.createGV()Lsun/font/StandardGlyphVector;+63
j  sun.font.ExtendedTextSourceLabel.getGV()Lsun/font/StandardGlyphVector;+9
j  sun.font.ExtendedTextSourceLabel.createLogicalBounds()Ljava/awt/geom/Rectangle2D;+1
j  sun.font.ExtendedTextSourceLabel.getAdvance()F+9
j  java.awt.font.TextLine.init()V+609
j  java.awt.font.TextLine.<init>(Ljava/awt/font/FontRenderContext;[Lsun/font/TextLineComponent;[F[CII[I[BZ)V+79
j  java.awt.font.TextLine.fastCreateTextLine(Ljava/awt/font/FontRenderContext;[CLjava/awt/Font;Lsun/font/CoreMetrics;Ljava/util/Map;)Ljava/awt/font/TextLine;+337
j  java.awt.font.TextLayout.fastInit([CLjava/awt/Font;Ljava/util/Map;Ljava/awt/font/FontRenderContext;)V+83
j  java.awt.font.TextLayout.<init>(Ljava/lang/String;Ljava/awt/Font;Ljava/awt/font/FontRenderContext;)V+124
j  JDK8020190.main([Ljava/lang/String;)V+41
v  ~StubRoutines::call_stub

I traced two issues:

1. LayoutEngine *engine = LayoutEngine::layoutEngineFactory(&fia, script, lang, typo_flags & TYPO_MASK, success);
can return NULL but the code never checks it. Thus, as soon as it
is dereferenced, the JVM segfaults. A fix for this is in OpenJDK
8: "8023052: JVM crash in native layout".

Because this was fixed in 8 before 8020190, this follow-on crash on 7 was missed.

2. Tracing through the code that leads to this NULL return, it
seems odd. In the LayoutEngineFactory function, the line
LEReferenceTo<GlyphSubstitutionTableHeader> gsubTable(fontInstance,gsubTableTag,success)
sets success to LE_INDEX_OUT_OF_BOUNDS_ERROR because
of an attempt to access element 12 from an array of length 10.
This return value is never checked at this point or reset.
So the same success value is used in:
LEReferenceTo<MorphTableHeader2> morxTable(fontInstance, morxTableTag, success);
LEReferenceTo<MorphTableHeader> mortTable(fontInstance, mortTableTag, success);
new LayoutEngine(fontInstance, scriptCode, languageCode, typoFlags, success);

causing them all to immediately fail, the LayoutEngine gets deleted
and NULL is returned.

Running the test on either OpenJDK 8 (with #1 fixed) or manually resetting
success on OpenJDK 7:

(gdb) print success
$31 = (LEErrorCode &) @0x3b31694bdb4: LE_INDEX_OUT_OF_BOUNDS_ERROR
(gdb) set success = LE_NO_ERROR
(gdb) next
717                                result = new LayoutEngine(fontInstance, scriptCode, languageCode, typoFlags, success);
(gdb) print success
$37 = (LEErrorCode &) @0x3b31694bdb4: LE_NO_ERROR
(gdb) next
724            if (result && LE_FAILURE(success)) {
(gdb) print success
$38 = (LEErrorCode &) @0x3b31694bdb4: LE_NO_ERROR

allows it to complete with "java.awt.Rectangle[x=-1,y=-1,width=2,height=2]"

Comment 11 errata-xmlrpc 2015-07-22 05:54:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Comment 12 Andrew John Hughes 2015-10-12 14:49:03 UTC
Issues in comment 9 will be fixed in http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=2509

Comment 13 Andrew John Hughes 2015-10-12 14:50:54 UTC
and #2 in http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=2512

Note You need to log in before you can comment on or make changes to this bug.