Bug 2030399

Summary: OpenJDK crashes when using JFR
Product: Red Hat Enterprise Linux 8 Reporter: Lei Yu <leiyu>
Component: java-1.8.0-openjdkAssignee: Zhengyu <zgu>
Status: CLOSED ERRATA QA Contact: OpenJDK QA <java-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 8.5CC: ahughes, jandrlik, jvanek, lkonno, neugens, pmikova
Target Milestone: rcKeywords: Triaged, ZStream
Target Release: ---Flags: pm-rhel: mirror+
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: java-1.8.0-openjdk-1.8.0.322.b04-0.2.ea.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2038935 (view as bug list) Environment:
Last Closed: 2022-05-10 13:35:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2038935    
Deadline: 2022-01-17   

Description Lei Yu 2021-12-08 16:58:59 UTC
The customer is using java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.ppc64le on powerpc, and the jvm crashes with core dumps. The following was noticed in the core dump:

[1] (gdb) f 15
#15 JfrStackTrace::record_thread (this=0x7ffea6b5e398, thread=..., frame=...) at /usr/src/debug/java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.ppc64le/openjdk/hotspot/src/share/vm/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp:408
408	  vframeStreamSamples st(&thread, frame, false);

(gdb) p thread._osthread 
$1 = (OSThread *) 0x7ffecc0eb090
(gdb) p *(OSThread *) 0x7ffecc0eb090
$2 = {
  <CHeapObj<(MemoryType)2>> = {<No data fields>}, 
  members of OSThread: 
  _start_proc = 0x0, 
  _start_parm = 0x0, 
  _state = RUNNABLE, 
  _interrupted = 0, 
  _thread_type = 3, 
  _pthread_id = 140727742361968, 
  _caller_sigmask = {
    __val = {4, 0, 0, 0, 0, 0, 0, 0, 0, 140727742359120, 0, 0, 0, 0, 0, 140732322017280}
  }, 
  sr = {
    _state = os::SuspendResume::SR_SUSPENDED
  }, 
  _siginfo = 0x7ffdbb17bb28, 
  _ucontext = 0x7ffdbb17adb0, 
  _expanding_stack = 0, 
  _alt_sig_stack = 0x0, 
  _startThread_lock = 0x7ffecc1570b0, 
  _thread_id = 2812522
}

(gdb) thr find 2812522
Thread 139 has target id 'Thread 0x7ffdbb17f170 (LWP 2812522)'
(gdb) thr 139
[Switching to thread 139 (Thread 0x7ffdbb17f170 (LWP 2812522))]
#0  0x00007fffbb5b3c0c in __GI___sigsuspend (set=0x7ffdbb17ac68) at ../sysdeps/unix/sysv/linux/sigsuspend.c:26
26	  return SYSCALL_CANCEL (rt_sigsuspend, set, _NSIG / 8);
(gdb) bt
#0  0x00007fffbb5b3c0c in __GI___sigsuspend (set=0x7ffdbb17ac68) at ../sysdeps/unix/sysv/linux/sigsuspend.c:26
#1  0x00007fffbaf86e68 in SR_handler (sig=<optimized out>, siginfo=<optimized out>, context=<optimized out>) at /usr/src/debug/java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.ppc64le/openjdk/hotspot/src/os/linux/vm/os_linux.cpp:4366
#2  <signal handler called>
#3  OptoRuntime::new_array_C (array_type=0x7fff2f800208, len=292, thread=0x7ffecc0ff000) at /usr/src/debug/java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.ppc64le/openjdk/hotspot/src/share/vm/opto/runtime.cpp:286
#4  0x00007fff740a45b8 in ?? ()
#5  0x00007fff76e57928 in ?? ()
#6  0x00007fff7727b40c in ?? ()
#7  0x00007fff7400a004 in ?? ()
#8  0x00007fff74009ea0 in ?? ()
#9  0x00007fff7727cef0 in ?? ()
#10 0x00007fff7400a004 in ?? ()
#11 0x00007fff74009ea0 in ?? ()
#12 0x00007fff74009ea0 in ?? ()
#13 0x00007fff74009ea0 in ?? ()
#14 0x00007fff774b6b90 in ?? ()
#15 0x00007fff774ae8c8 in ?? ()
#16 0x00007fff74f4f9d4 in ?? ()
#17 0x00007fff7727b9e8 in ?? ()
#18 0x00007fff7400a004 in ?? ()
#19 0x00007fff74009ea0 in ?? ()
#20 0x00007fff74009ea0 in ?? ()
#21 0x00007fff74009ea0 in ?? ()
#22 0x00007fff74009ea0 in ?? ()
#23 0x00007fff74009ea0 in ?? ()
#24 0x00007fff7400a004 in ?? ()
#25 0x00007fff74009ea0 in ?? ()
#26 0x00007fff7400a004 in ?? ()
#27 0x00007fff74009ea0 in ?? ()
#28 0x00007fff7400a004 in ?? ()
#29 0x00007fff74009ea0 in ?? ()
#30 0x00007fff7764c3c4 in ?? ()
#31 0x00007fff74009ea0 in ?? ()
#32 0x00007fff74009ea0 in ?? ()
#33 0x00007fff74009910 in ?? ()
#34 0x00007fff74009910 in ?? ()
#35 0x00007fff74009ea0 in ?? ()
#36 0x00007fff7764c3c4 in ?? ()
#37 0x00007fff74009ea0 in ?? ()
#38 0x00007fff74009ea0 in ?? ()
#39 0x00007fff74009ea0 in ?? ()
#40 0x00007fff74009ea0 in ?? ()
#41 0x00007fff7400a004 in ?? ()
#42 0x00007fff7400a004 in ?? ()
#43 0x00007fff7400a004 in ?? ()
#44 0x00007fff7400a004 in ?? ()
#45 0x00007fff7400a074 in ?? ()
#46 0x00007fff7400093c in ?? ()
#47 0x00007fffbac0a850 in JavaCalls::call_helper (result=0x6c4b13600, m=<optimized out>, args=0x6c4a00018, __the_thread__=0x10)
    at /usr/src/debug/java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.ppc64le/openjdk/hotspot/src/share/vm/runtime/javaCalls.hpp:221
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Maybe jfr does not handle well signal handler frames (possibly only on powerpc).

Comment 4 Zhengyu 2021-12-10 14:05:47 UTC
The code is missing on jdk15u for linux ppc as well, so the code probably never ported to linux ppc. I will ping Thomas Stuefe from SAP to see if it is intentional.

Comment 5 Zhengyu 2021-12-10 15:16:39 UTC
Thomas' reply:


Just an oversight. I actually unified all that coding into os_posix.cpp for JDK 17, so now the protector is present everywhere. But seems it's missing in 16 and earlier for ppc.

The problem is that ppc (all our ports, really) originally forked from either Linux or Solaris. And upstream devs only fixed things on Oracles main platforms so we had to regularly scan those for updates. Like a sideway upstream. Error-prone. It got better after we integrated our platforms into the OpenJDK and upstream became used to the idea of ppc and s390.

Cheers, Thomas

Comment 7 Zhengyu 2021-12-21 18:41:20 UTC
Filed upstream bug: https://bugs.openjdk.java.net/browse/JDK-8279077

Comment 8 Zhengyu 2021-12-21 20:38:49 UTC
Upstream bug is going to be fixed in jdk15u, then downport to 11u and 8u.

Comment 10 Zhengyu 2022-01-05 14:15:48 UTC
The upstream fix has been integrated to JDK15u and started to downport to JDK11u and JDK8u.

Comment 12 Zhengyu 2022-01-10 13:34:05 UTC
The patch has been integrated in upstream jdk15u, 11u and 8u.

Comment 22 errata-xmlrpc 2022-05-10 13:35:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (java-1.8.0-openjdk bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1775