Bug 1623579

Created attachment 1479539 [details]
reproducer-804632.tar.xz

Created attachment 1479540 [details]
rhel-7.5-pkgs-beaker-broken.txt

Created attachment 1479541 [details]
rhel-7.6-pkgs-beaker-working.txt

(In reply to zzambers from comment #0)
> When I was trying to run on reproducer for 804632 it failed with following
> output on rhel-7.5 distro in beaker:

Thant's bug 804632. The reproducer is:

$ cat Hello.java
public class Hello {
	public static void main(String[] args) {
		System.out.println("Hello World");
	}
}
$ javac Hello.java
$ sudo stap -e 'probe hotspot.jni.GetStringUTFChars { print_jstack_full(); }' -c 'java Hello'

print_jstack_full() comes from this file part of java-1.8.0-openjdk-devel:
/usr/share/systemtap/tapset/x86_64/jstack-1.8.0.181-3.b13.el7_5.x86_64.stp

$ rpm -qf /usr/share/systemtap/tapset/x86_64/jstack-1.8.0.181-3.b13.el7_5.x86_64.stp
java-1.8.0-openjdk-devel-1.8.0.181-3.b13.el7_5.x86_64

Solving the underlying problem doesn't look like a trivial fix, and may require rethinking the arc_priority representation in stapregex. Filed upstream as PR23608.

Documenting potential workarounds until the underlying bug is fixed:

- use a shorter regex (e.g. drop prefix "/usr/" in the 7.5 tapset)
- use a non-regex matching method to match the exact string (e.g. strpos() != -1 -- available as of SystemTap 3.2)

I've written a fix upstream as commit 3c18de01ef8.

https://sourceware.org/git/gitweb.cgi?p=systemtap.git;a=commit;h=3c18de01ef8cb267f8947c5572d3c140540dc591

Severin, a java-side workaround would be top stop using the regex operator in that context in your tapset script.  (Sorry if that was my suggestion in the first place.)  Instead of 

@define _check_match %(
  %( systemtap_v >= "3.0" %? if (pp() !~ "/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64/") next %)
%)

you could emit

@define _check_match %(
  %( systemtap_v >= "3.0" %? if (strpos(pp(),"/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64/") < 0) next %)
%)

(In reply to Frank Ch. Eigler from comment #15)
> Severin, a java-side workaround would be top stop using the regex operator
> in that context in your tapset script.  (Sorry if that was my suggestion in
> the first place.)  Instead of 
> 
> @define _check_match %(
>   %( systemtap_v >= "3.0" %? if (pp() !~
> "/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64/") next %)
> %)
> 
> you could emit
> 
> @define _check_match %(
>   %( systemtap_v >= "3.0" %? if
> (strpos(pp(),"/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64/")
> < 0) next %)
> %)

Thanks. Serguei said that strpos() is available from 3.2+. If so, shouldn't this be guarded by 3.2, like so?

@define _check_match %(
   %( systemtap_v >= "3.2" %? if
 (strpos(pp(),"/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64/")
 < 0) next %)

I'll proceed with getting an upstream IcedTea bug filed for the work-around. IcedTea maintains the tapsets for OpenJDK. We only consume it from there.

(In reply to Severin Gehwolf from comment #16)
> I'll proceed with getting an upstream IcedTea bug filed for the work-around.

https://icedtea.classpath.org/bugzilla/show_bug.cgi?id=3625

Yes, that conditional would work.
For operation on older systemtap, one can add a little more script code and make it work there too, if that is desirable.

The following upstream commits are relevant if doing a respin of systemtap:

- 3c18de01ef8cb267f8947c5572d3c140540dc591 PR23608: rebalance arc_priorities to avoid overflow.
- 64e5e3cc638d86614969a320657ce4250f06722f PR23608 refine_*: replace assertion failure with regex_error

Note that the jstack.stp fix did make it upstream, but is currently not in the java-1.8.0-openjdk-devel package for RHEL 7.6.

Should this bug not be moved to java-1.8.0-openjdk if that's where the fix is needed?

This bug will be used to drive a backport of the original systemtap-side regexp processing bug.  On the openjdk side, I know of two separate issues: a workaround for the this systemtap bug, and a porting-to-ppc64 one.

Yes, it wasn't clear if this bug was being used for the OpenJDK workaround or a SystemTap fix.
As it's the latter, I'll use the ppc64 bug to update the tapsets from upstream IcedTea, bringing in both fixes.

automatically included in 4.0 rebase

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2172