Bug 1315256

Actually the process also evades from any Attach API client, see for example the minimal Attach API example client in https://bugzilla.redhat.com/show_bug.cgi?id=1311638:

https://bugzilla.redhat.com/attachment.cgi?id=1130254

Thanks.

The issue is hidden in JVM_OPTS. Try to remove them from ./cassandra and you will get:
$jps
...
3270 CassandraDaemon
....

And looking more into ./cassandra script, you will see:
JVM_OPTS="$JVM_OPTS -Dcassandra.libjemalloc=$CASSANDRA_LIBJEMALLOC"
And it is not an cause.

Imho the casue is much more simple - the JPS getting down to MainClass. if it do not find maincalss, it do not include the process on output. So one of:

-Xloggc:./../logs/gc.log -ea -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:StringTableSize=1000003 -XX:+AlwaysPreTouch -XX:-UseBiasedLocking -XX:+UseTLAB -XX:+ResizeTLAB -XX:+PerfDisableSharedMem -Djava.net.preferIPv4Stack=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSWaitDuration=10000 -XX:+CMSParallelInitialMarkEnabled -XX:+CMSEdenChunksRecordAlways -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M -Xms3934M -Xmx3934M -Xmn800M -XX:CompileCommandFile=./../conf/hotspot_compiler -javaagent:./../lib/jamm-0.3.0.jar -Dcassandra.jmx.local.port=7199 -XX:+DisableExplicitGC -Djava.library.path=./../lib/sigar-bin

is an cause.

-XX:+PerfDisableSharedMem is the option preventing jps to see cassandra. This option disables the memory mapped file in /tmp, such as /tmp/hsperfdata_<user>/<pid>, which is used by jps to find Java processes. Try removing that option and you should see it showing up.

The guilty switch is -XX:+PerfDisableSharedMem

I'm really wondering why it is preventing *any* java app to hide from jps.

ok. Nice midair colision.  

Indeeed, this sounds like close-cant fix.

Or reassign to cassandra to clarify why they need it?

http://www.evanjones.ca/jvm-mmap-pause.html might be a reason why they disabled it.

So basically this means that jps (and, more importantly, Attach API) can't be trusted to actually find running JVMs? If so, are there any alternatives to reliably iterate local JVMs?

Thanks.

(In reply to Marko Myllynen from comment #8)
> So basically this means that jps (and, more importantly, Attach API) can't
> be trusted to actually find running JVMs?

This code:

List<VirtualMachineDescriptor> descriptors = VirtualMachine.list();

depends on /tmp/hsperdata_* to be there and readable, yes.

It depends what you mean by trusted. Even for JVMs which don't have the -XX:+PerfDisableSharedMem switch, jps only shows the JVMs for that user running jps. Unless that user is root. In a way that can also be seen as jps-cannot-be-trusted-to-find-running-jvms, even though it's by design.

> If so, are there any alternatives
> to reliably iterate local JVMs?

Not that I'm aware of, no.

(In reply to Severin Gehwolf from comment #9)
> (In reply to Marko Myllynen from comment #8)
> > So basically this means that jps (and, more importantly, Attach API) can't
> > be trusted to actually find running JVMs?
> 
> This code:
> 
> List<VirtualMachineDescriptor> descriptors = VirtualMachine.list();
> 
> depends on /tmp/hsperdata_* to be there and readable, yes.
> 
> It depends what you mean by trusted. Even for JVMs which don't have the
> -XX:+PerfDisableSharedMem switch, jps only shows the JVMs for that user
> running jps. Unless that user is root.

FWIW, jps works but Attach API doesn't (in a meaningful way) for root:

https://bugzilla.redhat.com/show_bug.cgi?id=1311638

> > If so, are there any alternatives
> > to reliably iterate local JVMs?
> 
> Not that I'm aware of, no.

Hmm, ok. So on command line pgrep(1) is probably the most suitable alternative to find Java processes for a user but API-wise there is nothing that could be used instead.

Thanks.

indeed. You are ending in prep and later/or dig from /proc/pid
Are we ok in closing? I doubt jps can reliably fall back to /proc/pid to replicate its original behaviour.

Ok, closing.