Bug 1076572

Description of problem:
The JBoss Marshaller for EJB remote calls fails with a StackOverflowError if the called method throws an exception, that is not part of the public contract but an implementation-private subclass of the exception defined in the contract.

Version-Release number of selected component (if applicable):
org.jboss.marshalling 1.3.16GA

How reproducible:
Always (see test case)

Steps to Reproduce:
1. Define two WARs (separate class loaders, that's important!)
2. Module 2 defines a method with "throws Exception_A"
3. In the implementation of Module 2, instead throw an Exception_B which extends Exception_A
4. In Module 1, call this method of module 2 through EJB remoting. So module 1 knows the contract of module 2 (and thus also Exception_A) but not the implementation details (so Exception_B is completely unknown to module 1)

See the attached maven project for a complete sample with two server modules and a standalone client that calls module 1, which in turn calls module 2 as described in the above description. To reproduce, compile the sample project, deploy modules "m1" and "m2" to a local JBoss installation and then launch the standalone client. the client should report the aforementioned StackOverflowError.


Actual results:
Infinite loop, leading to StackOverflowError in
Caused by: java.lang.StackOverflowError
	at java.util.IdentityHashMap.clear(IdentityHashMap.java:613) [rt.jar:1.7.0_45]
	at org.jboss.marshalling.cloner.SerializingCloner.reset(SerializingCloner.java:126)
	at org.jboss.marshalling.cloner.SerializingCloner.clone(SerializingCloner.java:139)
	at org.jboss.marshalling.cloner.SerializingCloner.initSerializableClone(SerializingCloner.java:274)
	at org.jboss.marshalling.cloner.SerializingCloner.initSerializableClone(SerializingCloner.java:276)
	at org.jboss.marshalling.cloner.SerializingCloner.initSerializableClone(SerializingCloner.java:276)
	at org.jboss.marshalling.cloner.SerializingCloner.initSerializableClone(SerializingCloner.java:276)
	at org.jboss.marshalling.cloner.SerializingCloner.initSerializableClone(SerializingCloner.java:276)

Expected results:
Module 1 should correctly unmarshal the unknown exception subclass.

Additional info:
The problem is in method org.jboss.marshalling.cloner.SerializingCloner.initSerializableClone
Two bad things are happening here:
1. When unmarshalling the exception, SerializingCloner expects that the exception class is either completely known in the current class loader or not at all. But in our case, we have a mixture where the base class of the exception is actually known to the class loader of module 1 and module 2, but the specific subclass in defined in module 2 only and unknown to module 1. This results in a failure of line 276:
    if (cloneClass != clone(objClass))
because in this mixed case, the parent class is already known and doesn't need to be cloned, so actually cloneClass == objClass
2. If the aforementioned condition is true, then a recursive call to initSerializableClone is issued with the exact same parameters. This naturally leads to an endless recursion. A comment above suggest, that the recursive call should check a superclass instead of the class itself but it is unclear what exactly needs to be changed here.


We could solve the problem for our case by patching the aforementioned condition to
    if (cloneClass != objClass && cloneClass != clone(objClass))

This avoids going into the endless recursion and enabled module 1 to correctly unmarshal the (locally unknown) exception subclass. We aren't sure though, whether this really fixes the original problem or just the symptoms. With our fix, the exception subclass is correctly propagated to the calling client (where it fails with a ClassNotFoundException - so unmarshalling also seems to differ between EJB remote clients and remote calls inside the JBoss container, but that's another story).


The same goes for subclasses of RuntimeExeption which have an even higher propability of not being defined as part of the API contract.