Bug 1063918

Created attachment 861854 [details]
Showcase portlet

Automated WSRP test cases for CR01 passed. I checked the logs of several of the WSRP test jobs in jenkins and I didn't see the exception in any of them.

I also tried JSF & RF quickstarts and these don't show the error. So far I found this exception only while using Showcase or RichFaces Metamer.

Is it WSRP only or PBR in general?

I encountered this exception when testing Export/import + JSF-RF4 over WSRP, but I was unable to reproduce it.

Based on the conversation I had with Petr, it seems that this problem is somewhat "intermittent". I couldn't reproduce the exception attached to this issuee, but I could reproduce a NPE like this:

16:34:25,082 ERROR [org.exoplatform.webui.application.portlet.PortletApplicationController] (http-/127.0.0.1:8080-1) Error while rendering the porlet: org.exoplatform.groovyscript.TemplateRuntimeException: Groovy template exception at DataText[pos=Position[col=1,line=14],data=      	fullName = encoder.encode(uicomponent.getUser().getFullName());] for template app:/groovy/admintoolbar/webui/component/UIUserInfoPortlet.gtmpl
Caused by: java.lang.NullPointerException: Cannot invoke method getFullName() on null object


The workaround for this one seems to be logging out and in again. The workaround also did work for Petr, so, I'll be trying to trace down this exception, instead of the reported one, in the hope that it represents the same problem.

Further information: when an action on the UI sends a request from the remote portlet, WSRP proxies this request as GET to the remote. This means that POST requests are converted into GET requests, and the post parameters are lost. As this sounds like a bug to me, I'm adding support for it, and Petr has confirmed that it fixed the original issue.

That said, there are still some other misbehaves on the Showcase. For instance, the push example gets a new UUID every couple seconds when inside a local portlet, but doesn't do the same on a remote portlet. This seems to me like a bug on the PBR component, but I haven't discarded yet that the problem might be on WSRP. Further investigation is hence required. 

Note also that this change affects *all* requests being transited via WSRP, so, full regression testing is required before delivering this feature. 

Pull request was sent and merged.

This issue was fixed in DR1 already, but the target milestone was not set properly. Setting target milestone to DR2 to ensure that it gets validated by a QE.

I am afraid that this issue is back again with 6.2.0.ER1, I got similar exception while testing Showcase. Again, it's gtn.UsernameToken.ifAvailable missing but now the cause is javax.xml.ws.soap.SOAPFaultException: An error was discovered processing the <wsse:Security> header. 

Full stack trace - http://pastebin.com/KerPBDU0

Note that you also need to modify the Showcases web.xml and patch weld-core to newer version due to https://bugzilla.redhat.com/show_bug.cgi?id=1082651. So I enclose already modified Showcase portlet to make it easier to reproduce.

Created attachment 894016 [details]
patched showcase portlet

Petr, I'm seeing this exception once, upon creating the page. Is this also what you experience? Or do you see this exception more times?

So, I found the reason for the exception, and have two possible fixes. The issue here is not specific to Showcase, but the nature of it makes more evident, as it sends a good number of parallel requests. 

The problem is that the interceptor GTNSubjectCreatingInterceptor seems to be shared on a main request on all WS requests inside it. This interceptor plays with its properties, setting it to a temporary value at some point, and reverting it later. 

With one interceptor per thread, it would look like this: 

1 - t1 identifies property ACTION as gtn.UsernameToken.ifAvailable
2 - t1 sets property ACTION to UsernameToken
3 - t1 calls super.handleMessage, which uses ACTION
4 - t1 sets property to gtn.UsernameToken.ifAvailable

When more threads are involved, it would look like this:

1 - t1 identifies property ACTION as gtn.UsernameToken.ifAvailable
2 - t1 sets property ACTION to UsernameToken
3 - t2 identifies that the property ACTION is already UsernameToken
4 - t1 calls super.handleMessage, which uses ACTION
5 - t1 sets property to gtn.UsernameToken.ifAvailable
6 - t2 calls super.handleMessage, with ACTION=gtn.UsernameToken.ifAvailable

t2 fails, as this action is not recognizable. 

As of now, I see two possible solutions, which I still need to check the technical feasibility:

1) Isolate the interceptor instances, to have one per request. This might have an impact on the memory consumption, as we would then have more instances (short-lived), instead of one longer-lived.

2) Let the ACTION be always UsernameToken. This might prove to not be a big problem, as this is the reason why we see the exception only once (as another thread will set the value to UsernameToken but not revert it due to an exception). 

In any case, I'd need a couple more days to check what would be the "right" approach, as per WS-Security.

Created attachment 898418 [details]
Request log, showing the concurrency issue

Created attachment 937583 [details]
Byteman rule for easier reproducing of this issue

So I checked this issue using Byteman rule (which adds Thread#sleep into GTNSubjectCreatingInterceptor to simulate high-concurrency load) and I wasn't able to reproduce it again so I am marking this as verified. I am also attaching that rule in case somebody would need to debug this again.

Juraci Paixão Kröhling <jpkroehling> updated the status of jira GTNWSRP-372 to Closed

This product has been discontinued or is no longer tracked in Red Hat Bugzilla.