Bug 770441 - A lot of SerializationException in server.log after jboss starts
Summary: A lot of SerializationException in server.log after jboss starts
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-engine-webadmin
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Vojtech Szocs
QA Contact:
URL:
Whiteboard: infra
: 783985 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-12-26 15:53 UTC by Rami Vaknin
Modified: 2014-01-12 23:53 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-12-13 08:12:10 UTC
oVirt Team: ---


Attachments (Terms of Use)

Description Rami Vaknin 2011-12-26 15:53:58 UTC
I installed ovirt on devel env, when I build webadmin and UserPortal for all user agents, the following exception appears a lot of times in the server.log file.
Note that I have second environment which I build only for firefox user agent (-Dgwt.userAgent=gecko1_8), the exception does NOT appear there.

17:46:20,640 ERROR [[/webadmin]] Exception while dispatching incoming RPC call
com.google.gwt.user.client.rpc.SerializationException: Could not find ClientOracle data for permutation 5BEBCF60E1DF2A0BBFEB41433E3122B9
        at com.google.gwt.rpc.server.RpcServlet.findClientOracleData(RpcServlet.java:279)
        at com.google.gwt.rpc.server.RpcServlet.getClientOracle(RpcServlet.java:115)
        at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:194)
        at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
        at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
        at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
        at java.lang.Thread.run(Thread.java:679)

Comment 1 Vojtech Szocs 2012-01-09 14:31:58 UTC
How to reproduce this problem:
1. build oVirt, start JBoss, open WebAdmin in browser
2. stop JBoss, rebuild oVirt again, keep "old" WebAdmin opened in the browser
3. since JBoss was stopped, "old" WebAdmin will report this error message:
"Error: A Request to the Server failed with the following Status Code: 500"
4. start JBoss again, and watch the JBoss console, following exceptions appear:

ERROR [[/webadmin]] Exception while dispatching incoming RPC call
com.google.gwt.user.client.rpc.SerializationException: Could not find
ClientOracle data for permutation 9DA981D034209E3FF4BC2B631FDB3DA8

This is not a bug of WebAdmin. The problem is that "old" WebAdmin keeps trying
to reach JBoss backend, while WebAdmin server-side files were recompiled and
redeployed in the meantime. Simply close the "old" WebAdmin in your browser.

Explanation:

Each time you build oVirt, WebAdmin code is compiled into JavaScript by GWT
compiler. WebAdmin JavaScript is divided into several smaller permutations,
typically one per each supported browser. Each permutation has a strong name,
e.g. "9DA981D034209E3FF4BC2B631FDB3DA8".

WebAdmin server-side files are put to
/path-to-jboss/server/default/deploy/engine.ear/webadmin.war/webadmin, for
example: 9DA981D034209E3FF4BC2B631FDB3DA8.gwt.rpc

Since permutation strong name changes between GWT compilations, "old" WebAdmin
tries to reference server-side files that no longer exist.

Solution:

Just close the "old" WebAdmin in your browser after rebuilding/redeploying
oVirt, and open new browser tab (or window) requesting WebAdmin again.

Moving this to MODIFIED, rvaknin - please verify if the above solution works
for you.

Comment 2 Rami Vaknin 2012-01-10 05:18:15 UTC
Works for me, I needed to close the browser and clear its cache , tested on devel env, last commit's hash: da5f1e1391028f956d6f7040a1d6a1501f90d44c

Comment 3 Einav Cohen 2012-01-23 14:07:29 UTC
*** Bug 783985 has been marked as a duplicate of this bug. ***

Comment 4 Haim 2012-02-05 11:56:16 UTC
I still hit this issue; cache doesn't seem to be cleared. 


2012-02-05 06:51:29,622 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/webadmin]] (http--0.0.0.0-8080-2) Exception while dispatching incoming RPC call: com.google.gwt.user.client.rpc.SerializationException: C
ould not find ClientOracle data for permutation 343720C180478ED34BEDA7158D17C6DC
        at com.google.gwt.rpc.server.RpcServlet.findClientOracleData(RpcServlet.java:279) [gwt-servlet-2.3.0.jar:]
        at com.google.gwt.rpc.server.RpcServlet.getClientOracle(RpcServlet.java:115) [gwt-servlet-2.3.0.jar:]
        at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:194) [gwt-servlet-2.3.0.jar:]
        at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.3.0.jar:]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.3.Final.jar:]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.3.Final.jar:]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.3.Final.jar:]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.3.Final.jar:]
        at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:151) [jboss-as-web-7.1.0.Beta1b.jar:]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.3.Final.jar:]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.3.Final.jar:]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.3.Final.jar:]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:362) [jbossweb-7.0.3.Final.jar:]
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.3.Final.jar:]
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.3.Final.jar:]
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.3.Final.jar:]
        at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]

git hash: 9dacd053d6d011b374b8fc83b88f8e7de368aabf

Comment 5 Vojtech Szocs 2012-02-08 14:46:49 UTC
Hi, can you post exact steps to reproduce the issue?

The stack trace just says that server-side WebAdmin files (permutations) cannot be found by RPC servlet. This typically means that new WebAdmin version has been deployed on JBoss.

Try closing WebAdmin in your browser, then clear your browser cache, restart JBoss and request WebAdmin in browser again.

Comment 6 Haim 2012-02-12 08:05:05 UTC
(In reply to comment #5)
> Hi, can you post exact steps to reproduce the issue?
> 
> The stack trace just says that server-side WebAdmin files (permutations) cannot
> be found by RPC servlet. This typically means that new WebAdmin version has
> been deployed on JBoss.
> 
> Try closing WebAdmin in your browser, then clear your browser cache, restart
> JBoss and request WebAdmin in browser again.

- logged in to web-admin 
- stop service 
- recompile 
- start service again 
- try to relogin (during that time, browser is opened)

Comment 7 Vojtech Szocs 2012-02-13 14:11:58 UTC
I see. You should close WebAdmin tab in your browser before stopping JBoss service. WebAdmin periodically queries the backend when you are logged in. When you start the service again, "old" WebAdmin will try to query backend again.

WebAdmin and backend need to be in sync (that's what the RPC exception is about). When "old" WebAdmin queries the backend, "Could not find ClientOracle data" exceptions will occur since client and server are not in sync. It's not a bug of the application itself, it just means you should request WebAdmin in new tab each time you recompile/upgrade oVirt.

Comment 8 Itamar Heim 2012-02-14 04:17:08 UTC
shouldn't webadmin detect it is out of sync and go back to the login page for user to re-login?

Comment 9 Vojtech Szocs 2012-02-14 09:53:04 UTC
Well, what we can do is implement some detection logic (that runs periodically?) that checks if client and server are in sync, and if not, reload the whole web page to restart the application. But if the user does "yum upgrade ovirt", he knows that the backend will change, and he knows he needs to restart WebAdmin in his browser. It should be as simple as pressing F5 (reload page).

One way of dealing with out-of-sync-detection by application, is catching RPC exceptions and for certain type of exception, we'd know that its due to out-of-sync and we can refresh the page. But this approach just automates what users can do anyway since they know when the backend has changed.

Comment 10 Itamar Heim 2012-12-13 08:12:10 UTC
Closing old bugs. If this issue is still relevant/important in current version, please re-open the bug.


Note You need to log in before you can comment on or make changes to this bug.