Bug 1245925 - [ovirt-webadmin] web admain is very slow almost impossible to work with
Summary: [ovirt-webadmin] web admain is very slow almost impossible to work with
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.6.0
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ---
: 3.6.0
Assignee: Sandro Bonazzola
QA Contact:
URL:
Whiteboard: integration
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-07-23 06:54 UTC by Israel Pinto
Modified: 2015-09-11 09:51 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-09-11 09:51:55 UTC
oVirt Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
engine log (2.46 MB, text/plain)
2015-07-23 12:20 UTC, Nisim Simsolo
no flags Details
server.log (227.37 KB, text/plain)
2015-07-23 12:20 UTC, Nisim Simsolo
no flags Details
After configure etc_hosts (229.50 KB, application/x-bzip)
2015-07-23 13:48 UTC, Israel Pinto
no flags Details
host_deploy_log (255.88 KB, application/x-bzip)
2015-08-03 12:00 UTC, Israel Pinto
no flags Details
sosreport-LogCollector (18.10 MB, application/x-xz)
2015-08-19 14:12 UTC, Nisim Simsolo
no flags Details

Description Israel Pinto 2015-07-23 06:54:31 UTC
Description of problem:
While using WebAdmain via FireFox,Chrome it response very slow  
sometimes stuck the browser, and we need to restart it.
It happened is every action: 
 1. Open dialog
 2. Switch between tabs
 3. Event tab
    

Version-Release number of selected component (if applicable):
oVirt Engine Version: 3.6.0-0.0.master.20150627185750.git6f063c1.el6 
Browser versions:
FireFox:38.0.1
Chrome:43.0.2357.125 


How reproducible:
All the time.

Comment 1 Oved Ourfali 2015-07-23 12:08:09 UTC
To get better understanding of the issue, and whether it is a UI or backend related, can you attach engine.log? server.log?

Comment 2 Nisim Simsolo 2015-07-23 12:20:07 UTC
Created attachment 1055298 [details]
engine log

Comment 3 Nisim Simsolo 2015-07-23 12:20:28 UTC
Created attachment 1055299 [details]
server.log

Comment 4 Israel Pinto 2015-07-23 12:22:40 UTC
Nisim added logs

Comment 5 Nisim Simsolo 2015-07-23 12:23:14 UTC
occurred also on my setup: 
oVirt Engine Version: 3.6.0-0.0.master.20150627185750.git6f063c1.el6
Severity changed to urgent since it's impossible to use UI even for basic actions. 
server.log and engine.log attached.

Comment 6 Oved Ourfali 2015-07-23 12:24:57 UTC
The build is there for a month, and first time we hear such an issue....
So, it might be an environment issue. Reducing urgency.

Comment 7 Oved Ourfali 2015-07-23 12:29:57 UTC
I see two issues in the server.log:
1. A JPA issue - we already addressed that. It shouldn't cause any performance issues. Just inconvenience in the log. 
2. UX issue (See below): - perhaps it is responsible for the issue.
Marking as UX for examination of the UX team. Einav?

If it is irrelevant as well then I suggest to test move to MODIFIED, and test on the next build, as these are the only two issues I see.


2015-07-22 12:16:43,506 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: WildFly 8.2.0.Final "Tweek" started in 40488ms - Started 1094 of 1197 services (176 services are lazy, passive or on-demand)
2015-07-22 12:18:15,928 WARNING [com.google.gwt.rpc.server.WebModePayloadSink] (default task-18) Skipped sending the field org.ovirt.engine.core.common.errors.VdcFault.privateDetails because it's unused in the client. It should either be changed to transient or removed.
2015-07-22 12:18:15,930 WARNING [com.google.gwt.rpc.server.WebModePayloadSink] (default task-18) Skipped sending the field org.ovirt.engine.core.common.errors.VdcFault.privateSessionID because it's unused in the client. It should either be changed to transient or removed.
2015-07-22 12:50:30,081 ERROR [io.undertow.servlet] (default task-12) Exception while dispatching incoming RPC call: com.google.gwt.user.client.rpc.RpcTokenException: Invalid RPC token (Invalid XSRF token)
	at org.ovirt.engine.ui.frontend.server.gwt.XsrfProtectedRpcServlet.validateXsrfToken(XsrfProtectedRpcServlet.java:31) [frontend.jar:]
	at org.ovirt.engine.ui.frontend.server.gwt.AbstractXsrfProtectedRpcServlet.onAfterRequestDeserialized(AbstractXsrfProtectedRpcServlet.java:57) [frontend.jar:]
	at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:171) [gwt-servlet.jar:]
	at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) [gwt-servlet.jar:]
	at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet.jar:]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
	at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) [frontend.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) [branding.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:65) [utils.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:74) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:132) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:90) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:73) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet.jar:1.1.5.Final]
	at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:51) [undertow-core.jar:1.1.5.Final]
	at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core.jar:1.1.5.Final]
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:248) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:77) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:167) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199) [undertow-core.jar:1.1.5.Final]
	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:766) [undertow-core.jar:1.1.5.Final]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_79]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_79]
	at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79]

2015-07-22 13:46:54,663 ERROR [io.undertow.servlet] (default task-4) Exception while dispatching incoming RPC call: com.google.gwt.user.client.rpc.RpcTokenException: Invalid RPC token (Invalid XSRF token)
	at org.ovirt.engine.ui.frontend.server.gwt.XsrfProtectedRpcServlet.validateXsrfToken(XsrfProtectedRpcServlet.java:31) [frontend.jar:]
	at org.ovirt.engine.ui.frontend.server.gwt.AbstractXsrfProtectedRpcServlet.onAfterRequestDeserialized(AbstractXsrfProtectedRpcServlet.java:57) [frontend.jar:]
	at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:171) [gwt-servlet.jar:]
	at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) [gwt-servlet.jar:]
	at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet.jar:]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]
	at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) [frontend.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) [branding.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:65) [utils.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:74) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:132) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:90) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:73) [aaa.jar:]
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet.jar:1.1.5.Final]
	at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:51) [undertow-core.jar:1.1.5.Final]
	at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core.jar:1.1.5.Final]
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:248) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:77) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:167) [undertow-servlet.jar:1.1.5.Final]
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199) [undertow-core.jar:1.1.5.Final]
	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:766) [undertow-core.jar:1.1.5.Final]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_79]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_79]
	at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79]

Comment 8 Einav Cohen 2015-07-23 12:38:54 UTC
assigning to Alexander for investigation.

Comment 9 Alexander Wels 2015-07-23 12:54:44 UTC
This is the culprit:

2015-07-22 12:16:08,411 WARN  [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) ARJUNA012210: Unable to use InetAddress.getLocalHost() to resolve address.

The engine cannot resolve itself. Either properly configure DNS or add the engine to /etc/hosts. Once you do that the engine will work fine.

Comment 11 Alexander Wels 2015-07-23 13:21:35 UTC
On the ENGINE box, in the /etc/hosts do you have line like this:

127.0.0.1               localhost.localdomain localhost <hostname>

Where <hostname> is the name you see when you type hostname on the command line? I can guarantee after seeing the warning in the log you don't.

I will let you close this, once you have properly configured your ENGINE /etc/hosts

Comment 12 Israel Pinto 2015-07-23 13:48:01 UTC
Created attachment 1055387 [details]
After configure etc_hosts

Comment 13 Israel Pinto 2015-07-23 13:49:33 UTC
I configure /etc/host like in comment 11.
And FireFox is working fine, But Chrome is stuck.
Added logs

Comment 14 Oved Ourfali 2015-07-23 13:56:21 UTC
(In reply to Israel Pinto from comment #13)
> I configure /etc/host like in comment 11.
> And FireFox is working fine, But Chrome is stuck.
> Added logs

Not a UI expert, but perhaps you need to clear your cache?

Comment 15 Alexander Wels 2015-07-23 14:04:39 UTC
Yes I would close and restart chrome. Also technically I believe Chrome is not a supported browser, but it usually works fine.

Comment 16 Einav Cohen 2015-07-23 19:57:15 UTC
Israel, please clear Chrome's cache and restart the Chrome browser (as instructed in Comment #14 , Comment #15) and report back the results when attempting to access the web-admin. Thanks.

Comment 17 Israel Pinto 2015-07-25 17:03:50 UTC
For Chrome: I clean the browser cache and restart it.
The performance improve.

About comment 11:
If we need to configure hostname in /etc/hosts it should be done by installation
- engine-setup, or at least written in release notes.

Comment 18 Alexander Wels 2015-07-27 12:15:32 UTC
No, the /etc/hosts thing is to allow it to work when you don't have a working DNS setup. In most cases there will be a working DNS and this will not be an issue, we definitely don't want to start writing to /etc/hosts and mess up the host name resolution that someone has configured in their environment.

Also I am 99% sure that when you run engine-setup, it displays a warning that it can't resolve the engine host name. Not sure what else you want us to do?

Closing again as its definitely not a bug.

Comment 19 Israel Pinto 2015-08-03 12:00:42 UTC
Created attachment 1058762 [details]
host_deploy_log

Comment 20 Einav Cohen 2015-08-03 12:22:35 UTC
I assume that this BZ has been re-opened since there is no warning in the engine-setup log about not being able to resolve the engine host name. 

To my understanding, it doesn't changes the fact that the engine host name is not resolvable. 
Therefore, if there is a bug somewhere, it is in the engine-setup that doesn't display/log a proper warning - this is not a web-admin / ux bug. 

Changing assignee/Whiteboard/component accordingly.

Comment 21 Israel Pinto 2015-08-03 13:28:43 UTC
I see that the FQDN is resolved in the engine-setup:

2015-07-20 07:35:43 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:218 DIALOG:SEND                 --== NETWORK CONFIGURATION ==--
2015-07-20 07:35:43 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:218 DIALOG:SEND
2015-07-20 07:35:43 DEBUG otopi.context context._executeMethod:141 Stage customization METHOD otopi.plugins.ovirt_engine_common.base.network.hostname.Plugin._customization
2015-07-20 07:35:43 DEBUG otopi.plugins.otopi.dialog.human human.queryString:156 query OVESETUP_NETWORK_FQDN_this
2015-07-20 07:35:43 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:218 DIALOG:SEND                 Host fully qualified DNS name of this server [vm-161-169.scl.lab.tlv.redhat.com]:
2015-07-20 07:35:45 DEBUG otopi.plugins.ovirt_engine_common.base.network.hostname hostname._validateFQDNresolvability:195 vm-161-169.scl.lab.tlv.redhat.com resolves to: set(['10.35.161.169'])
(attached log)
The log is before we add the FQDN to /etc/hosts.
Is it mean that the DNS is OK?

Comment 22 Einav Cohen 2015-08-03 17:46:16 UTC
(In reply to Israel Pinto from comment #21)
> I see that the FQDN is resolved in the engine-setup:
> 
> 2015-07-20 07:35:43 DEBUG otopi.plugins.otopi.dialog.human
> dialog.__logString:218 DIALOG:SEND                 --== NETWORK
> CONFIGURATION ==--
> 2015-07-20 07:35:43 DEBUG otopi.plugins.otopi.dialog.human
> dialog.__logString:218 DIALOG:SEND
> 2015-07-20 07:35:43 DEBUG otopi.context context._executeMethod:141 Stage
> customization METHOD
> otopi.plugins.ovirt_engine_common.base.network.hostname.Plugin._customization
> 2015-07-20 07:35:43 DEBUG otopi.plugins.otopi.dialog.human
> human.queryString:156 query OVESETUP_NETWORK_FQDN_this
> 2015-07-20 07:35:43 DEBUG otopi.plugins.otopi.dialog.human
> dialog.__logString:218 DIALOG:SEND                 Host fully qualified DNS
> name of this server [vm-161-169.scl.lab.tlv.redhat.com]:
> 2015-07-20 07:35:45 DEBUG
> otopi.plugins.ovirt_engine_common.base.network.hostname
> hostname._validateFQDNresolvability:195 vm-161-169.scl.lab.tlv.redhat.com
> resolves to: set(['10.35.161.169'])
> (attached log)
> The log is before we add the FQDN to /etc/hosts.
> Is it mean that the DNS is OK?

I have no idea; if this is correct, then it may contradict Comment #9; note that the log above and the log quoted in Comment #9 are 2 days apart, so not really sure what's going on here. 
@Alexander - any thoughts?

Comment 23 Alexander Wels 2015-08-05 18:11:53 UTC
I also have no idea if this is correct or not. From the engine-setup log it looks correct, but when I looked at the engine log it definitely had the warning in there it couldn't resolve itself, so the results there don't match.

In any case, the webadmin can't do anything about it, so maybe move this to the appropriate team?

Comment 24 Einav Cohen 2015-08-05 18:29:34 UTC
(In reply to Alexander Wels from comment #23)
> In any case, the webadmin can't do anything about it, so maybe move this to
> the appropriate team?

already set on the engine-setup component, whiteboard is 'integration', assignee was reset. thanks, Alexander.

Comment 25 Gil Klein 2015-08-06 11:36:39 UTC
I'm raising the severity cause I've got lot of report for different QE teams about this slowness.

Comment 26 Sandro Bonazzola 2015-08-12 11:11:34 UTC
Lev, please try to reproduce.

Comment 27 Sandro Bonazzola 2015-08-12 14:26:31 UTC
I'm moving back to infra instead of closing as not a bug.

Issue is that the host has been configured with hostname RHEL7.1Server while the dhcp assigned vm-161-169.scl.lab.tlv.redhat.com.
While engine-setup works with vm-161-169.scl.lab.tlv.redhat.com engine fails to resolve RHEL7.1Server, at least if I've understood correctly the engine logs.

Please provide a full sos report if you move again to integration.

Comment 28 Oved Ourfali 2015-08-12 18:17:34 UTC
(In reply to Sandro Bonazzola from comment #27)
> I'm moving back to infra instead of closing as not a bug.
> 
> Issue is that the host has been configured with hostname RHEL7.1Server while
> the dhcp assigned vm-161-169.scl.lab.tlv.redhat.com.
> While engine-setup works with vm-161-169.scl.lab.tlv.redhat.com engine fails
> to resolve RHEL7.1Server, at least if I've understood correctly the engine
> logs.
> 
> Please provide a full sos report if you move again to integration.

So according to that the issue isn't infra, and you're missing logs. 
I suggest to test that again with the latest build (to be released this week) and reopen if the issue still exists. Tens of people tested it on the test day, and there were no performance issues reported, so obviously it is a specific issue, or not reproduced anymore.

Comment 29 Gil Klein 2015-08-19 09:17:05 UTC
(In reply to Sandro Bonazzola from comment #27)
> I'm moving back to infra instead of closing as not a bug.
> 
> Issue is that the host has been configured with hostname RHEL7.1Server while
> the dhcp assigned vm-161-169.scl.lab.tlv.redhat.com.
> While engine-setup works with vm-161-169.scl.lab.tlv.redhat.com engine fails
> to resolve RHEL7.1Server, at least if I've understood correctly the engine
> logs.
Snadro, if this is the case, shouldn't we warn the user during the engine-setup phase? 

The impact on the UI performance is very bad, so IMHO we should at least warn, and allow the use to fix it before he deploy/upgrade the system.

> 
> Please provide a full sos report if you move again to integration.

Comment 30 Sandro Bonazzola 2015-08-19 12:55:57 UTC
Moving back to QA as per comment #28.
Gil, during setup phase FQDN was correctly resolved, it already warns if something is wrong there.

Comment 31 Nisim Simsolo 2015-08-19 14:10:13 UTC
Reproduced on my setup also:
engine: 
3.6.0-0.11.master.el6
host: 
vdsm-4.17.2-1.el7ev.noarch
sanlock-3.2.4-1.el7.x86_64
qemu-kvm-rhev-2.3.0-18.el7.x86_64
libvirt-client-1.2.17-4.el7.x86_64

LogCollector attached.

Comment 32 Nisim Simsolo 2015-08-19 14:12:25 UTC
Created attachment 1064879 [details]
sosreport-LogCollector

Comment 33 Yaniv Lavi 2015-09-08 11:09:30 UTC
I the cause is unstable networking that that is the problem to be resolved, not oVirt working around this. Removing the blocker.

Comment 34 Sandro Bonazzola 2015-09-11 09:51:55 UTC
(In reply to Nisim Simsolo from comment #32)
> Created attachment 1064879 [details]
> sosreport-LogCollector

engine-setup configured fqdn is: dhcp163-68.scl.lab.tlv.redhat.com
hostname command outputs dhcp163-68.scl.lab.tlv.redhat.com

/etc/hosts has been set in order to resolve to a different IP than the one associated to the FQDN:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 dhcp163-68.scl.lab.tlv.redhat.com

while engine-setup resolved:
dhcp163-68.scl.lab.tlv.redhat.com. 299 IN A	10.35.163.68

Please avoid to mess with address resolution while testing.

above line should be 
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

without the hostname added to it.


Note You need to log in before you can comment on or make changes to this bug.