Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1107727

Summary: NullPointerException raised while perform REST API request api/vms/***/applications for VM under load
Product: Red Hat Enterprise Virtualization Manager Reporter: Yuri Obshansky <yobshans>
Component: ovirt-engine-restapiAssignee: Juan Hernández <juan.hernandez>
Status: CLOSED CURRENTRELEASE QA Contact: Shai Revivo <srevivo>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.4.0CC: acathrow, bazulay, gklein, iheim, oramraz, pstehlik, Rhev-m-bugs, yeylon
Target Milestone: ---Keywords: Triaged
Target Release: 3.5.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: infra
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-07-08 18:51:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Engine.log
none
Server.log none

Description Yuri Obshansky 2014-06-10 13:31:55 UTC 
Description of problem:
NullPointerException raised while I execute REST API calls ->
api/vms/......../applications for VMs which have installed applications
and under load (200 concurrent threads).
It didn't happened when I perform single REST API call to the same VM w/o load.

Version-Release number of selected component (if applicable):
RHEVM: 3.4.0-0.16.rc.el6ev

How reproducible:
100% reproducible

Steps to Reproduce:
1. Run load test RHEVM_3_4_PHX_USER_PORTAL_FLOW.jmx using JMeter
200 threads for 200 VMs and 200 users
2. Request looks like: GET
https://*.*.*.*/api/vms/5c4b79a1-d032-4985-b8eb-c242020f713f/applications 


Actual results:
Response:
<html><head><title>JBoss Web/7.3.0.Final-redhat-2 - JBWEB000064: Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>JBWEB000065: HTTP Status 500 - java.lang.NullPointerException</h1><HR size="1" noshade="noshade"><p><b>JBWEB000309: type</b> JBWEB000066: Exception report</p><p><b>JBWEB000068: message</b> <u>java.lang.NullPointerException</u></p><p><b>JBWEB000069: description</b> <u>JBWEB000145: The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>JBWEB000070: exception</b> <pre>org.jboss.resteasy.spi.UnhandledException: java.lang.NullPointerException
org.jboss.resteasy.core.SynchronousDispatcher.handleApplicationException(SynchronousDispatcher.java:365)
org.jboss.resteasy.core.SynchronousDispatcher.handleException(SynchronousDispatcher.java:233)
org.jboss.resteasy.core.SynchronousDispatcher.handleInvokerException(SynchronousDispatcher.java:209)
org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:557)
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524)
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126)
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
org.ovirt.engine.core.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:80)
</pre></p><p><b>JBWEB000071: root cause</b> <pre>java.lang.NullPointerException
org.ovirt.engine.api.restapi.resource.BackendVmApplicationsResource.addParents(BackendVmApplicationsResource.java:40)
org.ovirt.engine.api.restapi.resource.BackendVmApplicationsResource.addParents(BackendVmApplicationsResource.java:12)
org.ovirt.engine.api.restapi.resource.AbstractBackendResource.addLinks(AbstractBackendResource.java:226)
org.ovirt.engine.api.restapi.resource.AbstractBackendResource.addLinks(AbstractBackendResource.java:218)
org.ovirt.engine.api.restapi.resource.BackendVmApplicationsResource.list(BackendVmApplicationsResource.java:28)
    sun.reflect.GeneratedMethodAccessor236.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.lang.reflect.Method.invoke(Method.java:606)
org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167)
org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269)
org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227)
org.jboss.resteasy.core.ResourceLocator.invokeOnTargetObject(ResourceLocator.java:159)
org.jboss.resteasy.core.ResourceLocator.invoke(ResourceLocator.java:107)
org.jboss.resteasy.core.ResourceLocator.invokeOnTargetObject(ResourceLocator.java:154)
org.jboss.resteasy.core.ResourceLocator.invoke(ResourceLocator.java:92)
org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:542)
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524)
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126)
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
org.ovirt.engine.core.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:80)
</pre></p><p><b>JBWEB000072: note</b> <u>JBWEB000073: The full stack trace of the root cause is available in the JBoss Web/7.3.0.Final-redhat-2 logs.</u></p><HR size="1" noshade="noshade"><h3>JBoss Web/7.3.0.Final-redhat-2</h3></body></html> 

Expected results:
No error
List of installed applications: Kernel and guest-agent

Additional info:
Comment 1 Yuri Obshansky 2014-06-10 13:32:33 UTC 
Created attachment 907262 [details]
Engine.log
Comment 2 Yuri Obshansky 2014-06-10 13:32:59 UTC 
Created attachment 907263 [details]
Server.log
Comment 3 Juan Hernández 2014-07-08 18:51:15 UTC 
The only way this can happen is if the list of applications of the VM contains empty strings. In that case the RESTAPI will try to convert those empty strings to dummy UUIDs, and that will trigger a division by zero in the corresponding constructor of the Guid class. The solution to that is to avoid considering empty strings as valid application names. This was already done as part of the fix for bug 1092380, and included in 3.4.0-17. This bug was reported with 3.4.0-0.16.rc.el6ev, so I'm closing it fixed in the current release.