Bug 1154393 - [scale] REST API: VM started with error "Failed to run VM..." in spite of that VM started successfully
Summary: [scale] REST API: VM started with error "Failed to run VM..." in spite of tha...
Keywords:
Status: CLOSED DUPLICATE of bug 1148583
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.5.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: 3.5.0
Assignee: Arik
QA Contact:
URL:
Whiteboard: virt
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-10-19 12:32 UTC by Yuri Obshansky
Modified: 2014-10-26 09:32 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-10-26 09:32:12 UTC
oVirt Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
vdsm.log (2.28 MB, application/x-gzip)
2014-10-21 12:55 UTC, Yuri Obshansky
no flags Details
engine.log (1.06 MB, application/x-gzip)
2014-10-21 12:55 UTC, Yuri Obshansky
no flags Details

Description Yuri Obshansky 2014-10-19 12:32:58 UTC
Description of problem:
During REST API User Portal performance test of 50 concurrent threads on 50 VMs detected errors >> "Failed to run VM scale-50 (User: scale-50)"  in spite of that VM started successfully.
As results performance degradation.
VM shutdowm response time 90% line in RHEV-M 3.4 = 5749 ms compare to RHEV-M 3.5 = 241912 ms.
Engine.log errors:
2014-10-15 12:46:58,027 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (ajp-/127.0.0.1:8702-9) [6c470701] Command CreateVDSCommand(HostName = HOST-REAL-1, HostId = 932aaea7-5e19-473f-964f-9d26b71adf05, vmId=8c54a225-048e-49f6-9415-fa0f2b3cd29d, vm=VM [scale-1]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues'
2014-10-15 12:46:58,027 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (ajp-/127.0.0.1:8702-9) [6c470701] FINISH, CreateVDSCommand, log id: 76454652
2014-10-15 12:46:58,027 ERROR [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (ajp-/127.0.0.1:8702-9) [6c470701] VDS::create Failed creating vm scale-1 in vds = 932aaea7-5e19-473f-964f-9d26b71adf05 : HOST-REAL-1 error = org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues'
2014-10-15 12:46:58,027 INFO  [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (ajp-/127.0.0.1:8702-9) [6c470701] FINISH, CreateVmVDSCommand, return: Down, log id: 6c18eef4
2014-10-15 12:46:58,027 INFO  [org.ovirt.engine.core.bll.RunVmOnceCommand] (ajp-/127.0.0.1:8702-9) [6c470701] Lock freed to object EngineLock [exclusiveLocks= key: 8c54a225-048e-49f6-9415-fa0f2b3cd29d value: VM
, sharedLocks= ]
....................
2014-10-15 12:46:58,027 ERROR [org.ovirt.engine.core.bll.RunVmOnceCommand] (ajp-/127.0.0.1:8702-9) [6c470701] Command org.ovirt.engine.core.bll.RunVmOnceCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' (Failed with error VDS_NETWORK_ERROR and code 5022)

Version-Release number of selected component (if applicable):
RHEV-M: 3.5.0-0.14.beta
RHEL: 6Server - 6.5.0.1.el6
Kernel Version: 2.6.32 - 431.29.2.el6.x86_64
KVM Version: 0.12.1.2 - 2.415.el6_5.10
LIBVIRT Version: libvirt-0.10.2-29.el6_5.11
VDSM Version: vdsm-4.16.6-1.el6ev

How reproducible:
Run automatic test which shutdowm/start VMs using user portal REST API with load of 50 concurrent threads.


Steps to Reproduce:
1.
2.
3.

Actual results:
Errors
Performance degradation

Expected results:
No errors
No performance degradation


Additional info:

Comment 1 Omer Frenkel 2014-10-20 10:39:49 UTC
please attach vdsm log
might be duplicate of bug 1148583

Comment 2 Yuri Obshansky 2014-10-21 12:55:19 UTC
Created attachment 948948 [details]
vdsm.log

Comment 3 Yuri Obshansky 2014-10-21 12:55:43 UTC
Created attachment 948949 [details]
engine.log

Comment 4 Omer Frenkel 2014-10-26 09:32:12 UTC

*** This bug has been marked as a duplicate of bug 1148583 ***


Note You need to log in before you can comment on or make changes to this bug.