Hide Forgot
Created attachment 1130162 [details] VDSM & Engine logs Description of problem: This bug was introduced by dev-unix-virtualization, in Bug 1249717. Quoting from comment 9 in Bug 1249717: Hi, We recently hit this issue. RHEVM version is 3.5.7. We are trying to activate the disk and getting below response. I have attached the VDSM and Engine logs. 484 20c 02/17 20:03:02 108261 CVLibCurl::CVLibCurlSendHttpReq() - curl-err:[No error] http-resp:[400] url:[https://172.16.61.116/api/vms/2CBCD16F-6BC8-40FA-9722-291164CFD7B6/disks/023a8251-4109-493c-a572-ae0684b0a07b/activate] server-resp:[<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <action> <async>false</async> <detach>false</detach> <status> <state>failed</state> </status> <fault> <reason>Operation Failed</reason> <detail>[Unexpected exception]</detail> </fault> </action> ] resp-size:[299] Version-Release number of selected component (if applicable): 3.5.7
dev-unix-virtualization@commvault, can you please add the following info, if you have it? How reproducible: Steps to Reproduce: 1. 2. 3. Thank you!
The problem in this flow is that preparing the volume path before plugging the disk fails on timeout. So we need to do two things here: 1. Understand what causes the timeout, and fix it if it's a bug. 2. Add a proper error message in case of prepare volume path timeout
dev-unix-virtualization@commvault, please provide full engine+vdsm logs. Looking at the engine's log- The error we encounter happens also at 2016-02-17 13:33:21, 2016-02-16 16:13:09, 2016-02-16 14:26:28, and maybe before. Note, that this is not directly related to your hotplug failure, but something in the system which is faulted and makes the hotplug fail among other actions you are performing.
Seems like this is a scale issue. please also provide psql dump, to see how many items this domain contains. Did you try to configure VDSM to run with a single core, using the VDSM's cpu_affinity configuration? it can possibly resolve scale issues in case your host is heavily loaded.
dev-unix-virtualization@commvault, Can you please also elaborate on the environment you are using? is your storage/connectivity heavily loaded?
As the error message in this flow is vague, BUG 1316128 opened to make better error messages that explain such cases more gracefully in vm related flows like hotplug/ unplug.
Please reopen if you can provide the needed info.