Bug 1311465 - HotPlugDisk fails with "Bad volume specification" error [NEEDINFO]
HotPlugDisk fails with "Bad volume specification" error
Status: CLOSED INSUFFICIENT_DATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.5.7
x86_64 Linux
medium Severity high
: ovirt-3.6.5
: ---
Assigned To: Amit Aviram
Aharon Canan
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-24 05:12 EST by Amit Aviram
Modified: 2016-03-21 08:20 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-21 08:20:00 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
aaviram: needinfo? (dev-unix-virtualization)
tnisan: needinfo? (dev-unix-virtualization)


Attachments (Terms of Use)
VDSM & Engine logs (2.13 MB, application/zip)
2016-02-24 05:12 EST, Amit Aviram
no flags Details

  None (edit)
Description Amit Aviram 2016-02-24 05:12:45 EST
Created attachment 1130162 [details]
VDSM & Engine logs

Description of problem:
This bug was introduced by dev-unix-virtualization@commvault.com, in Bug 1249717.

Quoting from comment 9 in Bug 1249717:
Hi,
We recently hit this issue. RHEVM version is 3.5.7. We are trying to activate the disk and getting below response. I have attached the VDSM and Engine logs.

484 20c 02/17 20:03:02 108261 CVLibCurl::CVLibCurlSendHttpReq() - curl-err:[No error] http-resp:[400] url:[https://172.16.61.116/api/vms/2CBCD16F-6BC8-40FA-9722-291164CFD7B6/disks/023a8251-4109-493c-a572-ae0684b0a07b/activate] server-resp:[<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<action>
    <async>false</async>
    <detach>false</detach>
    <status>
        <state>failed</state>
    </status>
    <fault>
        <reason>Operation Failed</reason>
        <detail>[Unexpected exception]</detail>
    </fault>
</action>
] resp-size:[299]
 

Version-Release number of selected component (if applicable):
3.5.7
Comment 1 Amit Aviram 2016-02-24 05:20:09 EST
dev-unix-virtualization@commvault, can you please add the following info, if you have it?

How reproducible:


Steps to Reproduce:
1.
2.
3.

Thank you!
Comment 3 Amit Aviram 2016-02-24 08:24:08 EST
The problem in this flow is that preparing the volume path before plugging the disk fails on timeout. 
So we need to do two things here:

1. Understand what causes the timeout, and fix it if it's a bug.
2. Add a proper error message in case of prepare volume path timeout
Comment 4 Amit Aviram 2016-02-25 07:20:30 EST
dev-unix-virtualization@commvault, please provide full engine+vdsm logs. 
Looking at the engine's log- The error we encounter happens also at 2016-02-17 13:33:21, 2016-02-16 16:13:09, 2016-02-16 14:26:28, and maybe before.

Note, that this is not directly related to your hotplug failure, but something in the system which is faulted and makes the hotplug fail among other actions you are performing.
Comment 5 Amit Aviram 2016-02-25 10:35:07 EST
Seems like this is a scale issue. please also provide psql dump, to see how many items this domain contains. 
Did you try to configure VDSM to run with a single core, using the VDSM's cpu_affinity configuration? it can possibly resolve scale issues in case your host is heavily loaded.
Comment 6 Amit Aviram 2016-03-02 04:21:40 EST
dev-unix-virtualization@commvault, Can you please also elaborate on the environment you are using? is your storage/connectivity heavily loaded?
Comment 7 Amit Aviram 2016-03-09 08:39:00 EST
As the error message in this flow is vague, BUG 1316128 opened to make better error messages that explain such cases more gracefully in vm related flows like hotplug/ unplug.
Comment 8 Yaniv Lavi (Dary) 2016-03-21 08:20:00 EDT
Please reopen if you can provide the needed info.

Note You need to log in before you can comment on or make changes to this bug.