Bug 1269906 - Error creating a new volume during creation of vm pool
Error creating a new volume during creation of vm pool
Product: vdsm
Classification: oVirt
Component: General (Show other bugs)
Unspecified Unspecified
unspecified Severity high (vote)
: ovirt-3.6.1
: 4.17.10
Assigned To: Tal Nisan
Aharon Canan
: 1269903 1269905 1269911 (view as bug list)
Depends On:
  Show dependency treegraph
Reported: 2015-10-08 09:12 EDT by Eyal Shenitzky
Modified: 2016-02-10 13:48 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2015-11-08 08:02:37 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
amureini: ovirt‑3.6.z?
rule-engine: planning_ack?
eshenitz: devel_ack?
rule-engine: testing_ack?

Attachments (Terms of Use)

  None (edit)
Description Eyal Shenitzky 2015-10-08 09:12:30 EDT
Description of problem:
After creation of template from vm with 2 disk attached (first is nfs disk thin-provision, the second is iscsi disk pre-allocated)
when trying to create a pool of vms (10) from the template an
"Error creating a new volume" massage received

from engine log:
2015-10-08 11:47:16,626 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) [5f1601f0] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Messa
ge: VDSM H_1 command failed: Error creating a new volume
2015-10-08 11:47:16,721 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-69) [5f1601f0] BaseAsyncTask::onTaskEndSuccess: Task '6033427e-6b79-4c41-9318-24d591a66169' (Parent Comma
nd 'AddVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully.
2015-10-08 11:47:16,723 INFO  [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-69) [5f1601f0] Task with DB Task ID '4bf3991c-1f04-4085-beca-5d001e3098e1' and VDSM Task ID '38b2c158
-a673-4ff6-ac07-9f54784e66d9' is in state Polling. End action for command 5dede124-25af-4000-ae65-c5ca4fcc28e0 will proceed when all the entity's tasks are completed.
2015-10-08 11:47:16,725 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-69) [5f1601f0] SPMAsyncTask::PollTask: Polling task '38b2c158-a673-4ff6-ac07-9f54784e66d9' (Parent Comman
d 'AddVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanFailure'.
2015-10-08 11:47:16,879 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-69) [5f1601f0] BaseAsyncTask::logEndTaskFailure: Task '38b2c158-a673-4ff6-ac07-9f54784e66d9' (Parent Comm
and 'AddVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure:
-- Result: 'cleanFailure'
-- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Error creating a new volume, code = 205',
-- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Error creating a new volume, code = 205'

from vdsm log:
d7877ac6-84c4-4990-aa96-96bc646c7551::ERROR::2015-10-08 12:03:09,018::volume::476::Storage.Volume::(create) Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/volume.py", line 470, in create
    cls.newVolumeLease(metaId, sdUUID, volUUID)
  File "/usr/share/vdsm/storage/blockVolume.py", line 631, in newVolumeLease
    sanlock.init_resource(sdUUID, volUUID, [(leasePath, leaseOffset)])
SanlockException: (-202, 'Sanlock resource init failure', 'Sanlock exception')
d7877ac6-84c4-4990-aa96-96bc646c7551::DEBUG::2015-10-08 12:03:09,019::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'c2c7bde5-2e16-4a8b-8d56-c876b103ff8b_imageNS.433cf15d-411d-4146-a0e9-150bf072718d'
d7877ac6-84c4-4990-aa96-96bc646c7551::DEBUG::2015-10-08 12:03:09,020::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'c2c7bde5-2e16-4a8b-8d56-c876b103ff8b_imageNS.433cf15d-411d-4146-a0e9-150bf072718d' (0 active users)
d7877ac6-84c4-4990-aa96-96bc646c7551::DEBUG::2015-10-08 12:03:09,020::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'c2c7bde5-2e16-4a8b-8d56-c876b103ff8b_imageNS.433cf15d-411d-4146-a0e9-150bf072718d' is free, finding out if anyone is waiting for it.
d7877ac6-84c4-4990-aa96-96bc646c7551::DEBUG::2015-10-08 12:03:09,020::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'c2c7bde5-2e16-4a8b-8d56-c876b103ff8b_imageNS.433cf15d-411d-4146-a0e9-150bf072718d', Clearing records.
d7877ac6-84c4-4990-aa96-96bc646c7551::ERROR::2015-10-08 12:03:09,021::task::866::Storage.TaskManager.Task::(_setError) Task=`d7877ac6-84c4-4990-aa96-96bc646c7551`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/storage/task.py", line 332, in run
    return self.cmd(*self.argslist, **self.argsdict)
  File "/usr/share/vdsm/storage/securable.py", line 77, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/share/vdsm/storage/sp.py", line 1867, in createVolume
    desc=desc, srcImgUUID=srcImgUUID, srcVolUUID=srcVolUUID)
  File "/usr/share/vdsm/storage/sd.py", line 487, in createVolume
    preallocate, diskType, volUUID, desc, srcImgUUID, srcVolUUID)
  File "/usr/share/vdsm/storage/volume.py", line 478, in create
    (volUUID, e))
VolumeCreationError: Error creating a new volume: (u"Volume creation 1a2ebe61-4f58-4ffc-8359-dfd6b9307a06 failed: (-202, 'Sanlock resource init failure', 'Sanlock exception')",)
d7877ac6-84c4-4990-aa96-96bc646c7551::DEBUG::2015-10-08 12:03:09,021::task::885::Storage.TaskManager.Task::(_run) Task=`d7877ac6-84c4-4990-aa96-96bc646c7551`::Task._run: d7877ac6-84c4-4990-aa96-96bc646c7551 () {} failed - stopping task

Version-Release number of selected component (if applicable):
engine version:3.6.0-0.12.master.el6
vdsm version:

How reproducible:
some times

Steps to Reproduce:

1.Create vm with os installed
2.Make a template to the vm
3.Create vm pool of 10 vms

Actual results:
Not all vms created successfully 

Expected results:
all vms created successfully

Additional info:
Comment 1 Aharon Canan 2015-10-08 11:05:31 EDT
*** Bug 1269903 has been marked as a duplicate of this bug. ***
Comment 2 Aharon Canan 2015-10-08 11:05:42 EDT
*** Bug 1269905 has been marked as a duplicate of this bug. ***
Comment 3 Allon Mureinik 2015-10-11 08:13:08 EDT
*** Bug 1269911 has been marked as a duplicate of this bug. ***
Comment 4 Allon Mureinik 2015-10-11 08:14:51 EDT
Thanks for reporting this bug.

Please attach the full logs and version information for vdsm* and sanlock*.
Comment 5 Red Hat Bugzilla Rules Engine 2015-10-19 07:01:34 EDT
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Comment 6 Yaniv Lavi 2015-10-29 08:39:00 EDT
In oVirt testing is done on single release by default. Therefore I'm removing the 4.0 flag. If you think this bug must be tested in 4.0 as well, please re-add the flag. Please note we might not have testing resources to handle the 4.0 clone.
Comment 7 Eyal Shenitzky 2015-11-08 08:02:37 EST
this bug is not longer reproducible.
closing bug (reopen in case it happens again)

Note You need to log in before you can comment on or make changes to this bug.