Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1461329

Summary: HSMGetAllTasksStatusesVDS failed: Cannot create Logical Volume
Product: [oVirt] vdsm Reporter: Yanqiu Zhang <yanqzhan>
Component: GeneralAssignee: Ala Hino <ahino>
Status: CLOSED DUPLICATE QA Contact: Raz Tamir <ratamir>
Severity: high Docs Contact:
Priority: high    
Version: 4.19.18CC: ahino, amureini, bugs, chhu, dyuan, hhan, jiyan, michal.skrivanek, mzhan, xuzhang, yanqzhan
Target Milestone: ovirt-4.1.4Keywords: Regression
Target Release: ---Flags: rule-engine: ovirt-4.1+
rule-engine: blocker+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-06-15 12:18:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yanqiu Zhang 2017-06-14 08:48:00 UTC
Description:
On rhevm server with iscsi storage, "HSMGetAllTasksStatusesVDS failed: Cannot create Logical Volume" while live storage migration for vm

Versions:
*vdsm-4.19.18-1.el7ev.x86_64*
libvirt-3.2.0-9.el7.x86_64
qemu-kvm-rhev-2.9.0-10.el7.x86_64

How reproducible:
100%


Steps:
1. Prepare rhevm env
 1 host:hp
 2 iscsi storages:IS1,IS2
 1 vm:h

2.start the vm

3.select vm h, click on 'disk' tab, select the disk h_Disk1 and click on 'Move' to do live storage migration:
Operation failed with following errors:
(1)On rhevm server:
Jun 14, 2017 3:06:35 AM   VDSM hp command HSMGetAllTasksStatusesVDS failed: Cannot create Logical Volume
Jun 14, 2017 3:06:42 AM   User admin@internal-authz have failed to move disk h_Disk1 to domain IS1.
...
Jun 14, 2017 3:06:49 AM   VM h has been paused.
...
(2)vdsm.log on host:
2017-06-14 03:06:35,484-0400 ERROR (tasks/2) [storage.Volume] Failed to create volume /rhev/data-center/ad36c80c-ff51-464b-8927-887c70d4880e/49858257-ecc7-4ce7-86b4-6ed8410a9e17/images/ede0bc84-9c25-47c1-b884-3cb0e32b0f8a/5d2fef3d-ceb9-4876-9c2d-a06f2a2f5ac5: Cannot create Logical Volume: ('49858257-ecc7-4ce7-86b4-6ed8410a9e17', '5d2fef3d-ceb9-4876-9c2d-a06f2a2f5ac5') (volume:1070)
2017-06-14 03:06:35,484-0400 ERROR (tasks/2) [storage.Volume] Unexpected error (volume:1107)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/volume.py", line 1067, in create
    initialSize=initialSize)
  File "/usr/share/vdsm/storage/blockVolume.py", line 528, in _create
    initialTags=(sc.TAG_VOL_UNINIT,))
  File "/usr/share/vdsm/storage/lvm.py", line 1158, in createLV
    raise se.CannotCreateLogicalVolume(vgName, lvName)
CannotCreateLogicalVolume: Cannot create Logical Volume: ('49858257-ecc7-4ce7-86b4-6ed8410a9e17', '5d2fef3d-ceb9-4876-9c2d-a06f2a2f5ac5')
2017-06-14 03:06:35,485-0400 ERROR (tasks/2) [storage.Image] Unexpected error (image:467)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/image.py", line 450, in _createTargetImage
    srcVolUUID=volParams['parent'])
  File "/usr/share/vdsm/storage/sd.py", line 758, in createVolume
    initialSize=initialSize)
  File "/usr/share/vdsm/storage/volume.py", line 1067, in create
    initialSize=initialSize)
  File "/usr/share/vdsm/storage/blockVolume.py", line 528, in _create
    initialTags=(sc.TAG_VOL_UNINIT,))
  File "/usr/share/vdsm/storage/lvm.py", line 1158, in createLV
    raise se.CannotCreateLogicalVolume(vgName, lvName)
CannotCreateLogicalVolume: Cannot create Logical Volume: ('49858257-ecc7-4ce7-86b4-6ed8410a9e17', '5d2fef3d-ceb9-4876-9c2d-a06f2a2f5ac5')
2017-06-14 03:06:35,486-0400 ERROR (tasks/2) [storage.TaskManager.Task] (Task='0600a365-fa4f-4324-acfc-69d91b291324') Unexpected error (task:870)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 877, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/storage/task.py", line 333, in run
    return self.cmd(*self.argslist, **self.argsdict)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/share/vdsm/storage/sp.py", line 1712, in cloneImageStructure
    img.cloneStructure(sdUUID, imgUUID, dstSdUUID)
  File "/usr/share/vdsm/storage/image.py", line 676, in cloneStructure
    self._createTargetImage(sdCache.produce(dstSdUUID), sdUUID, imgUUID)
  File "/usr/share/vdsm/storage/image.py", line 450, in _createTargetImage
    srcVolUUID=volParams['parent'])
  File "/usr/share/vdsm/storage/sd.py", line 758, in createVolume
    initialSize=initialSize)
  File "/usr/share/vdsm/storage/volume.py", line 1067, in create
    initialSize=initialSize)
  File "/usr/share/vdsm/storage/blockVolume.py", line 528, in _create
    initialTags=(sc.TAG_VOL_UNINIT,))
  File "/usr/share/vdsm/storage/lvm.py", line 1158, in createLV
    raise se.CannotCreateLogicalVolume(vgName, lvName)
CannotCreateLogicalVolume: Cannot create Logical Volume: ('49858257-ecc7-4ce7-86b4-6ed8410a9e17', '5d2fef3d-ceb9-4876-9c2d-a06f2a2f5ac5')
2017-06-14 03:06:48,814-0400 ERROR (periodic/0) [virt.vm] (vmId='7947a8ba-8a9c-4588-9781-b67fd4dcf6aa') Improbable extension request for volume 7615613b-da59-4df0-b726-e9a7c03d3769 on domain d5a4bab3-315c-4814-8ffd-12997537d4c5, pausing the VM to avoid corruptions (capacity: 8589934592, allocated: 8589934592, physical: 1073741824, next physical size: 2147483648) (vm:873)
2017-06-14 03:07:40,919-0400 ERROR (jsonrpc/0) [storage.TaskManager.Task] (Task='aa5477bd-398d-4bf6-a891-977a380702d2') Unexpected error (task:870)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 877, in _run
    return fn(*args, **kargs)
  File "<string>", line 2, in getVolumeInfo
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method
    ret = func(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 3046, in getVolumeInfo
    volUUID=volUUID).getInfo()
  File "/usr/share/vdsm/storage/sd.py", line 743, in produceVolume
    volUUID)
  File "/usr/share/vdsm/storage/blockVolume.py", line 479, in __init__
    manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID)
  File "/usr/share/vdsm/storage/blockVolume.py", line 75, in __init__
    volUUID)
  File "/usr/share/vdsm/storage/volume.py", line 86, in __init__
    self.validate()
  File "/usr/share/vdsm/storage/blockVolume.py", line 168, in validate
    raise se.VolumeDoesNotExist(self.volUUID)
VolumeDoesNotExist: Volume does not exist: (u'7615613b-da59-4df0-b726-e9a7c03d3769',)
2017-06-14 03:07:40,920-0400 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH getVolumeInfo error=Volume does not exist: (u'7615613b-da59-4df0-b726-e9a7c03d3769',) (dispatcher:78)


Actual result
As in step3, "HSMGetAllTasksStatusesVDS failed: Cannot create Logical Volume" while live storage migration for vm with iscsi storage


Expected result
Should succeed to finish live storage migration for vm with iscsi storage without error.


Additional info:

Comment 2 Allon Mureinik 2017-06-15 12:13:29 UTC
Ala, doesn't this look like a dupe of something you're already working on?

Comment 3 Red Hat Bugzilla Rules Engine 2017-06-15 12:13:35 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 4 Allon Mureinik 2017-06-15 12:18:01 UTC

*** This bug has been marked as a duplicate of bug 1461468 ***

Comment 5 Ala Hino 2017-06-20 08:07:57 UTC
This bug closed as duplicate. Cancelling the needoinfo.