Bug 1441706

Summary: [engine-backend] Block storage domain creation fails: CreateVG deviceList is passed with duplicated PV names
Product: [oVirt] ovirt-engine Reporter: Elad <ebenahar>
Component: Backend.CoreAssignee: Maor <mlipchuk>
Status: CLOSED CURRENTRELEASE QA Contact: Elad <ebenahar>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: futureCC: amureini, bugs, ebenahar, lveyde, mavital, tnisan
Target Milestone: ovirt-4.1.2Keywords: Regression
Target Release: 4.1.2Flags: rule-engine: ovirt-4.1+
rule-engine: blocker+
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-23 08:12:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine.log and vdsm.log none

Description Elad 2017-04-12 14:01:34 UTC
Created attachment 1271164 [details]
engine.log and vdsm.log

Description of problem:
iSCSI storage domain creation fails in case the initiator is connected to the LUN through multiple targets. This happens because engine sends to vdsm multiple PVs, with the same PV name, according to the targets number in CreateVG.

Version-Release number of selected component (if applicable):
ovirt-engine-4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
vdsm-4.20.0-612.git378d837.el7.centos.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create an iSCSI storage domain with a single LUN that is exposed through multiple iSCSI targets


Actual results:

CreateVG with duplicated PV names:

ameters:{runAsync='true', hostId='b2861c21-5218-4500-85ec-b8290f0b1aba', storageDomainId='5cd63cd6-3e51-461f-b0b8-b521ef04125b', deviceList='[3514f0c5a5160058c, 3514f0c5a5160058c, 3514f0c5a5
160058c, 3514f0c5a5160058c]', force='false'}), log id: 5679430b

createVG fails in vdsm:

2017-04-12 15:59:11,604+0300 INFO  (jsonrpc/1) [dispatcher] Run and protect: createVG(vgname='5cd63cd6-3e51-461f-b0b8-b521ef04125b', devlist=['3514f0c5a5160058c', '3514f0c5a5160058c', '3514f0c5a5160058c', '3514f0c5a5160058c'], force=False, options=None) (logUtils:51)
2017-04-12 15:59:11,737+0300 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:570)
2017-04-12 15:59:11,964+0300 ERROR (jsonrpc/1) [storage.TaskManager.Task] (Task='5aab3164-d549-48d8-b5ad-75af8cd17d67') Unexpected error (task:871)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 878, in _run
    return fn(*args, **kargs)
  File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 52, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 2120, in createVG
    (force.capitalize() == "True")))
  File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 1004, in createVG
    raise se.VolumeGroupCreateError(vgName, pvs)
VolumeGroupCreateError: Cannot create Volume Group: "vgname=5cd63cd6-3e51-461f-b0b8-b521ef04125b, devname=['/dev/mapper/3514f0c5a5160058c', '/dev/mapper/3514f0c5a5160058c', '/dev/mapper/3514f0c5a5160058c', '/dev/mapper/3514f0c5a5160058c']"


Expected results:
CreateVG should be sent with the real amount of PVs.

Additional info:
engine.log and vdsm.log


CreateVG in 4.1 with LUN connected through multiple targets for comparison:

2017-04-12 16:26:22,055+03 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-14) [2d20f467] START, CreateVGVDSCommand(HostName = host_mixed_3, CreateVGVDSCommandParameters:{runAsync='true', hostId='9992b4ec-1c5c-4528-8be0-506422bec790', storageDomainId='89b8bacf-5f6e-405f-9ab9-60845678aff5', deviceList='[3514f0c5a516008d8]', force='false'}), log id: 5309b1ad

Comment 1 Red Hat Bugzilla Rules Engine 2017-04-12 14:13:16 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 2 Allon Mureinik 2017-04-12 14:39:08 UTC
(In reply to Elad from comment #0)
> Steps to Reproduce:
> 1. Create an iSCSI storage domain with a single LUN that is exposed through
> multiple iSCSI targets

How is this done? Via the webadmin GUI? REST API? Does it matter?

Comment 3 Elad 2017-04-12 18:37:43 UTC
(In reply to Allon Mureinik from comment #2)

> How is this done? Via the webadmin GUI? REST API? Does it matter?
Reproduced only via Webadmin. 
Tested with REST API, the following works well, the PV list contains only one PV as requested:


<storage_domain>
  <name>iscsi_1</name>
  <type>data</type>
<storage>
<type>iscsi</type>
<logical_units>
<logical_unit id="3514f0c5a5160058c"/>
</logical_units>
</storage>
  <host>
    <name>host1</name>
  </host>
</storage_domain>




2017-04-12 21:34:51,111+03 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-6) [6d9586b8] START, CreateVGVDSCommand(HostName = host1, CreateVGVDSCommandPara
meters:{runAsync='true', hostId='b2861c21-5218-4500-85ec-b8290f0b1aba', storageDomainId='68f0c68d-1058-41f3-9124-b2e7988016f4', deviceList='[3514f0c5a5160058c]', force='false'}), log id: 44d
3eabb

Comment 4 Elad 2017-05-04 08:32:09 UTC
CreateVG is sent with the right amount of devices in deviceList when creating iSCSI domain via Webadmin and the devices are exposed via multiple paths:

2017-05-04 11:28:42,325+03 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-3) [5b25e4e5] START, CreateVGVDSCommand(HostName = host_mixed_2, CreateVGVDSComm
andParameters:{runAsync='true', hostId='a1a83368-4c95-4b86-9f29-f0ce4598ac51', storageDomainId='83f1fde2-9998-428c-82b8-ea1ae8ef23ba', deviceList='[3514f0c5a516004ad]', force='false'}), log 
id: 694df394



Domain creation succeeded.



Used:
rhevm-4.1.2-0.1.el7.noarch
vdsm-4.19.11-1.el7ev.x86_64