Bug 1460501 - Failed to extend block-based storage domain, ExtendSANStorageDomainCommand pass multiple PVs with the same name
Failed to extend block-based storage domain, ExtendSANStorageDomainCommand pa...
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage (Show other bugs)
4.2.0
Unspecified Unspecified
unspecified Severity high (vote)
: ovirt-4.1.3
: 4.1.3.3
Assigned To: Daniel Erez
Eyal Shenitzky
:
Depends On:
Blocks: 1454864
  Show dependency treegraph
 
Reported: 2017-06-11 07:30 EDT by Eyal Shenitzky
Modified: 2017-07-06 09:42 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-07-06 09:42:33 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.1+


Attachments (Terms of Use)
engine and vdsm logs (1009.11 KB, application/x-gzip)
2017-06-11 07:30 EDT, Eyal Shenitzky
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 78148 master MERGED webadmin: filter out duplicate luns on extend SD 2017-06-14 04:44 EDT
oVirt gerrit 78162 master MERGED core: lunIds - migrating type to Set instead of List 2017-06-29 07:57 EDT
oVirt gerrit 78163 ovirt-engine-4.1 MERGED webadmin: filter out duplicate luns on extend SD 2017-06-14 04:58 EDT

  None (edit)
Description Eyal Shenitzky 2017-06-11 07:30:03 EDT
Created attachment 1286824 [details]
engine and vdsm logs

Description of problem:
iSCSI storage domain extention fails in case the initiator is connected to the LUN through multiple targets. This happens because engine sends to vdsm multiple PVs, with the same PV name, according to the targets number in CreateVG.

VDSM command ExtendStorageDomainVDS failed: Cannot extend Volume Group: "vgname=c00b0577-4be8-434d-ac6d-fad06daa7172, devname=['/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd']"

Engine log:

2017-06-11 13:58:20,481+03 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.ExtendStorageDomainVDSCommand] (default task-23) [118515b7] Failed in 'ExtendStorageDomainVDS' method
2017-06-11 13:58:20,500+03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-23) [118515b7] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command ExtendStorageDomainVDS failed: Cannot extend Volume Group: "vgname=c00b0577-4be8-434d-ac6d-fad06daa7172, devname=['/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd']"
2017-06-11 13:58:20,500+03 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (default task-23) [118515b7] IrsBroker::Failed::ExtendStorageDomainVDS: IRSGenericException: IRSErrorException: Failed to ExtendStorageDomainVDS, error = Cannot extend Volume Group: "vgname=c00b0577-4be8-434d-ac6d-fad06daa7172, devname=['/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd']", code = 503
2017-06-11 13:58:20,522+03 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ExtendStorageDomainVDSCommand] (default task-23) [118515b7] FINISH, ExtendStorageDomainVDSCommand, log id: 40c6a546
2017-06-11 13:58:20,523+03 ERROR [org.ovirt.engine.core.bll.storage.domain.ExtendSANStorageDomainCommand] (default task-23) [118515b7] Command 'org.ovirt.engine.core.bll.storage.domain.ExtendSANStorageDomainCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to ExtendStorageDomainVDS, error = Cannot extend Volume Group: "vgname=c00b0577-4be8-434d-ac6d-fad06daa7172, devname=['/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd']", code = 503 (Failed with error VolumeGroupExtendError and code 503)

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Create a bloc-based storage domain
2. Extend the storage-domain using free device

Actual results:
Storage domain extension failed

Expected results:
Storage domain extension should succeed

Additional info:
VDSM and engine log attached
Comment 1 Eyal Shenitzky 2017-06-11 08:01:46 EDT
Versions:
VDSM - 4.20.0-970.gitf0c24e5.el7.centos.x86_64
Engine - unregistered
Comment 2 Eyal Shenitzky 2017-06-11 08:02:35 EDT
Engine - 4.2.0-0.0.master.20170602194647.gitaf23eb5.el7.centos
Comment 3 Allon Mureinik 2017-06-12 04:59:02 EDT
Sounds like a similar issue to bug 1441706. If so, it's probably an easy fix.
Comment 4 Daniel Erez 2017-06-12 08:59:11 EDT
@Eyal - I've tried the described scenario (using the WebAdmin) with two luns exposed from multiple targets - seems to behave as expected (i.e. the used luns are grayed-out in the extend dialog). Can you please describe your exact luns/targets configuration? Did you able to select the same (used) lun? Did you try it only from rest-api?
Comment 5 Eyal Shenitzky 2017-06-13 01:12:37 EDT
I try it using the web admin and the Rest-API,
All the flow worked well, the engine sends a message that the storage-domain updated and afterward the error occurred.
We are using XIO as backend if it is relevant.
Comment 6 Eyal Shenitzky 2017-06-13 10:47:55 EDT
Steps to reproduce:

1. Create a block-based storage domain
2. Press the 'Manage Domain' button
3. Press 'Target > Luns' side tab
4. Select free device
5. Extend the storage domain
Comment 7 rhev-integ 2017-06-16 12:44:34 EDT
INFO: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Open patch attached]

For more info please contact: infra@ovirt.org
Comment 8 Eyal Shenitzky 2017-06-20 07:15:00 EDT
Verified with the following code:
---------------------------------------
VDSM - 4.19.19-1.el7ev.x86_64
RHEVM - 4.1.3.4-0.1.el7
 
Steps to reproduce:
------------------------
1. Create a block-based storage domain
2. Press the 'Manage Domain' button
3. Press 'Target > Luns' side tab
4. Select free device
5. Extend the storage domain

Storage Extended as expected

Moving to VERIFIED

Note You need to log in before you can comment on or make changes to this bug.