Bug 1460501 - Failed to extend block-based storage domain, ExtendSANStorageDomainCommand pass multiple PVs with the same name
Summary: Failed to extend block-based storage domain, ExtendSANStorageDomainCommand pa...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.1.3
: 4.1.3.3
Assignee: Daniel Erez
QA Contact: Eyal Shenitzky
URL:
Whiteboard:
Depends On:
Blocks: 1454864
TreeView+ depends on / blocked
 
Reported: 2017-06-11 11:30 UTC by Eyal Shenitzky
Modified: 2017-07-06 13:42 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-07-06 13:42:33 UTC
oVirt Team: Storage
Embargoed:
rule-engine: ovirt-4.1+


Attachments (Terms of Use)
engine and vdsm logs (1009.11 KB, application/x-gzip)
2017-06-11 11:30 UTC, Eyal Shenitzky
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 78148 0 master MERGED webadmin: filter out duplicate luns on extend SD 2020-02-28 13:51:46 UTC
oVirt gerrit 78162 0 master MERGED core: lunIds - migrating type to Set instead of List 2020-02-28 13:51:46 UTC
oVirt gerrit 78163 0 ovirt-engine-4.1 MERGED webadmin: filter out duplicate luns on extend SD 2020-02-28 13:51:46 UTC

Description Eyal Shenitzky 2017-06-11 11:30:03 UTC
Created attachment 1286824 [details]
engine and vdsm logs

Description of problem:
iSCSI storage domain extention fails in case the initiator is connected to the LUN through multiple targets. This happens because engine sends to vdsm multiple PVs, with the same PV name, according to the targets number in CreateVG.

VDSM command ExtendStorageDomainVDS failed: Cannot extend Volume Group: "vgname=c00b0577-4be8-434d-ac6d-fad06daa7172, devname=['/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd']"

Engine log:

2017-06-11 13:58:20,481+03 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.ExtendStorageDomainVDSCommand] (default task-23) [118515b7] Failed in 'ExtendStorageDomainVDS' method
2017-06-11 13:58:20,500+03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-23) [118515b7] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command ExtendStorageDomainVDS failed: Cannot extend Volume Group: "vgname=c00b0577-4be8-434d-ac6d-fad06daa7172, devname=['/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd']"
2017-06-11 13:58:20,500+03 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (default task-23) [118515b7] IrsBroker::Failed::ExtendStorageDomainVDS: IRSGenericException: IRSErrorException: Failed to ExtendStorageDomainVDS, error = Cannot extend Volume Group: "vgname=c00b0577-4be8-434d-ac6d-fad06daa7172, devname=['/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd']", code = 503
2017-06-11 13:58:20,522+03 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ExtendStorageDomainVDSCommand] (default task-23) [118515b7] FINISH, ExtendStorageDomainVDSCommand, log id: 40c6a546
2017-06-11 13:58:20,523+03 ERROR [org.ovirt.engine.core.bll.storage.domain.ExtendSANStorageDomainCommand] (default task-23) [118515b7] Command 'org.ovirt.engine.core.bll.storage.domain.ExtendSANStorageDomainCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to ExtendStorageDomainVDS, error = Cannot extend Volume Group: "vgname=c00b0577-4be8-434d-ac6d-fad06daa7172, devname=['/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd', '/dev/mapper/3514f0c5a516008dd']", code = 503 (Failed with error VolumeGroupExtendError and code 503)

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Create a bloc-based storage domain
2. Extend the storage-domain using free device

Actual results:
Storage domain extension failed

Expected results:
Storage domain extension should succeed

Additional info:
VDSM and engine log attached

Comment 1 Eyal Shenitzky 2017-06-11 12:01:46 UTC
Versions:
VDSM - 4.20.0-970.gitf0c24e5.el7.centos.x86_64
Engine - unregistered

Comment 2 Eyal Shenitzky 2017-06-11 12:02:35 UTC
Engine - 4.2.0-0.0.master.20170602194647.gitaf23eb5.el7.centos

Comment 3 Allon Mureinik 2017-06-12 08:59:02 UTC
Sounds like a similar issue to bug 1441706. If so, it's probably an easy fix.

Comment 4 Daniel Erez 2017-06-12 12:59:11 UTC
@Eyal - I've tried the described scenario (using the WebAdmin) with two luns exposed from multiple targets - seems to behave as expected (i.e. the used luns are grayed-out in the extend dialog). Can you please describe your exact luns/targets configuration? Did you able to select the same (used) lun? Did you try it only from rest-api?

Comment 5 Eyal Shenitzky 2017-06-13 05:12:37 UTC
I try it using the web admin and the Rest-API,
All the flow worked well, the engine sends a message that the storage-domain updated and afterward the error occurred.
We are using XIO as backend if it is relevant.

Comment 6 Eyal Shenitzky 2017-06-13 14:47:55 UTC
Steps to reproduce:

1. Create a block-based storage domain
2. Press the 'Manage Domain' button
3. Press 'Target > Luns' side tab
4. Select free device
5. Extend the storage domain

Comment 7 rhev-integ 2017-06-16 16:44:34 UTC
INFO: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Open patch attached]

For more info please contact: infra

Comment 8 Eyal Shenitzky 2017-06-20 11:15:00 UTC
Verified with the following code:
---------------------------------------
VDSM - 4.19.19-1.el7ev.x86_64
RHEVM - 4.1.3.4-0.1.el7
 
Steps to reproduce:
------------------------
1. Create a block-based storage domain
2. Press the 'Manage Domain' button
3. Press 'Target > Luns' side tab
4. Select free device
5. Extend the storage domain

Storage Extended as expected

Moving to VERIFIED


Note You need to log in before you can comment on or make changes to this bug.