Created attachment 1365815 [details]
VDSM & Engine logs

Description of problem:
Trying to create new iSCSI domain after reprovision of host & engine and getting an error (from UI event log):
"Error while executing action New SAN Storage Domain: Physical device initialization failed. Please check that the device is empty and accessible by the host."
LUN was in use in previous setup.


Version-Release number of selected component (if applicable):
Engine: 4.2.0-0.6.el7
VDSM: 4.20.9-1.el7ev

How reproducible:


Steps to Reproduce(via Webadmin):
1.Create new iSCSI domain after reprovision for both host & engine
2.Select LUN that was in use in previous setup & create SD


Actual results:
"Error while executing action New SAN Storage Domain: Physical device initialization failed. Please check that the device is empty and accessible by the host."

Expected results:
New domain created

Additional info:
vdsm log:
2017-12-11 09:16:33,184+0200 WARN  (jsonrpc/5) [storage.HSM] getPV failed for guid: 3514f0c5a51600ffa (hsm:1980)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1977, in _getDeviceList
    pv = lvm.getPV(guid)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 852, in getPV
    raise se.InaccessiblePhysDev((pvName,))
InaccessiblePhysDev: Multipath cannot access physical device(s): "devices=(u'3514f0c5a51600ffa',)"

2017-12-11 09:16:38,772+0200 ERROR (jsonrpc/0) [storage.TaskManager.Task] (Task='3a24f663-5696-40f9-8ae8-5b9da2ad5a50') Unexpected error (task:875)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run
    return fn(*args, **kargs)
  File "<string>", line 2, in createVG
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method
    ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2128, in createVG
    force=force)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 983, in createVG
    _initpvs(pvs, metadataSize, force)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 756, in _initpvs
    raise se.PhysDevInitializationError(str(devices))
PhysDevInitializationError: Failed to initialize physical device: ("[u'/dev/mapper/3514f0c5a51600ffa']",)


Engine log:
2017-12-11 09:16:38,776+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-12) [70a80370] Failed in 'CreateVGVDS' method
2017-12-11 09:16:38,783+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-12) [70a80370] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM h1 command CreateVGVDS failed: Failed to initialize physical device: ("[u'/dev/mapper/3514f0c5a51600ffa']",)
2017-12-11 09:16:38,784+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-12) [70a80370] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand' return value 'OneUuidReturn:{status='Status [code=601, message=Failed to initialize physical device: ("[u'/dev/mapper/3514f0c5a51600ffa']",)]'}'
2017-12-11 09:16:38,784+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-12) [70a80370] HostName = h1
2017-12-11 09:16:38,784+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-12) [70a80370] Command 'CreateVGVDSCommand(HostName = h1, CreateVGVDSCommandParameters:{hostId='8aae390b-768a-4792-9387-f09737b18973', storageDomainId='595d57bb-c6ad-4d28-90df-3b0a819bfa0b', deviceList='[3514f0c5a51600ffa]', force='true'})' execution failed: VDSGenericException: VDSErrorException: Failed to CreateVGVDS, error = Failed to initialize physical device: ("[u'/dev/mapper/3514f0c5a51600ffa']",), code = 601
2017-12-11 09:16:38,784+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (default task-12) [70a80370] FINISH, CreateVGVDSCommand, log id: 738e816
2017-12-11 09:16:38,784+02 ERROR [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-12) [70a80370] Command 'org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to CreateVGVDS, error = Failed to initialize physical device: ("[u'/dev/mapper/3514f0c5a51600ffa']",), code = 601 (Failed with error PhysDevInitializationError and code 601)
2017-12-11 09:16:38,788+02 INFO  [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-12) [70a80370] Command [id=124728ca-040d-4172-bc48-b60ddacfce5f]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.profiles.DiskProfile; snapshot: bed74add-9a0d-4bbf-9f0f-a487b28b5c35.
2017-12-11 09:16:38,790+02 INFO  [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-12) [70a80370] Command [id=124728ca-040d-4172-bc48-b60ddacfce5f]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StorageDomainDynamic; snapshot: 595d57bb-c6ad-4d28-90df-3b0a819bfa0b.
2017-12-11 09:16:38,791+02 INFO  [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-12) [70a80370] Command [id=124728ca-040d-4172-bc48-b60ddacfce5f]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StorageDomainStatic; snapshot: 595d57bb-c6ad-4d28-90df-3b0a819bfa0b.
2017-12-11 09:16:38,804+02 ERROR [org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand] (default task-12) [70a80370] Transaction rolled-back for command 'org.ovirt.engine.core.bll.storage.domain.AddSANStorageDomainCommand'.
2017-12-11 09:16:38,812+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-12) [70a80370] EVENT_ID: USER_ADD_STORAGE_DOMAIN_FAILED(957), Failed to add Storage Domain iSCSI_sd2. (User: admin@internal-authz)
2017-12-11 09:19:57,055+02 INFO  [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'default' is using 0 threads out of 1 and 5 tasks are waiting in the queue.