Bug 1218657

Summary: Creating Block Domain using a "dirty" LUN fails upon partition creation , "pvcreate failed with rc=5"
Product: Red Hat Enterprise Virtualization Manager Reporter: Ori Gofen <ogofen>
Component: ovirt-engineAssignee: Fred Rolland <frolland>
Status: CLOSED UPSTREAM QA Contact: Aharon Canan <acanan>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.5.1CC: acanan, amureini, bazulay, bugs, ecohen, frolland, gklein, lpeer, lsurette, mgoldboi, rbalakri, Rhev-m-bugs, tnisan, yeylon, ylavi
Target Milestone: ---Keywords: Regression
Target Release: 3.5.5   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1215427 Environment:
Last Closed: 2015-08-10 11:58:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1215427    
Bug Blocks: 1185865    
Attachments:
Description Flags
logs from RHEVM setup none

Description Ori Gofen 2015-05-05 13:42:53 UTC
Created attachment 1022198 [details]
logs from RHEVM setup

+++ This bug was initially created as a clone of Bug #1215427 +++

Description of problem:

Creating or extending a Block domain while using "dirty" Luns as targets fails.
This behavior is similar to bz #1185865, but happens regardless to what protocol (have reproduced with xml, Json) is used.

The Function _initpvs fails with "PhysDevInitializationError"

engine log:
2015-04-26 16:03:44,871 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (ajp--127.0.0.1-8702-5) [6a090930] Failed in 'CreateVGVDS' method
2015-04-26 16:03:44,872 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (ajp--127.0.0.1-8702-5) [6a090930] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand' return value 'OneUuidReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=601, mMessage=Failed to initialize physical device: ("['/dev/mapper/360060160f4a03000fa65675991dbe311', '/dev/mapper/360060160f4a03000fe65675991dbe311', '/dev/mapper/360060160f4a030007beed85291dbe311', '/dev/mapper/360060160f4a03000fc65675991dbe311', '/dev/mapper/360060160f4a03000fb65675991dbe311', '/dev/mapper/360060160f4a030007ceed85291dbe311']",)]]'
2015-04-26 16:03:44,878 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (ajp--127.0.0.1-8702-5) [6a090930] HostName = fury66.tlv.redhat.com
2015-04-26 16:03:44,880 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (ajp--127.0.0.1-8702-5) [6a090930] Command 'CreateVGVDSCommand(HostName = fury66.tlv.redhat.com, HostId = ebbc3458-d9cd-45a8-bb8d-3e2ebe0b4d6f, storageDomainId=9c5b47ed-3fd4-4022-987e-3d33985b751b, deviceList=[360060160f4a03000fa65675991dbe311, 360060160f4a03000fe65675991dbe311, 360060160f4a030007beed85291dbe311, 360060160f4a03000fc65675991dbe311, 360060160f4a03000fb65675991dbe311, 360060160f4a030007ceed85291dbe311], force=true)' execution failed: VDSGenericException: VDSErrorException: Failed to CreateVGVDS, error = Failed to initialize physical device: ("['/dev/mapper/360060160f4a03000fa65675991dbe311', '/dev/mapper/360060160f4a03000fe65675991dbe311', '/dev/mapper/360060160f4a030007beed85291dbe311', '/dev/mapper/360060160f4a03000fc65675991dbe311', '/dev/mapper/360060160f4a03000fb65675991dbe311', '/dev/mapper/360060160f4a030007ceed85291dbe311']",), code = 601
2015-04-26 16:03:44,888 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVGVDSCommand] (ajp--127.0.0.1-8702-5) [6a090930] FINISH, CreateVGVDSCommand, log id: 557da39b

vdsm:
Thread-1798::DEBUG::2015-04-26 16:07:39,125::lvm::301::Storage.Misc.excCmd::(cmd) FAILED: <err> = '  WARNING: lvmetad is running but disabled. Restart lvmetad before enabling it!\n  Device /dev/mapper/360060160f4a03000fe65675991dbe311 not found (or ignored by filtering).\n'; <rc> = 5
Thread-1798::DEBUG::2015-04-26 16:07:39,125::lvm::492::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalida
te operation' got the operation mutex
Thread-1798::DEBUG::2015-04-26 16:07:39,126::lvm::495::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalida
te operation' released the operation mutex
Thread-1798::ERROR::2015-04-26 16:07:39,126::lvm::737::Storage.LVM::(_initpvs) pvcreate failed with rc=5
Thread-1798::ERROR::2015-04-26 16:07:39,126::lvm::738::Storage.LVM::(_initpvs) ['  Physical volume "/dev/mapper/360060160f4a03000fa65675991dbe311" successfully created', '  Physical volume "/dev/mapper/360060160f4a030007beed85291dbe311" successfully created', '  Physical volume "/dev/mapper/360060160f4a03000fc65675991dbe311" successfully created', '  Physical volume "/dev/mapper/360060160f4a03000fb65675991dbe311" successfully created'], ['  WARNING: lvmetad is running but disabled. Restart lvmetad before enabling it!', '  Device /dev/mapper/360060160f4a03000fe65675991dbe311 not found (or ignored by filtering).']
Thread-1798::ERROR::2015-04-26 16:07:39,126::task::863::Storage.TaskManager.Task::(_setError) Task=`767b5c1d-7c56-45cb-bf99-4b69c54c237a`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 870, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 49, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 2142, in createVG
    (force.capitalize() == "True")))
  File "/usr/share/vdsm/storage/lvm.py", line 920, in createVG
    _initpvs(pvs, metadataSize, force)
  File "/usr/share/vdsm/storage/lvm.py", line 739, in _initpvs
    raise se.PhysDevInitializationError(str(devices))
PhysDevInitializationError: Failed to initialize physical device: ("['/dev/mapper/360060160f4a03000fa65675991dbe311',

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Ori Gofen on 2015-04-27 04:44:11 EDT ---

Version-Release number of selected component (if applicable):
3.6.0 master

How reproducible:
100%

Steps to Reproduce:
1.create Block domain using "dirty lunes"

Actual results:
operation fails

Expected results:
operation should be successful

Additional info:

Comment 1 Ori Gofen 2015-05-05 13:45:55 UTC
This bug was created on oVirt 3.6 and reproduced on rhevm vt14.3.
it is not an 'rpc' bug.

step to reproduce are:
1.create a Block domain by choosing all available luns, while making sure some of them are 'dirty'