Bug 852003 - createVG failures are not logged in vdsm logs
Summary: createVG failures are not logged in vdsm logs
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: unspecified
Hardware: All
OS: Linux
Target Milestone: ---
: 3.4.0
Assignee: Vered Volansky
QA Contact: Leonid Natapov
Whiteboard: storage
Depends On:
Blocks: rhev3.4beta 1142926
TreeView+ depends on / blocked
Reported: 2012-08-27 09:57 UTC by Gadi Ickowicz
Modified: 2016-02-10 20:24 UTC (History)
11 users (show)

Fixed In Version: ovirt-3.4.0-alpha1
Doc Type: Bug Fix
Doc Text:
VDSM now provides greater detail on the nature of errors that occurr when attempting to create a new storage domain and a host connected to the storage server on which the domain is being created is disconnected during the operation.
Clone Of:
Last Closed: 2014-06-09 13:24:27 UTC
oVirt Team: Storage
Target Upstream Version:

Attachments (Terms of Use)
vdsm (231.83 KB, application/x-gzip)
2012-08-28 07:00 UTC, Gadi Ickowicz
no flags Details

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:0504 0 normal SHIPPED_LIVE vdsm 3.4.0 bug fix and enhancement update 2014-06-09 17:21:35 UTC
oVirt gerrit 21089 0 None None None Never
oVirt gerrit 21168 0 None None None Never

Description Gadi Ickowicz 2012-08-27 09:57:04 UTC
Description of problem:
When attempting to create a new iscsi storage domain createVG fails with Invalid physical device.

Between the time getDeviceList was requested and the vgCreate the host suffered a disconnection from the storage server, so the /sys/block/dm-*/dm directory was not present anymore.

Thread-281::INFO::2012-08-27 09:12:59,238::logUtils::39:ispatcher:wrapper) Run and protect: getDeviceList, Return response: {'devList': [{'status': 'used', 'vendorID': 'IET'
, 'capacity': '21474836480', 'fwrev': '0001', 'vgUUID': 'lFOk00-QNSe-Z20g-osyk-rzpV-1fBe-FytEvU', 'pathlist': [{'initiatorname': 'default', 'connection': 'orion2.qa.lab.tlv.redh
at.com', 'iqn': 'iqn.1994-05.com.redhat:awinter_0_20120826164454668563', 'portal': '1', 'password': '******', 'port': '3260'}], 'logicalblocksize': '512', 'pathstatus': [{'physd
ev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': 'efiQe1-cgGI-MeQI-mwbv-RMKk-lD4A-YCdQmB', 'serial': 'SIET
_VIRTUAL-DISK', 'GUID': '1492565d9-28fb-4858', 'productID': 'VIRTUAL-DISK'}]}

Thread-881::INFO::2012-08-27 09:29:56,303::logUtils::37:ispatcher:wrapper) Run and protect: createVG(vgname='b1d90f0a-981f-4203-a9c2-f8f309409846', devlist=['1492565d9-28fb-
4858'], force=True, options=None)
Thread-881::ERROR::2012-08-27 09:29:56,304::task::853::TaskManager.Task:_setError) Task=`ca9e63ff-f252-4820-b557-57c2ace3c18b`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 861, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 1713, in createVG
    raise se.InvalidPhysDev(dev)
InvalidPhysDev: Invalid physical device: ('1492565d9-28fb-4858',)

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. connect host to storage server
2. manually (not through vdsm) disconnect the storage server
3. send createVG

Actual results:

Expected results:
It may be more informative to provide slightly more information in the logging error message, e.g. information about what part of the process fails - if the device is not in  
/sys/block/dm-*/dm or multipath, or if it fails finding the specific GUID etc.

Additional info:

Comment 2 Ayal Baron 2012-08-28 00:36:35 UTC
No vdsm log

Comment 3 Gadi Ickowicz 2012-08-28 07:00:07 UTC
Created attachment 607419 [details]

1. VDSM log attached
2.Eduardo managed to reproduce the bug with haim: 

>Reproduced with Haim in the same host.
> In between the getDeviceList and the createVG vdsm commands the host
> was (or suffered) a disconection of the storage server.
> The disconnect was not issued with vdsm.
> In this case the interval was about ~17 minutes.
> When the createVG was issued the /sys/block/dm-*/dm directory was not
> present anymore, therefore not listed by
> multipath.getMPDeviceIter().
> This and undeliying functions swallow all errors and we get only the
> cryptic: "InvalidPhysDev: Invalid physical device:" error.
> Since we use the GUID for pvcreate may be the whole use of the
> iterator in createVG can be avoided.

Comment 4 Sandro Bonazzola 2014-01-14 08:42:50 UTC
ovirt 3.4.0 alpha has been released

Comment 5 Leonid Natapov 2014-01-23 14:18:27 UTC
verified. vdsm-4.14.1-2.el6.x86_64

Comment 9 errata-xmlrpc 2014-06-09 13:24:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.