Bug 704182 - [VDSM] [Storage] - If storage domain attach fails to update domain metadata operation is not recovered correctly
Summary: [VDSM] [Storage] - If storage domain attach fails to update domain metadata o...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm
Version: 6.1
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Saggi Mizrahi
QA Contact: Omri Hochman
URL:
Whiteboard: storage
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-05-12 11:28 UTC by yeylon@redhat.com
Modified: 2016-04-18 06:40 UTC (History)
7 users (show)

Fixed In Version: vdsm-4.9-68.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-12-06 07:18:07 UTC
Target Upstream Version:


Attachments (Terms of Use)
vdsm (701.07 KB, application/x-zip-compressed)
2011-05-12 11:28 UTC, yeylon@redhat.com
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2011:1782 0 normal SHIPPED_LIVE new packages: vdsm 2011-12-06 11:55:51 UTC

Description yeylon@redhat.com 2011-05-12 11:28:03 UTC
Created attachment 498514 [details]
vdsm

Description of problem:

i have a shared ISO domain i've imported in to my setup (rhel6 host 2.3 DC)
the import was successful.

i've attached the ISO domain into a DC and it become inactive.

once trying to activate this ISO domain i get the following error:

it looks like no exception is being raise when the metadata roleback fail

Thread-230146::DEBUG::2011-05-12 05:37:11,393::persistentDict::154::Storage.PersistentDict::(transaction) Error in transaction, rolling back changes
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/persistentDict.py", line 151, in transaction
    self.flush(self._metadata)
  File "/usr/share/vdsm/storage/persistentDict.py", line 269, in flush
    self._metaRW.writelines(lines)
  File "/usr/share/vdsm/storage/fileSD.py", line 48, in writelines
    oop.writeLines(tmpFilePath, metadata)
  File "/usr/share/vdsm/storage/processPool.py", line 35, in wrapper
    return self.runExternally(func, *args, **kwds)
  File "/usr/share/vdsm/storage/processPool.py", line 72, in runExternally
    raise err
IOError: [Errno 5] Input/output error


VDSM:
=====

Thread-231964::ERROR::2011-05-12 06:25:13,138::task::855::TaskManager.Task::(_setError) Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 863, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/storage/spm.py", line 115, in run
    return self.func(*args, **kwargs)
  File "/usr/share/vdsm/storage/spm.py", line 1205, in public_activateStorageDomain
    pool.activateSD(sdUUID)
  File "/usr/share/vdsm/storage/sp.py", line 806, in activateSD
    self.validateAttachedDomain(sdUUID)
  File "/usr/share/vdsm/storage/sp.py", line 219, in validateAttachedDomain
    raise se.StorageDomainNotInPool(self.spUUID, sdUUID)
  
Thread-231964::ERROR::2011-05-12 06:25:13,149::dispatcher::103::Storage.Dispatcher.Protect::(run) {'status': {'message': "Storage domain not in pool: 'domain=2e46da5a-2ad6-4672-a297-02b6d3713274, pool=1570c166-bf52-4d94-950b-8ac3aef95c0d'", 'code': 353}, 'args': ['domain=2e46da5a-2ad6-4672-a297-02b6d3713274, pool=1570c166-bf52-4d94-950b-8ac3aef95c0d']}

RHEVM
======

2011-05-12 13:29:50,820 ERROR [com.redhat.rhevm.vdsbroker.irsbroker.IrsBrokerCommand] (pool-11-thread-43) IrsBroker::Failed::ActivateStorageDomainVDS due to: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain not in pool: 'domain=2e46da5a-2ad6-4672-a297-02b6d3713274, pool=1570c166-bf52-4d94-950b-8ac3aef95c0d'

----------------------------------------------------------------

the thing is that when looking at the DC info this domain is there....

[root@south-04 ~]# vdsClient -s 0 getStorageDomainsList 1570c166-bf52-4d94-950b-8ac3aef95c0d
371bf022-5624-4fe3-877f-d1b73726c7c6
2e46da5a-2ad6-4672-a297-02b6d3713274
83160733-0b82-4fb1-96f5-1c079cc4a54e
9d03a995-600d-4ae0-a974-445acb3d49ed


[root@south-04 ~]# vdsClient -s 0 getStorageDomainInfo 2e46da5a-2ad6-4672-a297-02b6d3713274
        uuid = 2e46da5a-2ad6-4672-a297-02b6d3713274
        pool = ['6e4a0f57-646b-4624-9662-6e8a06b36973', 'a2f277d8-2772-4765-bb67-1bb699fcbfb4', '7a5cc7c9-13d4-466f-8374-e532947be240', 'c62d3b8f-e458-453e-aa53-ee48f2db1888', 'ade9f73e-272a-42bb-8b8c-bb3288e0637e', '4fa85879-ce59-45e5-835d-dccb607720e7', '6017f655-4872-4175-adf5-743a6599be2b', '101eff42-1a51-4707-801b-184e01550e07', '55e2de78-0568-4c0d-ae9a-1e596c9ed4da', '2f964da4-d1bd-4886-afd9-dad7ba68d9a6', '5e4af8a5-9a0c-49bb-9869-2a3cb4b47141', '2d660ad4-7ecc-41e2-8809-b9efe913d813', 'fc11ea2b-457d-4169-8636-8936af56b8b0', '47620fa0-8fdf-4ba2-8be3-6db2bfb16a37']
        lver = -1
        version = 0
        role = Regular
        remotePath = rhev-i8c-01.mpc.lab.eng.bos.redhat.com:/ISO/isoDomain
        spm_id = -1
        type = NFS
        class = Iso
        master_ver = 0
        name = iso-i8c-01




Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 Saggi Mizrahi 2011-05-16 11:49:22 UTC
http://gerrit.usersys.redhat.com/408

Comment 4 Ortal 2011-05-31 11:52:02 UTC
Verified on VDSM Version: vdsm-4.9-70.el6.x86_64

Checked when:
1. Adding role using iptables command to reject iso domain
2. Can't activate ISO domain attached to the DC
3. Getting appropriate error message
4. Flash iptables
5. ISO domain activation successful.

Comment 5 errata-xmlrpc 2011-12-06 07:18:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2011-1782.html


Note You need to log in before you can comment on or make changes to this bug.