Bug 842737 - negative case - getSpmStatus keeps reporting "Meta Data parameter invalid" after connection to master domain was restored (stale cache)
negative case - getSpmStatus keeps reporting "Meta Data parameter invalid" af...
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm (Show other bugs)
x86_64 Linux
unspecified Severity high
: ---
: 3.1.0
Assigned To: Ayal Baron
Depends On:
  Show dependency treegraph
Reported: 2012-07-24 09:42 EDT by Haim
Modified: 2016-02-10 11:28 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-04-04 11:14:42 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
vdsm.log (490.81 KB, application/x-gzip)
2012-07-24 09:44 EDT, Haim
no flags Details

  None (edit)
Description Haim 2012-07-24 09:42:16 EDT
Description of problem:

I have a case of stale cache which prevents vdsm from re-connecting to pool, which looks as follows:

- local host with local storage domain (domain exists under /opt/dom1/<sdUUID>
- mv /opt/dom1/<sdUUID> /tmp/
- perform an SPM action 
- pool becomes non-responsive, engine send getSpmStatus to vdsm, vdsm returns:

Thread-117003::INFO::2012-07-24 19:27:11,860::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='13e32227-648f-4240-8eb1-45c1b4e423d4
', options=None)
Thread-117003::ERROR::2012-07-24 19:27:11,861::hsm::535::Storage.HSM::(getSpmStatus) Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/hsm.py", line 527, in getSpmStatus
    status = {'spmStatus':pool.getSpmRole(), 'spmLver': pool.getSpmLver(), 'spmId':pool.getSpmId()}
  File "/usr/share/vdsm/storage/sp.py", line 133, in getSpmLver
    return self.getMetaParam(PMDK_LVER)
  File "/usr/share/vdsm/storage/sp.py", line 1447, in getMetaParam
    return self._metadata[key]
  File "/usr/share/vdsm/storage/persistentDict.py", line 85, in __getitem__
    return dec(self._dict[key])
  File "/usr/share/vdsm/storage/persistentDict.py", line 195, in __getitem__
    raise KeyError(key)
Thread-117003::ERROR::2012-07-24 19:27:11,861::task::853::TaskManager.Task::(_setError) Task=`e62289e7-e9b4-48bc-b135-e8f45ce5b8d7`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 861, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 536, in getSpmStatus
    raise se.MetaDataParamError("Version or spm id invalid")
MetaDataParamError: Meta Data parameter invalid: ('Version or spm id invalid',

- after that, I restore the storage domain to its original place
  * mv /tmp/sdUUID /opt/dom1/
- vdsm keeps reporting the above message forever although connection was restored, I even executed repoStats command and the results were positive:

 [root@nott-vds1 ~]# vdsClient -s 0 repoStats
Domain 77f493a8-078f-4e06-9506-4bd2ca4a8960 {'delay': '0.0029981136322', 'lastCheck': 1343141463.5053949, 'code': 0, 'valid': True}
Domain 7436be21-badb-4296-a274-c9dd010582f5 {'delay': '0.00148296356201', 'lastCheck': 1343141460.641876, 'code': 0, 'valid': True}
Domain 174d28bf-a3a6-49c8-9107-d3ffa2d5252c {'delay': '0.00143098831177', 'lastCheck': 1343141464.5150161, 'code': 0, 'valid': True}

[root@nott-vds1 ~]# less /var/log/vdsm/vdsm.log
[root@nott-vds1 ~]# vdsClient -s 0 getSpmStatus 13e32227-648f-4240-8eb1-45c1b4e423d4
Meta Data parameter invalid: ('Version or spm id invalid',)

The above test implies that vdsm reads from cache and not access to meta-data, in any-case, workaround  is to restart vdsm (which shows its a cache issue).
Comment 1 Haim 2012-07-24 09:44:13 EDT
Created attachment 600032 [details]
Comment 5 RHEL Product and Program Management 2012-12-14 02:59:01 EST
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Note You need to log in before you can comment on or make changes to this bug.