Bug 674128 - [vdsm] [storage] volume meta-data error in create volume task 'missing offset tag on volume'
Summary: [vdsm] [storage] volume meta-data error in create volume task 'missing offset...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm
Version: 6.1
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Eduardo Warszawski
QA Contact: Daniel Paikov
URL:
Whiteboard:
: 597807 (view as bug list)
Depends On:
Blocks: 679048
TreeView+ depends on / blocked
 
Reported: 2011-01-31 18:01 UTC by Haim
Modified: 2014-01-13 00:48 UTC (History)
11 users (show)

Fixed In Version: vdsm-4.9-49
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-08-19 15:26:29 UTC
Target Upstream Version:


Attachments (Terms of Use)
vdsm.log (2.53 MB, application/x-gzip)
2011-01-31 18:01 UTC, Haim
no flags Details

Description Haim 2011-01-31 18:01:59 UTC
Created attachment 476244 [details]
vdsm.log

Description of problem:

created lots of vms from pool, during creation process, noticed the following error: 

4a905402-8498-429c-b46c-083cc096a1b6::ERROR::2011-01-31 16:01:15,856::blockVolume::529::Storage.Volume::(getMetaOffset) missing offset tag on volume f967d8f8-3
5fd-4122-ad92-f1e0fc7c4c44
4a905402-8498-429c-b46c-083cc096a1b6::DEBUG::2011-01-31 16:01:15,856::resourceManager::517::ResourceManager::(releaseResource) Trying to release resource 'a545
fb3b-027c-458c-b76f-5facf67ac8cc_imageNS.035b1408-3156-40ee-b872-3c3606f8d595'
4a905402-8498-429c-b46c-083cc096a1b6::DEBUG::2011-01-31 16:01:15,857::resourceManager::532::ResourceManager::(releaseResource) Released resource 'a545fb3b-027c
-458c-b76f-5facf67ac8cc_imageNS.035b1408-3156-40ee-b872-3c3606f8d595' (0 active users)
4a905402-8498-429c-b46c-083cc096a1b6::DEBUG::2011-01-31 16:01:15,857::resourceManager::537::ResourceManager::(releaseResource) Resource 'a545fb3b-027c-458c-b76f-5facf67ac8cc_imageNS.035b1408-3156-40ee-b872-3c3606f8d595' is free, finding out if anyone is waiting for it.
4a905402-8498-429c-b46c-083cc096a1b6::DEBUG::2011-01-31 16:01:15,858::resourceManager::544::ResourceManager::(releaseResource) No one is waiting for resource 'a545fb3b-027c-458c-b76f-5facf67ac8cc_imageNS.035b1408-3156-40ee-b872-3c3606f8d595', Clearing records.
4a905402-8498-429c-b46c-083cc096a1b6::ERROR::2011-01-31 16:01:15,858::task::854::TaskManager.Task::(_setError) Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 862, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/storage/task.py", line 299, in run
    return self.cmd(*self.argslist, **self.argsdict)
  File "/usr/share/vdsm/storage/spm.py", line 110, in run
    return self.func(*args, **kwargs)
  File "/usr/share/vdsm/storage/spm.py", line 885, in createVolume
    srcImgUUID=srcImgUUID, srcVolUUID=srcVolUUID)
  File "/usr/share/vdsm/storage/blockSD.py", line 355, in createVolume
    volUUID, desc, srcImgUUID, srcVolUUID)
  File "/usr/share/vdsm/storage/blockVolume.py", line 218, in create
    mysd.produceVolume(imgUUID, volUUID).setLegality(volume.LEGAL_VOL)
  File "/usr/share/vdsm/storage/volume.py", line 380, in setLegality
    self.setMetaParam(LEGALITY, legality)
  File "/usr/share/vdsm/storage/volume.py", line 756, in setMetaParam
    meta = self.getMetadata(nocache=nocache)
  File "/usr/share/vdsm/storage/blockVolume.py", line 542, in getMetadata
    offs = self.getMetaOffset()
  File "/usr/share/vdsm/storage/blockVolume.py", line 530, in getMetaOffset
    raise se.VolumeMetadataReadError("missing offset tag on volume %s" % self.volUUID)
VolumeMetadataReadError: Error while processing volume meta data: ('missing offset tag on volume f967d8f8-35fd-4122-ad92-f1e0fc7c4c44',)


Run and protect: createVolume, args: ( sdUUID=a545fb3b-027c-458c-
b76f-5facf67ac8cc spUUID=a8e3a5e0-1437-4dfb-9ac5-c6835227a074 imgUUID=035b1408-3156-40ee-b872-3c3606f8d595 size=5368709120 volFormat=4 preallocate=2 diskType=2
 volUUID=f967d8f8-35fd-4122-ad92-f1e0fc7c4c44 desc=_ActiveImage_FC-rhel6-194_Mon Jan 31 16:02:05 IST 2011 srcImgUUID=45465a33-75f6-447d-9fc4-dc9f4de8836a srcVo
lUUID=9a2a526f-1dbd-4665-8f7e-00220e424fbb)

Thread-6086::INFO::2011-01-31 16:16:22,881::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: createVolume, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': '28ee2a60-6e1d-49f2-aa69-fe6f873ec43a'}


repro steps: 

1) create vm 
2) create template 
3) create pool of 200 vms 
4) watch log.

Comment 2 Eduardo Warszawski 2011-02-10 13:17:23 UTC
The bug is a result of a race between the blockVolume.create() thread and another thread doing lvs on the same VG.

The race is removed when the legality is set with all the metadata instead to do an extra lvm operation.

Comment 3 Eduardo Warszawski 2011-02-24 08:23:42 UTC
*** Bug 597807 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.