Description of problem: flags is showing as Object Map invalid: "flags: object map invalid" rbd image 'OM_NEWClone_new4': size 11303 MB in 2826 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.77998515f007c format: 2 features: layering, exclusive, object map flags: object map invalid parent: Tanay-RBD/OM_NEW4@OM_NEWsnap4 overlap: 10240 MB Version-Release number of selected component (if applicable): ceph version 0.94.1 librbd1-0.94.1-10.el7cp.x86_64 How reproducible: 100% Steps to Reproduce: 1. Create a rbd Image with object map enabled. rbd image 'OM_NEWClone_new5': size 11037 MB in 2760 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.7799c79e2a9e3 format: 2 features: layering, exclusive, object map flags: parent: Tanay-RBD/OM_NEW5@OM_NEWsnap5 overlap: 10240 MB 2. Used this script to re-size and shrink the rbd image multiple times. #!/bin/python import os import random import time size=11000 i=0 new_size=0 sh_size=0 while i < 10: x=random.randint(1,500) new_size=size + x cmd1 = 'rbd resize Tanay-RBD/OM_NEWClone_new5 --size %s' %new_size print 'cmd is %s' %cmd1 os.system(cmd1) time.sleep(5) x=random.randint(1,100) sh_size= new_size - x cmd2 = 'rbd resize Tanay-RBD/OM_NEWClone_new5 --size %s --allow-shrink' %sh_size print 'cmd2 is %s' %cmd2 os.system(cmd2) i = i +1 Actual results: I am seeing the flags is showing flags: object map invalid Expected results: The flags should not be shown as invalid. Additional info:
I attempted to recreate this on RHEL7.1 and was unable to reproduce with the specified Ceph RPM version and script. Can you please repeat with "debug rbd = 20" and "log file = /path/to/client/logs/$name.$pid.log" in your ceph.conf "[client]" section and attach the client and OSD logs? For clarity, I issued the following rbd commands prior to running the Python script (modified for the new image name): # rbd create --image-format 2 --image-features 13 --size 10240 parent # rbd snap create parent@1 # rbd snap protect parent@1 # rbd clone --image-features=13 parent child # python test.py > /dev/null 2>&1 # rbd info child rbd image 'child': size 11410 MB in 2853 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.371674b0dc51 format: 2 features: layering, exclusive, object map flags: parent: rbd/parent@1 overlap: 10240 MB Thanks.
Jason, I have again reproduced it again 4/5 times. I am simply re-sizing the Clone image multiple times using the given script. PFA ( rbd creation and re-size script ) Even with "debug rbd = 20", i don's see any logs from OSD's. There is no client involved here. Thanks
Created attachment 1028990 [details] rdb create
Created attachment 1028991 [details] resize script
The rbd CLI is a Ceph client -- so all options under the "[client]" section of the conf file will apply to the program (and to other clients like QEMU). I am still unable to repeat, so I need those logs from the client -- I would not expect "debug rbd = 20" to have any effect on the OSD logs since it is a 100% client-side parameter. > Jason, > I have again reproduced it again 4/5 times. > I am simply re-sizing the Clone image multiple times using the given script. > PFA ( rbd creation and re-size script ) > > Even with "debug rbd = 20", i don's see any logs from OSD's. > > There is no client involved here. > > Thanks
PFA the client log, i was able to reproduce it in 2nd iterations.
Created attachment 1030694 [details] Client log
Please find the correct log.
Created attachment 1030713 [details] Correct log
It appears that in addition to running the attached script, 'rbd info' was run on the image concurrently. It was actually the 'rbd info' invocation which caused the object map to become invalidated since it opened the image mid-resize and noticed a discrepancy with the image size vs the object map size. This explains why I was not able to reproduce the issue with the provided script.
Hi Jason/ Josh, what is the impact of this defect ? is this going to affect the overall functionality of system ? When will this be fixed ? Regards, Kiran raje urs J
We'll want to fix this soon in upstream master, but I wouldn't treat this as a blocker for 1.3.0. Object map is a very useful optimization, but it is purely an optimization, so having an invalid object map is the same as not having it enabled. Some behavior will be the same speed as with it disabled (still the default) and correctness will be unaffected.
The patches we need are https://github.com/ceph/ceph/pull/5279/commits
Marking this Bug as verified, as it is working fine. The image is not getting "flags: object map invalid"
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2015:2512
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2015:2066