Description of problem: The OpenStack Manila's cephfs driver uses ceph_volume_client to interact with the Ceph cluster. The driver needs to store ganesha config data in RADOS objects to implement highly available Manila deployments. Adding get, put, delete object interfaces to ceph_volume_client will let the driver safely interact with RADOS objects. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Ramakrishnan you can test this as follows, provided the node with ceph_volume_client library has access to CephFS 1. Create a ceph client auth ID with r/w access to the entire CephFS . It will be used by the ceph volume client. $ sudo ceph auth get-or-create client.ram mon "allow *" osd "allow rw" mds "allow *" -o /etc/ceph/ceph.client.manila.keyring 2. Use the ceph_volume_client python library to put, get and delete RADOS objects in python2 interpreter. >>> import ceph_volume_client >>> vc = ceph_volume_client.CephFSVolumeClient("ram", "/etc/ceph/ceph.conf", "ceph") >>> vc.connect() >>> vc.put_object("cephfs_data", "trial", "Hello") >>> h = vc.get_object("cephfs_data", "trial") >>> assert h == "Hello" >>> h = vc.delete_object("cephfs_data", "trial") >>> h = vc.get_object("cephfs_data", "trial") # You'll hit the following error, rados.ObjectNotFound as the object was # just deleted >>> vc.disconnect() >>>
(In reply to Ram Raja from comment #22) > Ramakrishnan you can test this as follows, provided the node with > ceph_volume_client library has access to CephFS > > 1. Create a ceph client auth ID with r/w access to the entire CephFS . It > will be used by the ceph volume client. > $ sudo ceph auth get-or-create client.ram mon "allow *" osd "allow rw" mds > "allow *" -o /etc/ceph/ceph.client.manila.keyring Oops. Use the following command instead. $ sudo ceph auth get-or-create client.ram mon "allow *" osd "allow rw" mds "allow *" -o /etc/ceph/ceph.client.ram.keyring > > 2. Use the ceph_volume_client python library to put, get and delete RADOS > objects in python2 interpreter. > > >>> import ceph_volume_client > >>> vc = ceph_volume_client.CephFSVolumeClient("ram", "/etc/ceph/ceph.conf", "ceph") > >>> vc.connect() # Now, you're storing "Hello" in RADOS object named "trial" in the RADOS pool named "cephfs_data" > >>> vc.put_object("cephfs_data", "trial", "Hello") # You're checking whether "Hello" was stored in the RADOS object named "trial" > >>> h = vc.get_object("cephfs_data", "trial") > >>> assert h == "Hello" > >>> h = vc.delete_object("cephfs_data", "trial") > >>> h = vc.get_object("cephfs_data", "trial") > # You'll hit the following error, rados.ObjectNotFound as the object was > # just deleted > >>> vc.disconnect() > >>>
Thanks for the steps Ramana. Moving this bug to verified state, as per the steps in comment 23. Command output: ------------------------- [ubuntu@magna117 ceph]$ sudo ceph auth get-or-create client.ram mon "allow *" osd "allow rw" mds "allow *" -o /etc/ceph/ceph.client.ram.keyring [ubuntu@magna117 ceph]$ ls ceph.client.admin.keyring ceph.client.ram.keyring ceph.conf ceph.client.manila.keyring ceph.client.test1.keyring rbdmap [ubuntu@magna117 ceph]$ cd [ubuntu@magna117 ~]$ python Python 2.7.5 (default, Feb 20 2018, 09:19:12) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ceph_volume_client >>> vc = ceph_volume_client.CephFSVolumeClient("ram", "/etc/ceph/ceph.conf", "ceph") >>> vc.connect() >>> vc.put_object("cephfs_data", "trial", "Hello") >>> h = vc.get_object("cephfs_data", "trial") >>> assert h == "Hello" Tried with different value, got assert error as expected. >>> assert h == "Hello1" Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError >>> h = vc.delete_object("cephfs_data", "trial") Expected failure since object is deleted. >>> h = vc.get_object("cephfs_data", "trial") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1383, in get_object bytes_read = ioctx.read(object_name, max_size) File "rados.pyx", line 498, in rados.requires.wrapper.validate_func (/builddir/build/BUILD/ceph-12.2.4/build/src/pybind/rados/pyrex/rados.c:4651) File "rados.pyx", line 2695, in rados.Ioctx.read (/builddir/build/BUILD/ceph-12.2.4/build/src/pybind/rados/pyrex/rados.c:29537) rados.ObjectNotFound: [errno 2] Ioctx.read(cephfs_data): failed to read trial Tried creating one more object which worked as expected. >>> vc.put_object("cephfs_data", "test", "ceph") >>> h = vc.get_object("cephfs_data", "test") >>> assert h == "Hello" Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError >>> assert h == "ceph" >>> h = vc.delete_object("cephfs_data", "test") >>>
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1259