Bug 1462527 - Trying to allow/deny access to a manila share fails with CephFS Native
Trying to allow/deny access to a manila share fails with CephFS Native
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-manila (Show other bugs)
10.0 (Newton)
Unspecified Unspecified
medium Severity medium
: z4
: 10.0 (Newton)
Assigned To: Tom Barron
Dustin Schoenbrun
Don Domingo
: TestOnly, Triaged, ZStream
Depends On: 1462259
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-18 09:43 EDT by Tom Barron
Modified: 2017-09-06 13:06 EDT (History)
5 users (show)

See Also:
Fixed In Version: python-cephfs-10.2.7-27.el7cp
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1462259
Environment:
Last Closed: 2017-09-06 13:06:29 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Tom Barron 2017-06-18 09:43:44 EDT
+++ This bug was initially created as a clone of Bug #1462259 +++

Description of problem:

If one tries to allow access to a user and this user's access_key is dirty somehow, manila will show the share access status as "error" and the share.log will show:

------------------------------------

2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server [req-a5b409f8-4d1f-487a-a887-3dd6e000c840 05e228f17d8b426b9071b1ee0857b814 c8efa291fa31410fb999251c4c66d5af - - -] Exception during message handling
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 167, in wrapped
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     return f(self, *args, **kwargs)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/utils.py", line 519, in wrapper
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     return func(self, *args, **kwargs)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 3073, in update_access
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     share_server=share_server)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 280, in update_access_rules
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     share_server=share_server)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 319, in _update_access_rules
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     share_server)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 380, in _update_rules_through_share_driver
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     share_server=share_server
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 292, in update_access
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     access_key = self._allow_access(context, share, rule)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 241, in _allow_access
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     tenant_id=share['project_id'])
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 973, in authorize
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     key = self._authorize_volume(volume_path, auth_id, readonly)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1009, in _authorize_volume
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     key = self._authorize_ceph(volume_path, auth_id, readonly)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1086, in _authorize_ceph
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     'mon', cap['caps'].get('mon')]
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1275, in _rados_command
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server     raise rados.Error(outs)
2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server Error: Can't handle arrays of non-strings

------------------------------------

Then, when trying to deny access to the user, nothing happens, but share.log show the following error: 

------------------------------------

2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server [req-16ac61ab-1c5d-4cab-b685-e4eb030380e1 05e228f17d8b426b9071b1ee0857b814 c8efa291fa31410fb999251c4c66d5af - - -] Exception during message handling
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 167, in wrapped
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     return f(self, *args, **kwargs)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/utils.py", line 519, in wrapper
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     return func(self, *args, **kwargs)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 3073, in update_access
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     share_server=share_server)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 280, in update_access_rules
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     share_server=share_server)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 319, in _update_access_rules
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     share_server)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 380, in _update_rules_through_share_driver
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     share_server=share_server
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 296, in update_access
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     self._deny_access(context, share, rule)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 253, in _deny_access
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     access['access_to'])
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1133, in deauthorize
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server     self._recover_auth_meta(auth_id, auth_meta)
2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server TypeError: _recover_auth_meta() takes exactly 2 arguments (3 given)

------------------------------------


Version-Release number of selected component (if applicable):

puppet-manila-10.3.0-1.el7ost.noarch
openstack-manila-share-4.0.0-4.el7ost.noarch
python-manilaclient-1.14.0-1.el7ost.noarch
python-manila-4.0.0-4.el7ost.noarch
openstack-manila-4.0.0-4.el7ost.noarch
openstack-manila-ui-2.7.1-2.el7ost.noarch

Most relevant to the problem:
python-cephfs-10.2.5-37.el7cp.x86_64


Steps to Reproduce:
1. Create a user using the following command (note that alice could be any other user):

ceph --name=client.manila --keyring=/etc/ceph/ceph.client.manila.keyring auth get-or-create client.alice -o alice.keyring

2. Try to allow that user to have access to the share:

manila access-allow <share_name> cephx alice 

3. Run 'manila access-list <share>' and see that the share is in 'error' state

4. Try to run 'manila access-deny <share> <access-id>'


Actual results:

Share access in error state and error trace in /var/log/manila/share.log (in the node that runs manila share service)

Expected results:

Access removed successfully

---

Additional info:

The solution for this problem was addressed 8 months ago and is included in python-cephfs-10.2.7-27.el7cp.x86_64.rpm - we should make sure RHOS11 includes the package with the bugfix, or maybe backport the fix.
Comment 1 Tom Barron 2017-06-18 09:44:32 EDT
We have the same issue in OSP 10:

[root@compute-0 ~]# yum provides /usr/lib/python2.7/site-packages/ceph_volume_client.py
Loaded plugins: product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
1:python-cephfs-10.2.5-37.el7cp.x86_64 : Python libraries for Ceph distributed file system
Repo        : @rhos-10.0-ceph-2.0-mon-signed
Matched from:
Filename    : /usr/lib/python2.7/site-packages/ceph_volume_client.py
Comment 2 Red Hat Bugzilla Rules Engine 2017-06-19 08:32:39 EDT
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.
Comment 5 Dustin Schoenbrun 2017-08-25 15:41:24 EDT
Using puddle version 2017-08-22.2 I was able to log into the undercloud node in a deploy and verify that the python-cephfs-10.2.7-27.el7cp.x86_64.rpm package (or greater) is available.
Comment 7 errata-xmlrpc 2017-09-06 13:06:29 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2653

Note You need to log in before you can comment on or make changes to this bug.