Bug 1462259
| Summary: | Trying to allow/deny access to a manila share fails with CephFS Native | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Raissa Sarmento <rdearauj> | |
| Component: | openstack-manila | Assignee: | Tom Barron <tbarron> | |
| Status: | CLOSED ERRATA | QA Contact: | Dustin Schoenbrun <dschoenb> | |
| Severity: | medium | Docs Contact: | Don Domingo <ddomingo> | |
| Priority: | medium | |||
| Version: | 11.0 (Ocata) | CC: | bniver, jjoyce, scohen, slinaber, tbarron | |
| Target Milestone: | z2 | Keywords: | Triaged, ZStream | |
| Target Release: | 11.0 (Ocata) | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1462527 (view as bug list) | Environment: | ||
| Last Closed: | 2017-09-13 21:50:42 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1462527 | |||
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release. @Tom, we need to move this to either 12 or 13 if there's any CephFS work required, correct? This should be easy to verify with OSP-11 as the fix should be included in one of the latest builds. I have an environment set up to see if the new version was installed with the latest OSP-11 puddle and will report back accordingly. Using the latest OSP-11 puddle (2017-08-02.1), I was able to verify that the correct version of the python-cephfs package was installed on the controller and ceph nodes in the overcloud and that I was able to successfully grant access to a cephx ID as well as deny access to the same without issue. Logs are below for the granting and denying of access: 2017-08-09 21:33:43.974 195420 INFO ceph_volume_client [req-74209b11-13f6-4eac-8333-e31594e65589 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] create_volume: /volumes/_nogroup/8ef47735-efe1-4783-890c-a52bb4221d3f 2017-08-09 21:33:44.815 195420 INFO ceph_volume_client [req-74209b11-13f6-4eac-8333-e31594e65589 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] create_volume: None/8ef47735-efe1-4783-890c-a52bb4221d3f, using rados namespace fsvolumens_8ef47735-efe1-4783-890c-a52bb4221d3f to isolate data. 2017-08-09 21:33:44.838 195420 INFO ceph_volume_client [req-74209b11-13f6-4eac-8333-e31594e65589 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] get_mon_addrs 2017-08-09 21:33:44.902 195420 INFO manila.share.drivers.cephfs.cephfs_native [req-74209b11-13f6-4eac-8333-e31594e65589 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] Calculated export location for share 8ef47735-efe1-4783-890c-a52bb4221d3f: 172.17.3.11:6789,172.17.3.14:6789,172.17.3.23:6789:/volumes/_nogroup/8ef47735-efe1-4783-890c-a52bb4221d3f 2017-08-09 21:33:45.188 195420 INFO manila.share.manager [req-74209b11-13f6-4eac-8333-e31594e65589 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] Share instance 8ef47735-efe1-4783-890c-a52bb4221d3f created successfully. 2017-08-09 21:34:26.047 195420 INFO manila.share.manager [req-9572830f-4fdd-4828-96b9-c56d54fa5136 - - - - -] Updating share status 2017-08-09 21:34:44.670 195420 INFO manila.share.access [req-9d315340-a32e-4128-8aaa-e14567eb158a 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] Access rules were successfully modified for share instance 8ef47735-efe1-4783-890c-a52bb4221d3f belonging to share 4cc1e0de-6747-42dc-b58c-4af3493531a7. 2017-08-09 21:35:23.295 195420 INFO ceph_volume_client [req-f3357769-3d14-46f1-b373-4396781ee56f 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] evict clients with auth_name=alice, client_metadata.root=/volumes/_nogroup/8ef47735-efe1-4783-890c-a52bb4221d3f 2017-08-09 21:35:23.507 195420 INFO ceph_volume_client [req-f3357769-3d14-46f1-b373-4396781ee56f 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] evict: joined all 2017-08-09 21:35:23.783 195420 INFO manila.share.access [req-f3357769-3d14-46f1-b373-4396781ee56f 06ad0f74a2c74f6cb652a6ca593a7d0f 2112b6124a0941af8bf4e0806c681f31 - - -] Access rules were successfully modified for share instance 8ef47735-efe1-4783-890c-a52bb4221d3f belonging to share 4cc1e0de-6747-42dc-b58c-4af3493531a7. (In reply to Brett Niver from comment #4) > @Tom, we need to move this to either 12 or 13 if there's any CephFS work > required, correct? Dustin has verified that we have the fix in 11 so we don't need to move it out to a later release. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:2722 |
Description of problem: If one tries to allow access to a user and this user's access_key is dirty somehow, manila will show the share access status as "error" and the share.log will show: ------------------------------------ 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server [req-a5b409f8-4d1f-487a-a887-3dd6e000c840 05e228f17d8b426b9071b1ee0857b814 c8efa291fa31410fb999251c4c66d5af - - -] Exception during message handling 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 167, in wrapped 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/utils.py", line 519, in wrapper 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 3073, in update_access 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server share_server=share_server) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 280, in update_access_rules 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server share_server=share_server) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 319, in _update_access_rules 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server share_server) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 380, in _update_rules_through_share_driver 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server share_server=share_server 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 292, in update_access 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server access_key = self._allow_access(context, share, rule) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 241, in _allow_access 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server tenant_id=share['project_id']) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 973, in authorize 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server key = self._authorize_volume(volume_path, auth_id, readonly) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1009, in _authorize_volume 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server key = self._authorize_ceph(volume_path, auth_id, readonly) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1086, in _authorize_ceph 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server 'mon', cap['caps'].get('mon')] 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1275, in _rados_command 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server raise rados.Error(outs) 2017-06-16 12:28:19.179 193387 ERROR oslo_messaging.rpc.server Error: Can't handle arrays of non-strings ------------------------------------ Then, when trying to deny access to the user, nothing happens, but share.log show the following error: ------------------------------------ 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server [req-16ac61ab-1c5d-4cab-b685-e4eb030380e1 05e228f17d8b426b9071b1ee0857b814 c8efa291fa31410fb999251c4c66d5af - - -] Exception during message handling 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 167, in wrapped 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/utils.py", line 519, in wrapper 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 3073, in update_access 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server share_server=share_server) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 280, in update_access_rules 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server share_server=share_server) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 319, in _update_access_rules 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server share_server) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/access.py", line 380, in _update_rules_through_share_driver 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server share_server=share_server 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 296, in update_access 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server self._deny_access(context, share, rule) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 253, in _deny_access 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server access['access_to']) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 1133, in deauthorize 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server self._recover_auth_meta(auth_id, auth_meta) 2017-06-16 12:30:56.036 193387 ERROR oslo_messaging.rpc.server TypeError: _recover_auth_meta() takes exactly 2 arguments (3 given) ------------------------------------ Version-Release number of selected component (if applicable): puppet-manila-10.3.0-1.el7ost.noarch openstack-manila-share-4.0.0-4.el7ost.noarch python-manilaclient-1.14.0-1.el7ost.noarch python-manila-4.0.0-4.el7ost.noarch openstack-manila-4.0.0-4.el7ost.noarch openstack-manila-ui-2.7.1-2.el7ost.noarch Most relevant to the problem: python-cephfs-10.2.5-37.el7cp.x86_64 Steps to Reproduce: 1. Create a user using the following command (note that alice could be any other user): ceph --name=client.manila --keyring=/etc/ceph/ceph.client.manila.keyring auth get-or-create client.alice -o alice.keyring 2. Try to allow that user to have access to the share: manila access-allow <share_name> cephx alice 3. Run 'manila access-list <share>' and see that the share is in 'error' state 4. Try to run 'manila access-deny <share> <access-id>' Actual results: Share access in error state and error trace in /var/log/manila/share.log (in the node that runs manila share service) Expected results: Access removed successfully --- Additional info: The solution for this problem was addressed 8 months ago and is included in python-cephfs-10.2.7-27.el7cp.x86_64.rpm - we should make sure RHOS11 includes the package with the bugfix, or maybe backport the fix.