Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1862931

Summary: Share got stuck on "error_deleting" status after restarting manila share service
Product: Red Hat OpenStack Reporter: lkuchlan <lkuchlan>
Component: openstack-manilaAssignee: Tom Barron <tbarron>
Status: CLOSED INSUFFICIENT_DATA QA Contact: lkuchlan <lkuchlan>
Severity: medium Docs Contact: RHOS Documentation Team <rhos-docs>
Priority: medium    
Version: 17.0 (Wallaby)CC: gcharot, gfidente, gouthamr, vhariria, vimartin
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-01-11 19:30:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Manila logs none

Description lkuchlan 2020-08-03 09:04:57 UTC
Description of problem:
When trying to delete a manila share after restart manila share service
the share got stuck on "error_deleting" status, although the manila share 
service was up in the delete operation.

Version-Release number of selected component (if applicable):
puppet-manila-15.4.1-0.20200403160104.e41b1b6.el8ost.noarch
python3-manilaclient-1.29.0-0.20200310223441.1b2cafb.el8ost.noarch

How reproducible:
100%

Steps to Reproduce:
(.venv) (overcloud) [stack@undercloud-0 tempest]$ manila create NFS 1
+---------------------------------------+--------------------------------------+
| Property                              | Value                                |
+---------------------------------------+--------------------------------------+
| id                                    | d895f7bf-df9a-46a9-a4e3-dde32da8d55e |
| size                                  | 1                                    |
| availability_zone                     | None                                 |
| created_at                            | 2020-08-03T08:48:27.000000           |
| status                                | creating                             |
| name                                  | None                                 |
| description                           | None                                 |
| project_id                            | c790bd8deb344a3397550acfc22ea4b7     |
| snapshot_id                           | None                                 |
| share_network_id                      | None                                 |
| share_proto                           | NFS                                  |
| metadata                              | {}                                   |
| share_type                            | c775bef9-7e5e-45b5-9234-9d7f81a21640 |
| is_public                             | False                                |
| snapshot_support                      | True                                 |
| task_state                            | None                                 |
| share_type_name                       | default                              |
| access_rules_status                   | active                               |
| replication_type                      | None                                 |
| has_replicas                          | False                                |
| user_id                               | 3548c355531b4eb195dcbcb8774529e8     |
| create_share_from_snapshot_support    | False                                |
| revert_to_snapshot_support            | False                                |
| share_group_id                        | None                                 |
| source_share_group_snapshot_member_id | None                                 |
| mount_snapshot_support                | False                                |
| share_server_id                       | None                                 |
| host                                  |                                      |
+---------------------------------------+--------------------------------------+

(.venv) (overcloud) [stack@undercloud-0 tempest]$ manila list
+--------------------------------------+------+------+-------------+----------------+-----------+-----------------+-------------------------+-------------------+
| ID                                   | Name | Size | Share Proto | Status         | Is Public | Share Type Name | Host                    | Availability Zone |
+--------------------------------------+------+------+-------------+----------------+-----------+-----------------+-------------------------+-------------------+
| d895f7bf-df9a-46a9-a4e3-dde32da8d55e | None | 1    | NFS         | available      | False     | default         | hostgroup@cephfs#cephfs | nova              |
+--------------------------------------+------+------+-------------+----------------+-----------+-----------------+-------------------------+-------------------+

# Restart manila share service.

[root@controller-0 ~]# docker restart openstack-manila-share-podman-0

# Wait until manila share service is up.
 
[root@controller-0 ~]# docker ps | grep openstack-manila-share-podman-0
c8d8cf5fc065  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-manila-share:16.1_20200625.1                /bin/bash /usr/lo...  2 weeks ago  Up 5 minutes ago         openstack-manila-share-podman-0

(.venv) (overcloud) [stack@undercloud-0 tempest]$ manila delete d895f7bf-df9a-46a9-a4e3-dde32da8d55e

(.venv) (overcloud) [stack@undercloud-0 tempest]$ manila list
+--------------------------------------+------+------+-------------+----------------+-----------+-----------------+-------------------------+-------------------+
| ID                                   | Name | Size | Share Proto | Status         | Is Public | Share Type Name | Host                    | Availability Zone |
+--------------------------------------+------+------+-------------+----------------+-----------+-----------------+-------------------------+-------------------+
| d895f7bf-df9a-46a9-a4e3-dde32da8d55e | None | 1    | NFS         | error_deleting | False     | default         | hostgroup@cephfs#cephfs | nova              |
+--------------------------------------+------+------+-------------+----------------+-----------+-----------------+-------------------------+-------------------+

Actual results:
Share got stuck on "error_deleting" status

Expected results:
Share deletion should succeed

Comment 1 lkuchlan 2020-08-03 09:10:53 UTC
Created attachment 1703252 [details]
Manila logs

Comment 2 Victoria Martinez de la Cruz 2020-08-03 13:39:27 UTC
Have you checked pacemaker status? manila-shr is the only service for manila that is under pacemaker control, checking is status might provide it more info to us.

Comment 3 lkuchlan 2020-08-03 15:53:05 UTC
Pacemaker status was up.

Comment 8 Goutham Pacha Ravi 2022-01-11 19:30:10 UTC
(In reply to lkuchlan from comment #7)
> (In reply to Victoria Martinez de la Cruz from comment #6)
> > Liron, is this one still reproducible? Would you be able to check this?
> > Thanks!
> 
> I could not reproduce it.
> The share was deleted successfully after openstack-manila-share-podman-0
> recovery.

Awesome. "error_deleting" can only occur when the share driver fails during the deletion operation, not if the share manager service is unresponsive (stopped, dead, starting up or "not ready" yet). 

Since this isn't reproducible at the moment. I'm closing this bug. If this occurs, we should grab the logs to see what happened with the share driver and report a new bug.