Description of problem: There is a failure to create a manila share in the ceph environments (CephFS-Native and CephFS-NFS-Ganesha). create-share requests get to the manila-share service and then the child process handling the cephfs back end is killed for reasons not yet known and then gets restarted and reinitialized. Version-Release number of selected component (if applicable): puppet-manila-15.5.0-2.20210601014536.9c6604a.el8ost.2.noarch puppet-manila-15.5.0-2.20210601014536.9c6604a.el8ost.2.noarch puppet-ceph-3.1.2-2.20210603181657.ffa80da.el8ost.1.noarch ceph-ansible-4.0.57-1.el8cp.noarch How reproducible: 100% Steps to Reproduce: (overcloud) [stack@undercloud-0 ~]$ manila type-list +--------------------------------------+-----------+------------+------------+--------------------------------------+-------------------------------------+-------------+ | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | +--------------------------------------+-----------+------------+------------+--------------------------------------+-------------------------------------+-------------+ | 21a84950-d1d7-4bd4-9fd1-e410091e7479 | default | public | YES | driver_handles_share_servers : False | snapshot_support : True | None | +--------------------------------------+-----------+------------+------------+--------------------------------------+-------------------------------------+-------------+ (overcloud) [stack@undercloud-0 ~]$ manila create nfs 1 +---------------------------------------+--------------------------------------+ | Property | Value | +---------------------------------------+--------------------------------------+ | id | 4064a7e9-8827-45fe-a88c-9e1550a49cd4 | | size | 1 | | availability_zone | None | | created_at | 2021-06-28T04:02:26.000000 | | status | creating | | name | None | | description | None | | project_id | d0d3edbb67754c01a4b71f0b379ca120 | | snapshot_id | None | | share_network_id | None | | share_proto | NFS | | metadata | {} | | share_type | 21a84950-d1d7-4bd4-9fd1-e410091e7479 | | is_public | False | | snapshot_support | True | | task_state | None | | share_type_name | default | | access_rules_status | active | | replication_type | None | | has_replicas | False | | user_id | 345024b1d79c4a1780d88f796575097c | | create_share_from_snapshot_support | False | | revert_to_snapshot_support | False | | share_group_id | None | | source_share_group_snapshot_member_id | None | | mount_snapshot_support | False | | share_server_id | None | | host | | +---------------------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ manila list +--------------------------------------+------+------+-------------+----------+-----------+-----------------+-------------------------+-------------------+ | ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone | +--------------------------------------+------+------+-------------+----------+-----------+-----------------+-------------------------+-------------------+ | 4064a7e9-8827-45fe-a88c-9e1550a49cd4 | None | 1 | NFS | creating | False | default | hostgroup@cephfs#cephfs | nova | +--------------------------------------+------+------+-------------+----------+-----------+-----------------+-------------------------+-------------------+ Actual results: Creating a manila share fails and gets stuck in "creating" status. Expected results: Creating a manila share should be successful. Additional info: 2021-06-27 10:22:00.278 290 DEBUG manila.share.drivers.cephfs.driver [req-d9e9bf29-7254-4ae8-8739-6256298b768d 345024b1d79c4a1780d88f796575097c d0d3edbb67754c01a4b71f0b379ca120 - - -] create_share cephfs name=8dc0239c-1aef-4dc5-8647-f7bde9db4f66 size=1 share_group_id=None create_share /usr/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py:262 2021-06-27 10:22:00.288 290 INFO ceph_volume_client [req-d9e9bf29-7254-4ae8-8739-6256298b768d 345024b1d79c4a1780d88f796575097c d0d3edbb67754c01a4b71f0b379ca120 - - -] create_volume: /volumes/_nogroup/8dc0239c-1aef-4dc5-8647-f7bde9db4f66 2021-06-27 10:22:01.017 7 INFO oslo_service.service [req-54e69491-1385-4065-9f86-a77c606d65c6 - - - - -] Child 290 killed by signal 6 2021-06-27 10:22:01.023 7 DEBUG oslo_service.service [req-54e69491-1385-4065-9f86-a77c606d65c6 - - - - -] Started child 328 _start_child /usr/lib/python3.6/site-packages/oslo_service/service.py:579
Is that a regression? Isn't marked as such, but I assume it worked in previous releases?
(In reply to Yaniv Kaul from comment #3) > Is that a regression? Isn't marked as such, but I assume it worked in > previous releases? Liron told me in email "I checked the earlier 16.2 phase3 run and we used ceph tag 4-44, here we used 4-55" so it used to work in 16.2 candidate code (and works in earlier versions). Vida reported "Apparently the Ceph DFG job running the same pkg version had similar tempest failures." I'm leaving the NEEDINFO in case QE has additional information but yeah, I think this is a regression and should be marked as such.
(In reply to Yaniv Kaul from comment #3) > Is that a regression? Isn't marked as such, but I assume it worked in > previous releases? Right, in the previous phase 3 runs it's all been fine.
Verification steps: (overcloud) [stack@undercloud-0 ~]$ manila type-list +--------------------------------------+---------+------------+------------+--------------------------------------+-------------------------+-------------+ | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | +--------------------------------------+---------+------------+------------+--------------------------------------+-------------------------+-------------+ | 2649e34a-35b7-478f-a52f-cfe57523cedb | default | public | YES | driver_handles_share_servers : False | snapshot_support : True | None | +--------------------------------------+---------+------------+------------+--------------------------------------+-------------------------+-------------+ (overcloud) [stack@undercloud-0 ~]$ manila create nfs 1 +---------------------------------------+--------------------------------------+ | Property | Value | +---------------------------------------+--------------------------------------+ | id | b885b298-4693-416e-822f-21472cb4c622 | | size | 1 | | availability_zone | None | | created_at | 2021-07-15T09:46:14.000000 | | status | creating | | name | None | | description | None | | project_id | 5a25ffa19919411bb9c4a9405d8a677b | | snapshot_id | None | | share_network_id | None | | share_proto | NFS | | metadata | {} | | share_type | 2649e34a-35b7-478f-a52f-cfe57523cedb | | is_public | False | | snapshot_support | True | | task_state | None | | share_type_name | default | | access_rules_status | active | | replication_type | None | | has_replicas | False | | user_id | abd23c0cdc894c35849812e9e194bc21 | | create_share_from_snapshot_support | False | | revert_to_snapshot_support | False | | share_group_id | None | | source_share_group_snapshot_member_id | None | | mount_snapshot_support | False | | share_server_id | None | | host | | +---------------------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ manila list +--------------------------------------+------+------+-------------+-----------+-----------+-----------------+-------------------------+-------------------+ | ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone | +--------------------------------------+------+------+-------------+-----------+-----------+-----------------+-------------------------+-------------------+ | b885b298-4693-416e-822f-21472cb4c622 | None | 1 | NFS | available | False | default | hostgroup@cephfs#cephfs | nova | +--------------------------------------+------+------+-------------+-----------+-----------+-----------------+-------------------------+-------------------+