Bug 1886020

You need to use the 'cinder-manage' CLI tool to delete stale cinder-volume services leftover after deleting a DCN site that added them. Unfortunately, that command is part of cinder RPM package, and not the cinderclient. This means the command is only available from within containers running a cinder service (api, scheduler, volume or backup). But here are the details captured from a regular overcloud deployment:

()[root@overcloud-controller-0 /]# cinder-manage service remove --help
usage: cinder-manage service remove [-h] binary host_name

positional arguments:
  binary      Service to delete from the host.
  host_name   Host from which to remove the service.

optional arguments:
  -h, --help  show this help message and exit

()[root@overcloud-controller-0 /]# cinder-manage service list
Binary           Host                                 Zone             Status     State Updated At           RPC Version  Object Version  Cluster
cinder-scheduler overcloud-controller-0               nova             enabled    :-)   2020-10-07 19:24:43  3.11         1.38
cinder-volume    hostgroup@tripleo_iscsi              nova             enabled    :-)   2020-10-07 19:24:42  3.16         1.38

So in this example, the command would be:

% cinder-manage service remove cinder-volume hostgroup@tripleo_iscsi

Obviously the host_name argument will be unique for each site. Because cinder-volume will be running A/A, the command will need to be executed for each of the three c-vol hosts at the site.

Given the number of post-delete steps, that should be a subject of lifecycle automation for DF

(In reply to Bogdan Dobrelya from comment #9)
> Given the number of post-delete steps, that should be a subject of lifecycle
> automation for DF

Once this process is well defined a playbook could be written to do this. Either way we need to document the process.

From a cinder perspective, deleting an edge site is analogous to removing a cinder backend from the overcloud deployment. Any resource associated with the entity being removed (edge site or cinder backend) will also need to be removed. Simply removing the edge site or cinder backend from a deployment does not trigger any sort of automatic cleanup of resources associated with the site/backend.

I don't have an exhaustive list in mind (and I haven't been involved in customer cases where a cinder backend was removed), but the main things that come to mind that need to be cleaned up are the volumes and their snapshots, and the cinder-volume services.

- Volumes and snapshots should be deleted prior to deleting an edge site
- cinder-volume services should be disabled prior to deleting an edge site

cinder-volume services associated with the edge site will remain after the site is deleted (the status would be "down" and "disabled"). The cinder-manage CLI may be used to actually delete the stale services, but that command is included in the openstack-cinder package and NOT the cinderclient package. That means the only way to access the cinder-manage command is from within a cinder container. There may be other methods of deleting services (openstack-sdk?), but I've not investigated this.

I have no idea what sort of issues may pertain to glance images.

From a TLS-E perspective, there should be no difference between whether there are multiple stacks or a single stack.

Assuming that the deletion of a particular stack goes through the same client API code to delete a single stack, then
this will trigger the code to clean up ipa for that stack.