Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
This project is now read‑only. Starting Monday, February 2, please use https://ibm-ceph.atlassian.net/ for all bug tracking management.

Bug 1980808

Summary: cephadm: remove iscsi service fails due to incorrect gateway name
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Dimitri Savineau <dsavinea>
Component: CephadmAssignee: Sebastian Wagner <sewagner>
Status: CLOSED DUPLICATE QA Contact: Vasishta <vashastr>
Severity: medium Docs Contact: Karen Norteman <knortema>
Priority: unspecified    
Version: 5.0Keywords: Regression
Target Milestone: ---   
Target Release: 5.1   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-08-05 08:57:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dimitri Savineau 2021-07-09 15:12:27 UTC
Description of problem:

If the iscsi service is removed and the dashboard is deployed (dashboard mgr module enabled) then the cluster status goes to ERR and the removal is stuck is deleting state.

Version-Release number of selected component (if applicable):

# ceph --version
ceph version 16.2.0-98.el8cp (9c6352ff5276f8fb2029981206f3516707220054) pacific (stable)
# rpm -qa cephadm
cephadm-16.2.0-98.el8cp.noarch

How reproducible:
100%

Steps to Reproduce:
1. bootstrap a cluster with dashboard : cephadm bootstrap --mon-ip x.x.x.x
2. add some OSDs
3. deploy the iscsi service
4. remove iscsi with : ceph orch rm iscsi.iscsi

Actual results:

# ceph orch ls --service_type iscsi
NAME         RUNNING  REFRESHED   AGE  PLACEMENT  
iscsi.iscsi      0/1  <deleting>  5m   cephaio    

# ceph health detail
HEALTH_ERR Module 'cephadm' has failed: dashboard iscsi-gateway-rm failed: iSCSI gateway 'cephaio' does not exist retval: -2
[ERR] MGR_MODULE_ERROR: Module 'cephadm' has failed: dashboard iscsi-gateway-rm failed: iSCSI gateway 'cephaio' does not exist retval: -2
    Module 'cephadm' has failed: dashboard iscsi-gateway-rm failed: iSCSI gateway 'cephaio' does not exist retval: -2

Expected results:

The iscsi service should be removed correctly and the cluster status should be HEALTH_OK

Additional info:

This is a regression introduced by [1] which added the `ceph dashboard iscsi-gateway-rm xxx` command during the service removal but that operation doesn't use the same gateway name than used for adding the service.

# ceph dashboard iscsi-gateway-list
{"gateways": {"ceph-db307f56-e0c3-11eb-9d22-fa163e20e390-iscsi.iscsi.cephaio.rberjq": {"service_url": "http://admin:+xFRe+RES@7vg24n@192.168.100.14:5000"}}}

ceph-db307f56-e0c3-11eb-9d22-fa163e20e390-iscsi.iscsi.cephaio.rberjq is the container name while cephaio is the hostname of the node running the container.

# podman exec ceph-db307f56-e0c3-11eb-9d22-fa163e20e390-iscsi.iscsi.cephaio.rberjq python3 -c 'import socket; print(socket.getfqdn())'
ceph-db307f56-e0c3-11eb-9d22-fa163e20e390-iscsi.iscsi.cephaio.rberjq
# podman exec ceph-db307f56-e0c3-11eb-9d22-fa163e20e390-iscsi.iscsi.cephaio.rberjq python3 -c 'import socket; print(socket.gethostname())'
cephaio
# podman exec ceph-db307f56-e0c3-11eb-9d22-fa163e20e390-iscsi.iscsi.cephaio.rberjq cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.100.14 cephaio
127.0.1.1 cephaio cephaio ceph-db307f56-e0c3-11eb-9d22-fa163e20e390-iscsi.iscsi.cephaio.rberjq


This change is present upstream since v16.2.5 but has been cherry-picked downstream.

[1] https://github.com/ceph/ceph/commit/1b9e3edcfd1c1a3dd02d6eb14072494f57b086a8

Comment 3 Sebastian Wagner 2021-08-05 08:57:46 UTC

*** This bug has been marked as a duplicate of bug 1979449 ***