Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

This project is now read‑only. Starting Monday, February 2, please use https://ibm-ceph.atlassian.net/ for all bug tracking management.

Bug 1979445

Summary:	Two containers are created per gateway node when we deploy iscsi services via iscsi-gw.yml file
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Gopi <gpatta>
Component:	iSCSI	Assignee:	Ilya Dryomov <idryomov>
Status:	CLOSED NOTABUG	QA Contact:	Gopi <gpatta>
Severity:	low	Docs Contact:
Priority:	low
Version:	5.0	CC:	bniver, ceph-eng-bugs, ceph-qe-bugs, mmurthy, sewagner, vereddy
Target Milestone:	---	Keywords:	Reopened
Target Release:	5.0z1	Flags:	gpatta: needinfo+ gpatta: needinfo+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Known Issue
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-09-18 20:26:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Gopi 2021-07-06 04:53:11 UTC

Description of problem:

I deployed iscsi gateways using iscsi.yml file and i can see two gateways per node.


Version-Release number of selected component (if applicable):
ceph version 16.2.0-88.el8cp (39884f5728635903be37a5fa873e4ef9d8a0b04a) pacific (stable)

How reproducible:
100%

Steps to Reproduce:
1. Deploy a cluster using cephadm
2. Deploy iscsi gateways using iscsi.yml file
[ceph: root@magna007 ~]# cat iscsi.yml 
service_type: iscsi
service_id: iscsi
placement:
  hosts:
  - magna108
  - magna113
spec:
  pool: iscsi_pool
  trusted_ip_list: "10.8.128.108,10.8.128.113"
  api_user: admin
  api_password: admin
[ceph: root@magna007 ~]#

3. Go to gateway nodes and list out the containers using "podman ps"


Actual results:
Two containers seen per gateway node.
[root@magna108 ubuntu]# podman ps
CONTAINER ID  IMAGE                                                                                                                         COMMAND               CREATED      STATUS          PORTS   NAMES
6fb3438649f4  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa627b6861c456eb2011c3374d969d594bddbdee6fd1565d006cec4981df444a  -n client.crash.m...  6 hours ago  Up 6 hours ago          ceph-3ce40d5c-dd5a-11eb-8a7a-002590fc2538-crash.magna108
12e38d148b25  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa627b6861c456eb2011c3374d969d594bddbdee6fd1565d006cec4981df444a                        6 hours ago  Up 6 hours ago          ceph-3ce40d5c-dd5a-11eb-8a7a-002590fc2538-iscsi.iscsi.magna108.kkxugr-tcmu
c05a1dd59f40  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa627b6861c456eb2011c3374d969d594bddbdee6fd1565d006cec4981df444a                        6 hours ago  Up 6 hours ago          ceph-3ce40d5c-dd5a-11eb-8a7a-002590fc2538-iscsi.iscsi.magna108.kkxugr
[root@magna108 ubuntu]#

Expected results:
One gateway container should be there per gateway node.

Additional info:

Comment 2 Gopi 2021-07-07 10:18:00 UTC

Hi Brett,

Actually, we got this issue while working with another BZ:https://bugzilla.redhat.com/show_bug.cgi?id=1979449 . Moving this bug to 5.0z1 as this is not a blocking/regression

Comment 3 Sebastian Wagner 2021-07-07 15:22:49 UTC

cephadm deploys iscsi within two containers since June 2020: 

https://github.com/ceph/ceph/commit/eb604d3338b32a7a11e36e05df6556dbd95ac4a0#diff-d0f7acffbce59b9e36a1479d1b1f32955cd811ab8b0b1aa0018a2003f0c4d73eR486 

Thus, this is nothing new. Is there an issue with deploying iscsi using this setup?

Comment 4 Gopi 2021-07-08 06:49:51 UTC

 We haven't seen any issues till now, but we might face issues in future for iscsi with current cephadm design for iscsi containers, please see https://bugzilla.redhat.com/show_bug.cgi?id=1976128#c21

Comment 5 Sebastian Wagner 2021-08-09 09:56:00 UTC

Closing this issue, cause it's bad practice to run multiple programs within a single container. Fixing this would introduce a new bad practice into cephadm. Bad idea.

Comment 6 Gopi 2021-08-09 12:18:50 UTC

Hi Sebastian,

I can see only one iscsi service handling two containers, i mean if i stop iscsi service then both the containers being stopped. Is it expected that one service handles two containers?

 [root@magna083 ubuntu]# podman ps
CONTAINER ID  IMAGE                                                                                                                         COMMAND               CREATED     STATUS         PORTS   NAMES
afa15b8fc084  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a                        3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-iscsi-iscsi-magna083-zainbl-tcmu
7e15f7e33d64  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a  -n client.crash.m...  3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-crash-magna083
97aee5a6d297  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a                        3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-iscsi-iscsi-magna083-zainbl
d82e9d5ef668  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a  -n osd.14 -f --se...  3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-osd-14
25db429e790a  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a  -n osd.2 -f --set...  3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-osd-2
f0f1453ab2ed  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a  -n osd.8 -f --set...  3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-osd-8
[root@magna083 ubuntu]# 
[root@magna083 ubuntu]# systemctl list-units --all --type=service ceph*
UNIT                                                                          LOAD   ACTIVE SUB     DESCRIPTION                                                             
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.service              loaded active running Ceph crash.magna083 for f9faa6b2-ee9e-11eb-9821-002590fc2536            
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.magna083.zainbl.service loaded active running Ceph iscsi.iscsi.magna083.zainbl for f9faa6b2-ee9e-11eb-9821-002590fc2536
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.service                      loaded active running Ceph osd.14 for f9faa6b2-ee9e-11eb-9821-002590fc2536                    
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.service                       loaded active running Ceph osd.2 for f9faa6b2-ee9e-11eb-9821-002590fc2536                     
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.service                       loaded active running Ceph osd.8 for f9faa6b2-ee9e-11eb-9821-002590fc2536                     

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

5 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.

[root@magna083 ubuntu]# systemctl stop ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.magna083.zainbl.service
[root@magna083 ubuntu]# systemctl list-units --all --type=service ceph*
UNIT                                                                          LOAD   ACTIVE   SUB     DESCRIPTION                                                           
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.service              loaded active   running Ceph crash.magna083 for f9faa6b2-ee9e-11eb-9821-002590fc2536          
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.magna083.zainbl.service loaded inactive dead    Ceph iscsi.iscsi.magna083.zainbl for f9faa6b2-ee9e-11eb-9821-002590fc2>
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.service                      loaded active   running Ceph osd.14 for f9faa6b2-ee9e-11eb-9821-002590fc2536                  
ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536.service                       loaded active   running Ceph osd.8 for f9faa6b2-ee9e-11eb-9821-002590fc2536                   

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

4 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.

[root@magna083 ubuntu]# podman ps
CONTAINER ID  IMAGE                                                                                                                         COMMAND               CREATED     STATUS         PORTS   NAMES
7e15f7e33d64  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a  -n client.crash.m...  3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-crash-magna083
d82e9d5ef668  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a  -n osd.14 -f --se...  3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-osd-14
f0f1453ab2ed  registry-proxy.engineering.redhat.com/rh-osbs/rhceph@sha256:aa77269c3d0be39211c3b2edb0ce5d71350c424038dae860c1f3c1464e4ca95a  -n osd.8 -f --set...  3 days ago  Up 3 days ago          ceph-f9faa6b2-ee9e-11eb-9821-002590fc2536-osd-8

If this is expected and not cause any issues going further then we can close this bug.

Comment 7 Sebastian Wagner 2021-08-09 12:49:35 UTC

For now the time being: yes it is.