Bug 1947087

Summary:	[workload-dfg] Adding MONs via spec file removed bootstrap node
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Tim Wilkinson <twilkins>
Component:	Cephadm	Assignee:	Adam King <adking>
Status:	CLOSED ERRATA	QA Contact:	Rahul Lepakshi <rlepaksh>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	5.0	CC:	adking, agunn, jolmomar, racpatel, rlepaksh, tchandra, vereddy, vumrao
Target Milestone:	---
Target Release:	5.1
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	ceph-16.2.6-1.el8cp	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-04-04 10:20:36 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Comment 1 Juan Miguel Olmo 2021-04-07 17:10:26 UTC

The procedure followed as you has pointed in the description of the bug is not correct:

Steps to Reproduce:
==================
1. Bootstrap cluster, verify bootstrap node is acting MON/MGR   <------------------ you only have 1 node where only 1 monitor is running.
2. ceph orch apply -i mon_spec.yml  <---------------------------------------------- Wrong: You need first to add the hosts to the cluster 
3. Attempt to connect to cluster using bootstrap node   


Please use the following procedure:

1. Bootstrap cluster, verify bootstrap node is acting MON/MG
2. connect to cluster using bootstrap node   
3. Add new hosts
Once you have all the nodes in the cluster
4. apply your mon.spec file <----- but if you have set labels in your nodes (seems to be the case) .... is more easy to do "ceph orch apply mon label:monmgr


Note: if you move the monitor outside the bootstrap node you are not going to be able to access the ceph shell.

Comment 3 Juan Miguel Olmo 2021-04-08 09:08:33 UTC

Ok. thanks for the update.

What is the name of the bootstrap node?

Please attach the following files:

host_spec.yml
osd_spec.yml
rgw_spec.yml
mon_spec.yml (if it is different from the one you included in the bug description)
mgr.log

Comment 13 Juan Miguel Olmo 2021-04-20 09:34:06 UTC

Adam: can you check if this issue is a duplicate of:

https://bugzilla.redhat.com/show_bug.cgi?id=1945272

Comment 14 Adam King 2021-04-20 14:07:33 UTC

@JuanMi No, that bug only caused a nonexistent monitor to be marked as stray due to some data not being cleared in the mgr. That bug has nothing to do with monitors actually being added or removed incorrectly, it is purely an issue of reporting incorrect information.

Comment 17 Sebastian Wagner 2021-05-31 11:23:00 UTC

So, are we still looking for the cause described in the issue description? I'm a bit confused here by now

Comment 28 errata-xmlrpc 2022-04-04 10:20:36 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat Ceph Storage 5.1 Security, Enhancement, and Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1174