Bug 1222505

From my perspective, ceph-deploy is doing the right thing, and outputing the right info: "[ceph_deploy.install][ERROR ] custom cluster names are not supported on sysvinit hosts"

Ceph Hammer (and RHCS 1.3) does not include systemd scripts that I know of, thus we are using the sysvinit scripts, and thus custom cluster names are not supported.

But that's my perspective - I don't know if that's actually true or not.  :)  Are custom cluster names supposed to be a new feature in 1.3?  Were they supported on RHEL on 1.2.z?  I don't know the answers to those questions.

If custom names are not supported in 1.3 it should have failed when i attempted first time. It allowed first time and when tried re-installing after purge, it failed.

Travis, can we check "if distro.init == 'sysvinit' and args.cluster != 'ceph'" in new.py as we're doing in install.py?

(In reply to Ken Dreyer (Red Hat) from comment #4)
> Travis, can we check "if distro.init == 'sysvinit' and args.cluster !=
> 'ceph'" in new.py as we're doing in install.py?

Yeah, we can do that.  It would have the effect of checking if any of the monitors given in the 'ceph-deploy new' command were sysvinit systems.  If any of them were, it would abort.

I am very curious about what Harish described, though, that it seemed to work the first time, but then didn't on a re-install.  That doesn't seem right to me.

But yes, it is definitely a problem that you can define a {cluster}.conf file with the 'new' coommand that we have the ability to detect is not going to work.  The "new" command does not actually instally anything, it just creates a brand new {cluster}.conf file based on your input, and IP address info it collects from the monitors.  But since we are contacting the monitors, we *can* see if we are going to use sysvinit on them.  :)

Fix committed upstream: https://github.com/ceph/ceph-deploy/commit/63c4ff70ec1a09205a58c7514dbfcdb00b15d975

Hi Travis,

As per my understanding, the fix above is now preventing users to use custom cluster name for "ceph-deploy new <mon>" too. 

Did you get a chance to look into why ceph-deploy allowed first time to install with custom cluster name and not when re-installing? I feel we need to know that before we say this defect is fixed.

Harish

(In reply to Harish NV Rao from comment #7)
> Hi Travis,
> 
> As per my understanding, the fix above is now preventing users to use custom
> cluster name for "ceph-deploy new <mon>" too. 
> 
> Did you get a chance to look into why ceph-deploy allowed first time to
> install with custom cluster name and not when re-installing? I feel we need
> to know that before we say this defect is fixed.
> 
> Harish

Hi Harish,

Your understanding is correct.  I did not look into the other problem you described.  Would you happen to have the exact sequence of commands that you used with ceph-deploy to get into that scenario?

Created attachment 1027284 [details]
.bash_history contents.

.bash_history file

Here are the sequence of commands extracted from .bash_history on Admin node. Few commands are omitted to give clarity.

# go to ceph-config dir
cd ceph-config/
ceph-deploy --cluster ceph2 new c2Mon

# following two commands did not work in ceph-config directory
ceph-deploy install --cluster ceph2 c2Mon c2osd0 c2osd1 c2osd2 #incorrect syntax
ceph-deploy install c2Mon c2osd0 c2osd1 c2osd2 # hung for long time

# so created ceph2-config dir
cd ..
mkdir ceph2-config

# copied all the files related to ceph2 from ceph-config directory to ceph2-config directory
ll ceph-config/
cp ceph-config/ceph2* ceph2-config/.
cd ceph2-config/

# Ran install and mon create-initial commands in ceph2-config directory
ceph-deploy install c2Mon c2osd0 c2osd1 c2osd2
ceph-deploy mon create-initial
ceph-deploy mon create-initial
ceph-deploy --cluster ceph2 mon create-initial

sudo yum -y install salt-minion
sudo mkdir /etc/salt/minion.d

# copied ceph2 related files in ceph2-config dir to ceph-config dir
cp ceph2* ../ceph-config/

Please see the attached file "CmdSequence" for complete history of the commands [Line #471 onwards]

(In reply to Harish NV Rao from comment #10)
> Here are the sequence of commands extracted from .bash_history on Admin
> node. Few commands are omitted to give clarity.
Hi Harish,

I think we are looking at a mixture of too many commands that used and did not use a custom cluster name.  I don't think you ever achieved a working cluster with a custom name. I'll call out a few issues I see...

> 
> # go to ceph-config dir
> cd ceph-config/
> ceph-deploy --cluster ceph2 new c2Mon

When you did this, was this a new ceph-config directory?  Or did it have an existing ceph.conf already in it?  From the history you uploaded, I think you were reusing this.  This becomes important later.

> 
> # following two commands did not work in ceph-config directory
> ceph-deploy install --cluster ceph2 c2Mon c2osd0 c2osd1 c2osd2 #incorrect
> syntax

I don't think this is incorrect syntax. At this point, it should have errored and said that you can't use a custom cluster name on these nodes. The change we've made in this BZ is to catch this in the previous step, with "ceph-deploy new"

> ceph-deploy install c2Mon c2osd0 c2osd1 c2osd2 # hung for long time

I wish I knew why this had hung. If this was run from the ceph-config directory, and "ceph-deploy new" was run here without a custom name, this would have started to install ceph on the remote nodes just fine.

> 
> # so created ceph2-config dir
> cd ..
> mkdir ceph2-config
> 
> # copied all the files related to ceph2 from ceph-config directory to
> ceph2-config directory
> ll ceph-config/
> cp ceph-config/ceph2* ceph2-config/.
> cd ceph2-config/
> 
> # Ran install and mon create-initial commands in ceph2-config directory
> ceph-deploy install c2Mon c2osd0 c2osd1 c2osd2

Just like we don't allow custom cluster names on RHEL/CentOS machines, they can't have more than one cluster on them (same reason). From your history, it seemed like there might have been a previous install on these nodes?

> ceph-deploy mon create-initial
> ceph-deploy mon create-initial
> ceph-deploy --cluster ceph2 mon create-initial

I worried about sequences like this, where --cluster was left out the first time.  If there were any ceph.conf files laying around, they would have been picked up and used by default.

> 
> sudo yum -y install salt-minion
> sudo mkdir /etc/salt/minion.d
> 
> # copied ceph2 related files in ceph2-config dir to ceph-config dir
> cp ceph2* ../ceph-config/
> 
> Please see the attached file "CmdSequence" for complete history of the
> commands [Line #471 onwards]


If you are still concerned about this, I would start fresh and see if it happens again.  But since it's not a scenario we support, I'm not sure if it is worth your time.

Travis, I am sure that it got installed first time with custom cluster name. I am unable to reproduce this now. I am lowering the severity for now. I will retry it one more time with new build on a new system next week. I will close it If issue is not reproducible.

Verified that custom cluster names cannot be used while issuing "ceph-deploy new <mon node>" on a 7.1 system. This defect has been verified for the fix done in https://github.com/ceph/ceph-deploy/commit/63c4ff70ec1a09205a58c7514dbfcdb00b15d975. I don't have cycles now to complete the task as mentioned in Comment 14 above. Marking this defect as Verified for now.

Log:
====
[cephuser@Admin ceph-config]$ ceph-deploy --cluster Test1 new Mon
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephuser/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.25): /usr/bin/ceph-deploy --cluster Test1 new Mon
[ceph_deploy.new][DEBUG ] Creating new cluster named Test1
[ceph_deploy.new][INFO  ] making sure passwordless SSH succeeds
[Mon][DEBUG ] connected to host: Admin 
[Mon][INFO  ] Running command: ssh -CT -o BatchMode=yes Mon
[Mon][DEBUG ] connection detected need for sudo
[Mon][DEBUG ] connected to host: Mon 
[Mon][DEBUG ] detect platform information from remote host
[Mon][DEBUG ] detect machine type
[Mon][DEBUG ] find the location of an executable
[Mon][INFO  ] Running command: sudo /usr/sbin/ip link show
[Mon][INFO  ] Running command: sudo /usr/sbin/ip addr show
[Mon][DEBUG ] IP addresses found: ['10.12.27.14']

[ceph_deploy.new][ERROR ] custom cluster names are not supported on sysvinit hosts
[ceph_deploy][ERROR ] ClusterNameError: host Mon does not support custom cluster names

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:1183

RFE : https://bugzilla.redhat.com/show_bug.cgi?id=1267145
[RFE]Ceph support for custom cluster name for the sysvinit hosts