Bug 2059839

Summary:	[cee/sd][cephadm] cephadm bootstrap is not applying the configuration which passed via the option "--apply-spec"
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Geo Jose <gjose>
Component:	Cephadm	Assignee:	Guillaume Abrioux <gabrioux>
Status:	POST ---	QA Contact:	Sunil Kumar Nagaraju <sunnagar>
Severity:	medium	Docs Contact:	Karen Norteman <knortema>
Priority:	unspecified
Version:	5.0	CC:	adking, lithomas, rlepaksh, saraut, vereddy
Target Milestone:	---
Target Release:	7.0
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Geo Jose 2022-03-02 07:39:43 UTC

Description of problem:
 cephadm bootstrap is not applying the configuration which passed via the option "--apply-spec"

Version-Release number of selected component (if applicable):
  ceph version 16.2.0-152.el8cp

Steps to Reproduce:
1. Create a spec file with host, alertmanager, crash, grafana, mgr, mon, node-exporter, prometheus and osd.
2. Bootstrap the cluster using cephadm with the option "--apply-spec".


Actual results:
Not applying the configuration after bootstrap.

Expected results:
The configuration should be apply after bootstrap the cluster.

Comment 5 Geo Jose 2022-03-02 08:09:07 UTC

Getting the below error:

# cephadm --verbose bootstrap --mon-ip <IP> --registry-json <registry_details> --apply-spec <service.yaml> --allow-fqdn-hostname

[...]

Running command: /usr/bin/podman run --rm --ipc=host --authfile=/etc/ceph/podman-auth.json --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=registry.redhat.io/rhceph/rhceph-5-rhel8:latest -e NODE_NAME=rhcs51.sample.com -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5acfc0f4-996b-11ec-bb1b-001a4a00055c:/var/log/ceph:z -v /tmp/ceph-tmpjrl7v51u:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpfv5vddti:/etc/ceph/ceph.conf:z -v /usr/share/cephadm-ansible/service.yaml:/tmp/spec.yml:z registry.redhat.io/rhceph/rhceph-5-rhel8:latest orch apply -i /tmp/spec.yml
/usr/bin/ceph: Error: relabeling content in /usr is not allowed
Non-zero exit code 126 from /usr/bin/podman run --rm --ipc=host --authfile=/etc/ceph/podman-auth.json --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=registry.redhat.io/rhceph/rhceph-5-rhel8:latest -e NODE_NAME=rhcs51.sample.com -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5acfc0f4-996b-11ec-bb1b-001a4a00055c:/var/log/ceph:z -v /tmp/ceph-tmpjrl7v51u:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpfv5vddti:/etc/ceph/ceph.conf:z -v /usr/share/cephadm-ansible/service.yaml:/tmp/spec.yml:z registry.redhat.io/rhceph/rhceph-5-rhel8:latest orch apply -i /tmp/spec.yml
/usr/bin/ceph: stderr Error: relabeling content in /usr is not allowed
Traceback (most recent call last):
  File "/usr/sbin/cephadm", line 8140, in <module>
    main()
  File "/usr/sbin/cephadm", line 8128, in main
    r = ctx.func(ctx)
  File "/usr/sbin/cephadm", line 1730, in _default_image
    return func(ctx)
  File "/usr/sbin/cephadm", line 4156, in command_bootstrap
    out = cli(['orch', 'apply', '-i', '/tmp/spec.yml'], extra_mounts=mounts)
  File "/usr/sbin/cephadm", line 4050, in cli
    ).run(timeout=timeout)
  File "/usr/sbin/cephadm", line 3286, in run
    desc=self.entrypoint, timeout=timeout)
  File "/usr/sbin/cephadm", line 1424, in call_throws
    raise RuntimeError('Failed command: %s' % ' '.join(command))
RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --authfile=/etc/ceph/podman-auth.json --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=registry.redhat.io/rhceph/rhceph-5-rhel8:latest -e NODE_NAME=rhcs51.sample.com -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5acfc0f4-996b-11ec-bb1b-001a4a00055c:/var/log/ceph:z -v /tmp/ceph-tmpjrl7v51u:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpfv5vddti:/etc/ceph/ceph.conf:z -v /usr/share/cephadm-ansible/service.yaml:/tmp/spec.yml:z registry.redhat.io/rhceph/rhceph-5-rhel8:latest orch apply -i /tmp/spec.yml