Bug 2210873 - assimilate_{{ tripleo_cephadm_cluster }}.conf required if `--crush-hierarchy` is used
Summary: assimilate_{{ tripleo_cephadm_cluster }}.conf required if `--crush-hierarchy`...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: tripleo-ansible
Version: 17.1 (Wallaby)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: z2
: 17.1
Assignee: Manoj Katari
QA Contact: Alfredo
URL:
Whiteboard:
: 2228783 2245945 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-05-29 20:04 UTC by Emilien Macchi
Modified: 2024-03-21 15:03 UTC (History)
10 users (show)

Fixed In Version: tripleo-ansible-3.3.1-1.20230620010933.03ab6c5.el9ost
Doc Type: Bug Fix
Doc Text:
Before this update, deploying Red Hat Ceph Storage (RHCS) with the `--crush-hierachy` parameter failed with an 'assimilate.conf not found' error when setting crush rules. + With this update, the RHCS CLI does not bind mount `assimilate.conf` and RHCS sets crush rules successfully.
Clone Of:
Environment:
Last Closed: 2024-01-16 14:32:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 884597 0 None MERGED Fix issues while applying ceph crush rules 2023-06-21 09:24:35 UTC
Red Hat Issue Tracker OSP-25447 0 None None None 2023-05-29 20:05:42 UTC
Red Hat Product Errata RHBA-2024:0209 0 None None None 2024-01-16 14:32:29 UTC

Description Emilien Macchi 2023-05-29 20:04:13 UTC
Description of problem:
When using deployed-ceph with `--crush-hierarchy` parameter to the ceph deploy command, a file has to exist but should not be needed by default.

Version-Release number of selected component (if applicable):
RHOS-17.1-RHEL-9-20230525.n.1

How reproducible:
Deploy ceph with `--crush-hierarchy` parameter and without `--config` nor pre-created ceph.conf.


Actual results:

The ceph deployment will fail:

2023-05-29 14:59:42,544 p=165056 u=stack n=ansible | 2023-05-29 14:59:42.544281 | fa163e4e-4a57-698b-b613-0000000000d7 |      FATAL | insert new default crush rule into daemon to prevent restart | controller-0 -> 172.16.200.13 | item=controller-2 | error={"ansible_loop_var": "item", "changed": false, "cmd": ["podman", "run", "--rm", "--net=host", "--ipc=host", "--volume", "/var/lib/ceph/305f99dc-9741-55f8-879a-82b88672831a/config/:/etc/ceph:z", "--volume", "/home/ceph-admin/assimilate_ceph.conf:/home/assimilate_ceph.conf:z", "--volume", "/var/run/ceph/305f99dc-9741-55f8-879a-82b88672831a:/var/run/ceph:z", "--entrypoint", "ceph", "registry-proxy.engineering.redhat.com/rh-osbs/rhceph:6-115", "--admin-daemon", "/var/run/ceph/ceph-mon.controller-2.shiftstack.test.asok", "config", "set", "osd_pool_default_crush_rule", "3"], "delta": "0:00:00.048437", "end": "2023-05-29 18:59:42.521240", "item": "controller-2", "msg": "non-zero return code", "rc": 125, "start": "2023-05-29 18:59:42.472803", "stderr": "Error: statfs /home/ceph-admin/assimilate_ceph.conf: no such file or directory", "stderr_lines": ["Error: statfs /home/ceph-admin/assimilate_ceph.conf: no such file or directory"], "stdout": "", "stdout_lines": []}
2023-05-29 14:59:42,545 p=165056 u=stack n=ansible | [WARNING]: ('controller-0 -> 172.16.200.13',
'fa163e4e-4a57-698b-b613-0000000000d7') missing from stats

Expected results:

ceph.conf is not required if I don't want to provide one, so ceph-cli should not try to bind mount /home/ceph-admin/assimilate_ceph.conf which doesn't exist.

Comment 1 Emilien Macchi 2023-05-30 14:36:02 UTC
Also weird:

```
[root@controller-0 ~]# file /home/ceph-admin/assimilate_ceph.conf
/home/ceph-admin/assimilate_ceph.conf: ASCII text
```

The file exists...

Comment 2 Emilien Macchi 2023-05-30 14:36:48 UTC
And the file doesn't exist on the 2 other controllers.

Comment 3 John Fulton 2023-05-30 14:48:05 UTC
1. assimilate_ceph.conf is an argument passed to the cephadm bootstrap command.
2. cephadm boostrap is only run on one node (the first controller node by default)
3. Thus, it's expected that it not be present on the other two controllers

Comment 5 Manoj Katari 2023-05-31 10:57:47 UTC
Sure Francesco, working on it.

Comment 8 John Fulton 2023-06-07 17:41:20 UTC
Another way to encounter this bug is by using CephCrushRules in your Heat environment files. Same fix (884597) should address it though.

Comment 10 Francesco Pantano 2023-08-04 10:58:36 UTC
*** Bug 2228783 has been marked as a duplicate of this bug. ***

Comment 11 Manoj Katari 2023-08-07 05:32:48 UTC
Updated doc text looks good to me.

Comment 14 Manoj Katari 2023-09-25 02:39:04 UTC
I think it is better to add it as a known issue  in z1 release notes and the description section of the bug describes the problem clearly.

Comment 22 John Fulton 2023-12-04 14:42:34 UTC
*** Bug 2245945 has been marked as a duplicate of this bug. ***

Comment 31 errata-xmlrpc 2024-01-16 14:32:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 17.1.2 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:0209


Note You need to log in before you can comment on or make changes to this bug.