Bug 2006771
| Summary: | [RBD] - ISCSI - No error/warning displayed when incorrect parameters are specified in the yaml | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Preethi <pnataraj> |
| Component: | Cephadm | Assignee: | Adam King <adking> |
| Status: | CLOSED ERRATA | QA Contact: | Preethi <pnataraj> |
| Severity: | high | Docs Contact: | Karen Norteman <knortema> |
| Priority: | urgent | ||
| Version: | 5.0 | CC: | adking, akraj, ceph-eng-bugs, idryomov, kdreyer, vereddy |
| Target Milestone: | --- | Keywords: | Rebase |
| Target Release: | 5.2 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | ceph-16.2.8-2.el8cp | Doc Type: | Enhancement |
| Doc Text: |
.`cephadm` raises health warnings if it fails to apply a specification
Previously, failures to apply a specification would only be reported as a service event which the users would often not check.
With this release, `cephadm` raises health warnings if it fails to apply a specification, such as an incorrect pool name in an iscsi specification, to alert users.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-08-09 17:36:05 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 2102272 | ||
Deployment was successful and services are up when pool name is corrected in the yaml. Below is the output: [ceph: root@ceph-pnataraj-7ypsv7-node1-installer home]# ceph orch ls NAME RUNNING REFRESHED AGE PLACEMENT alertmanager 2/2 2m ago 4d ceph-pnataraj-7ypsv7-node1-installer;ceph-pnataraj-7ypsv7-node2 grafana 1/1 90s ago 4d ceph-pnataraj-7ypsv7-node1-installer iscsi.iscsi 4/4 0s ago 12s ceph-pnataraj-7ypsv7-node3;ceph-pnataraj-7ypsv7-node4;ceph-pnataraj-7ypsv7-node5;ceph-pnataraj-7ypsv7-node8 mgr 2/2 2m ago 66m ceph-pnataraj-7ypsv7-node1-installer;ceph-pnataraj-7ypsv7-node2;count:2 mon 3/3 3m ago 4d label:mon node-exporter 8/8 4m ago 4d * osd.all-available-devices 12/20 4m ago 4d * prometheus 1/1 90s ago 4d ceph-pnataraj-7ypsv7-node1-installer rgw.foo 2/2 3m ago 4d ceph-pnataraj-7ypsv7-node6;ceph-pnataraj-7ypsv7-node7;count:2 Thank you. We'll take it. upstream tracker: https://tracker.ceph.com/issues/44414 upstream master pr: https://github.com/ceph/ceph/pull/43376 backport still in progress The diff https://github.com/ceph/ceph/pull/43376/files is quite big. I'll wait for a rebase Am able to see the issue in latest 5.2 builds. please find the below snippet. I have given incorrect pool name i.e iscsi instead of iscsipool
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# ceph osd pool create iscsipool
pool 'iscsipool' created
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# ceph osd pool application enable iscsipool rbd
enabled application 'rbd' on pool 'iscsipool'
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# cat iscsi.yaml
service_type: iscsi
service_id: iscsi
placement:
hosts:
- ceph-pnataraj-srzgvw-node4
- ceph-pnataraj-srzgvw-node5
spec:
pool: iscsi
trusted_ip_list: "10.0.208.76,10.0.210.67"
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# ceph orch apply -i iscsi.yaml
Scheduled iscsi.iscsi update...
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# ceph orch ls
NAME PORTS RUNNING REFRESHED AGE PLACEMENT
iscsi.iscsi 0/2 - 12s ceph-pnataraj-srzgvw-node4;ceph-pnataraj-srzgvw-node5
mgr 2/2 6m ago 63m label:mgr
mon 3/3 6m ago 60m label:mon
osd.all-available-devices 10 81s ago 56m *
I also tried ceph orch deployment with incorrect parameters- below snippet
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# ceph orch apply iscsi iscsi --placement="ceph-pnataraj-srzgvw-node4,ceph-pnataraj-srzgvw-node5" --trusted_ip_list="10.0.208.76,10.0.210.67" admin admin
Scheduled iscsi.iscsi update...
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# ceph orch ls
NAME PORTS RUNNING REFRESHED AGE PLACEMENT
iscsi.iscsi 0/2 - 5s ceph-pnataraj-srzgvw-node4;ceph-pnataraj-srzgvw-node5
mgr 2/2 27s ago 67m label:mgr
mon 3/3 4m ago 65m label:mon
osd.all-available-devices 10 5m ago 61m *
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# ceph versions
{
"mon": {
"ceph version 16.2.8-22.el8cp (857c12016f00d5089a358b3bbc25abf43d84a7a2) pacific (stable)": 3
},
"mgr": {
"ceph version 16.2.8-22.el8cp (857c12016f00d5089a358b3bbc25abf43d84a7a2) pacific (stable)": 2
},
"osd": {
"ceph version 16.2.8-22.el8cp (857c12016f00d5089a358b3bbc25abf43d84a7a2) pacific (stable)": 10
},
"mds": {},
"overall": {
"ceph version 16.2.8-22.el8cp (857c12016f00d5089a358b3bbc25abf43d84a7a2) pacific (stable)": 15
}
}
**************************************************************************************************************
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]# ceph osd lspools
1 device_health_metrics
4 iscsipool
[ceph: root@ceph-pnataraj-srzgvw-node1-installer /]#
************************************************************************************************
node details - 10.0.210.185 cephuser/cephuser
Hi Adam,
Its working in the latest 5.2 builds. Below snippet for reference.
[ceph: root@ceph-pnataraj-3qg142-node1-installer /]# ceph orch apply -i iscsi.yaml
Scheduled iscsi.iscsi update...
[ceph: root@ceph-pnataraj-3qg142-node1-installer /]# ceph orch ls
NAME PORTS RUNNING REFRESHED AGE PLACEMENT
iscsi.iscsi 0/2 - 3s ceph-pnataraj-3qg142-node4;ceph-pnataraj-3qg142-node5
mgr 2/2 6m ago 18h label:mgr
mon 3/3 6m ago 10m ceph-pnataraj-3qg142-node2;ceph-pnataraj-3qg142-node3;ceph-pnataraj-3qg142-node1-installer
osd.all-available-devices 10 9m ago 18h *
[ceph: root@ceph-pnataraj-3qg142-node1-installer /]# ceph -s
cluster:
id: 1d209b94-e587-11ec-a583-fa163e2bf4f8
health: HEALTH_WARN
Failed to apply 1 service(s): iscsi.iscsi
services:
mon: 3 daemons, quorum ceph-pnataraj-3qg142-node1-installer,ceph-pnataraj-3qg142-node2,ceph-pnataraj-3qg142-node3 (age 8m)
mgr: ceph-pnataraj-3qg142-node2.cyjuuv(active, since 16h), standbys: ceph-pnataraj-3qg142-node1-installer.isfvkk
osd: 10 osds: 10 up (since 16h), 10 in (since 16h)
data:
pools: 1 pools, 1 pgs
objects: 0 objects, 0 B
usage: 65 MiB used, 200 GiB / 200 GiB avail
pgs: 1 active+clean
[ceph: root@ceph-pnataraj-3qg142-node1-installer /]# ceph health detail
HEALTH_WARN Failed to apply 1 service(s): iscsi.iscsi
[WRN] CEPHADM_APPLY_SPEC_FAIL: Failed to apply 1 service(s): iscsi.iscsi
iscsi.iscsi: Cannot find pool "iscsi" for service iscsi.iscsi
[ceph: root@ceph-pnataraj-3qg142-node1-installer /]# ceph version
ceph version 16.2.8-34.el8cp (da41b2a854c731f3062af5ca7b7aca470b0bec29) pacific (stable)
*** Bug 2069977 has been marked as a duplicate of this bug. *** Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Red Hat Ceph Storage Security, Bug Fix, and Enhancement Update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5997 |
Description of problem:RBD- ISCSI - No error/warning displayed when incorrect parameters are specified in the yaml file used for ISCSI deployment Version-Release number of selected component (if applicable): [ceph: root@ceph-pnataraj-7ypsv7-node1-installer /]# ceph version ceph version 16.2.0-79.el8cp (63c5c96018da6d39383c8f5ae534a0d1523fc274) pacific (stable) How reproducible: Steps to Reproduce: 1. Deploy 5.0 cluster and add ISCSI service using yaml specification 2. Create iscsi pool - ceph osd pool create iscsi 3. ceph osd pool application enable ISCSI rbd 4. use yaml specification to deploy ISCSI service as below. NOTE: i have given incorrect pool name in the below spec. Instead of iscsi, iscsipool is given which is incorrect poolname #############[ceph: root@ceph-pnataraj-7ypsv7-node3 ceph]# cat iscsi.yaml service_type: iscsi service_id: iscsi placement: hosts: - ceph-pnataraj-7ypsv7-node3 - ceph-pnataraj-7ypsv7-node4 - ceph-pnataraj-7ypsv7-node5 - ceph-pnataraj-7ypsv7-node8 spec: pool: iscsipool trusted_ip_list: "10.0.210.8,10.0.209.227,10.0.210.191,10.0.211.111" ###################### Output: [ceph: root@ceph-pnataraj-7ypsv7-node1-installer home]# ceph orch apply -i iscsi.yaml Scheduled iscsi.iscsi update.. [ceph: root@ceph-pnataraj-7ypsv7-node1-installer /]# ceph orch ls NAME RUNNING REFRESHED AGE PLACEMENT alertmanager 2/2 4m ago 4d ceph-pnataraj-7ypsv7-node1-installer;ceph-pnataraj-7ypsv7-node2 grafana 1/1 3m ago 4d ceph-pnataraj-7ypsv7-node1-installer iscsi.iscsi 0/4 - 22s ceph-pnataraj-7ypsv7-node3;ceph-pnataraj-7ypsv7-node4;ceph-pnataraj-7ypsv7-node5;ceph-pnataraj-7ypsv7-node8 mgr 2/2 4m ago 57m ceph-pnataraj-7ypsv7-node1-installer;ceph-pnataraj-7ypsv7-node2;count:2 mon 3/3 6m ago 4d label:mon node-exporter 8/8 7m ago 4d * osd.all-available-devices 12/20 7m ago 4d * prometheus 1/1 3m ago 4d ceph-pnataraj-7ypsv7-node1-installer rgw.foo 2/2 6m ago 4d ceph-pnataraj-7ypsv7-node6;ceph-pnataraj-7ypsv7-node7;count:2 Actual results: Invalid poolname is given in the yaml file and there was no error/warning inspite Deployment was successful but services was not up. There was no errors/msgs in the mgr logs as well. Expected results: error or warning msgs should be seen if we are providing invalid/wrong parameters Additional info: 10.0.209.88 bootstrap node cephuser@cephuser root/q