Bug 1396569 - OPS Tools | Availability Monitoring | OS Checks | oschecks-check_ceph_df failed
Summary: OPS Tools | Availability Monitoring | OS Checks | oschecks-check_ceph_df failed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: osops-tools-monitoring-oschecks
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: Upstream M1
: 13.0 (Queens)
Assignee: Martin Magr
QA Contact: Leonid Natapov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-18 16:09 UTC by Juan Badia Payno
Modified: 2020-12-14 07:53 UTC (History)
9 users (show)

Fixed In Version: osops-tools-monitoring-oschecks-0.1-0.9.23ee1b5git.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-27 13:08:58 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 400264 0 None master: MERGED osops-tools-monitoring: Added ceph user to ceph commands (Icb986bac88b52a123c2f99941fadab8c9e233b22) 2018-02-14 16:27:14 UTC
Red Hat Product Errata RHEA-2018:2084 0 None None None 2018-06-27 13:10:40 UTC

Description Juan Badia Payno 2016-11-18 16:09:25 UTC
Description of problem:
    The command /usr/libexec/openstack-monitoring/checks/oschecks-check_ceph_df fails due to some privileges issues on /etc/ceph/ceph.client.admin.keyring

Example:

[heat-admin@overcloud-novacompute-0 ~]$ /usr/libexec/openstack-monitoring/checks/oschecks-check_ceph_df 
CEPH UNKNOWN: 2016-11-18 11:03:02.338328 7fd6713c3700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory
2016-11-18 11:03:02.338336 7fd6713c3700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication
2016-11-18 11:03:02.338337 7fd6713c3700  0 librados: client.admin initialization error (2) No such file or directory
Error connecting to cluster: ObjectNotFound



Version-Release number of selected component (if applicable):
   centos-release-opstools.noarch             1-3.el7
   ceph.x86_64                                1:10.2.2-0.el7              


How reproducible:
  1.-Installed tripleO with ceph and availavility monitoring.
  2.-Log in into the overcloud-novacompute
  3.-Execute /usr/libexec/openstack-monitoring/checks/oschecks-check_ceph_df 
     The output will be something like: 
CEPH UNKNOWN: 2016-11-18 11:03:02.338328 7fd6713c3700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory
2016-11-18 11:03:02.338336 7fd6713c3700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication
2016-11-18 11:03:02.338337 7fd6713c3700  0 librados: client.admin initialization error (2) No such file or directory
Error connecting to cluster: ObjectNotFound


Additional info:
  There is a aworkaround:
     1.-sudo chmod 604 /etc/ceph/ceph.client.admin.keyring 
     2.-[heat-admin@overcloud-novacompute-0 ~]$ /usr/libexec/openstack-monitoring/checks/oschecks-check_ceph_df 
[OK] Ceph df avail. seems good (43372524MB/52416312MB)

Comment 2 Red Hat Bugzilla Rules Engine 2017-03-15 00:28:02 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 3 Juan Badia Payno 2017-04-04 10:02:17 UTC
This bugzilla was solve with this commit:
https://github.com/openstack/osops-tools-monitoring/commit/16aeaf8b112176dee716cae361060fcb7d985fe6

Some examples to use it: 

* oschecks-check_ceph_health
  example:
   oschecks-check_ceph_health ceph_client

* oschecks-check_ceph_df
  example:
   oschecks-check_ceph_df 80 90 ceph_client

Comment 4 Leonid Natapov 2017-04-05 09:00:35 UTC
ceph check is working.

usr/libexec/openstack-monitoring/checks/oschecks-check_ceph_df
[OK] Ceph df avail. seems good (43372524MB/52416312MB)

Comment 5 Cyril Lopez 2017-05-10 14:54:40 UTC
For science, in between wait for the path downstream, on all compute node :

setfacl -m u:sensu:r /etc/ceph/ceph.client.admin.keyring

Comment 16 Leonid Natapov 2018-03-26 15:22:59 UTC
[root@controller-0 checks]# ./oschecks-check_ceph_df
[OK] Ceph df avail. seems good (41286744MB/41397488MB)
[root@controller-0 checks]# ./oschecks-check_ceph_health
CEPH OK:

On compute machines there is no /etc/ceph/ceph.client.admin.keyring so this check won't work until you copy ceph.client.admin.keyring there.

I was told that it is not needed to have eph.client.admin.keyring file on compute nodes and we want to limit the access to the admin keyring and it was removed from compute nodes since OSP11.

Comment 18 errata-xmlrpc 2018-06-27 13:08:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2084


Note You need to log in before you can comment on or make changes to this bug.