Bug 1347137 - Calamari cli endpoint api returns errors for rados df
Summary: Calamari cli endpoint api returns errors for rados df
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Calamari
Version: 2.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: rc
: 2.1
Assignee: Boris Ranto
QA Contact: Vasishta
Depends On:
TreeView+ depends on / blocked
Reported: 2016-06-16 06:48 UTC by anmol babu
Modified: 2016-11-22 19:26 UTC (History)
7 users (show)

Fixed In Version: RHEL: calamari-server-1.4.9-1.el7cp Ubuntu: calamari-server_1.4.9-2redhat1xenial
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2016-11-22 19:26:29 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:2815 0 normal SHIPPED_LIVE Moderate: Red Hat Ceph Storage security, bug fix, and enhancement update 2017-03-22 02:06:33 UTC

Description anmol babu 2016-06-16 06:48:36 UTC
Description of problem:
Calamari cli endpoint api returns errors for rados df

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Create cluster
2. Create some rbds
3. Use calamari cli end point to get rados df --cluster <cluster_name> --format=json

Actual results:
Api: https://<calamari_node>:8002/api/v2/cluster/{cluster-id}/cli?format=api
{"command":["rados", "df", "--cluster", "<cluster_name>", "--format=json"]}

POST /api/v2/cluster/4c6ac35f-cfe3-48ea-a88e-dd3db7ac7b6c/cli?format=api
Vary: Accept
Content-Type: text/html; charset=utf-8

    "out": "", 
    "err": "no valid command found; 10 closest matches:\nosd setmaxosd <int[0-]>\nosd pause\nosd crush rule rm <name>\nosd crush tree\nosd crush rule create-simple <name> <root> <type> {firstn|indep}\nosd crush rule create-erasure <name> {<profile>}\nosd crush get-tunable straw_calc_version\nosd crush show-tunables\nosd crush tunables legacy|argonaut|bobtail|firefly|hammer|jewel|optimal|default\nosd crush set-tunable straw_calc_version <int>\nError EINVAL: invalid command\n", 
    "status": 22

Expected results:
Valid response

Additional info:

Comment 2 Christina Meno 2016-06-16 14:54:44 UTC
I don't expect that this works.
The CLI endpoint only supports commands that begin with ceph or rbd.

What is this bug's impact to storage console?

Comment 3 anmol babu 2016-06-16 15:35:27 UTC
Earlier, we used salt to get this job done so every 10 mins, we used to get the count of objects and count of degraded objects using this command but through salt. But, the problem we are facing is salt is not very reliable for triggering periodic jobs and so now, we are moving all such periodic salt-calls to happen through calamari api(cli end-point). As part of this exercise, we would like to have this information (object count and degraded object count) also served through the calamari api giving us a json reponse.

Comment 5 Christina Meno 2016-06-16 23:04:54 UTC
ok, I can understand that AND the old way you were getting the data will have to suffice till 2.1

Comment 7 Boris Ranto 2016-10-03 17:14:43 UTC
Upstream PR:


Comment 8 John Spray 2016-10-04 12:06:49 UTC
I wouldn't recommend calling the CLI binary to get the "rados df" data.  The CLI tool itself is just calling into librados, so for Calamari to provide the data, Calamari should just call into librados too.

If you create a REST endpoint that serves these stats (and does so by invoking a remote librados call), then it will be straightforward to port that forward to ceph-mgr, as at that point we'll just have these stats in memory anyway (ceph-mgr keeps a PGMap handy).

Comment 9 Boris Ranto 2016-10-11 20:52:48 UTC
New re-worked PR based on John's comments:


Comment 10 Boris Ranto 2016-10-12 12:58:18 UTC
(Tested) PR for 1.4 branch:


Comment 15 Vasishta 2016-10-26 16:32:43 UTC
Hi Anmol,

'stats' cli endpoints mentioned by Boris in Comment 14 have been tried and they are working fine and they are giving information that 'rados df' would give. 

Sample of information displayed by /api/v2/cluster/<fsid>/pool/stats (to get stats of all the pools) looks like -

    "name": "test_rbd", 
    "num_objects_unfound": 0, 
    "num_objects_missing_on_primary": 0, 
    "num_object_clones": 0, 
    "num_objects": 31, 
    "num_object_copies": 93, 
    "num_bytes": 510, 
    "num_rd_kb": 30, 
    "num_wr_kb": 20, 
    "num_kb": 1, 
    "num_wr": 80, 
    "num_objects_degraded": 0, 
    "num_rd": 69

Moving this bug to VERIFIED state, Please let me know if there are any concerns or issues.


Comment 17 errata-xmlrpc 2016-11-22 19:26:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.