Bug 1491170 - [MDS]: "ceph tell mds.* " commands not working for MDS
Summary: [MDS]: "ceph tell mds.* " commands not working for MDS
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: CephFS
Version: 3.0
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: z1
: 3.0
Assignee: Patrick Donnelly
QA Contact: Ramakrishnan Periyasamy
Aron Gunn
URL:
Whiteboard:
Depends On:
Blocks: 1494421
TreeView+ depends on / blocked
 
Reported: 2017-09-13 08:42 UTC by Ramakrishnan Periyasamy
Modified: 2018-03-08 15:51 UTC (History)
9 users (show)

Fixed In Version: RHEL: ceph-12.2.1-41.el7cp Ubuntu: ceph_12.2.1-43redhat1xenial
Doc Type: Bug Fix
Doc Text:
Previously, an attempt to use the --cluster option with the ceph tell mds commands fails with the RuntimeError exception message. With this release, the --cluster option works as expected, when using a non-default cluster name.
Clone Of:
Environment:
Last Closed: 2018-03-08 15:51:04 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Ceph Project Bug Tracker 21406 None None None 2017-09-15 19:17:50 UTC
Red Hat Product Errata RHBA-2018:0474 normal SHIPPED_LIVE Red Hat Ceph Storage 3.0 bug fix update 2018-03-08 20:51:53 UTC

Description Ramakrishnan Periyasamy 2017-09-13 08:42:58 UTC
Description of problem:

"ceph tell mds.<id> **" commands fails with RuntimeError exception. Commands related to OSD/MON working without any issues.

[ubuntu@host028 ~]$ sudo ceph tell mds.1 injectargs "--debug-mds 20" --cluster qetest
Traceback (most recent call last):
  File "/bin/ceph", line 1121, in <module>
    retval = main()
  File "/bin/ceph", line 1041, in main
    prefix='get_command_descriptions')
  File "/usr/lib/python2.7/site-packages/ceph_argparse.py", line 1346, in json_command
    raise RuntimeError('"{0}": exception {1}'.format(argdict, e))
RuntimeError: "None": exception "['{"prefix": "get_command_descriptions"}']": exception [Errno 2] error calling conf_read_file

[ubuntu@host028 ~]$ sudo ceph tell mds.0 --cluster qetest  client ls
Traceback (most recent call last):
  File "/bin/ceph", line 1121, in <module>
    retval = main()
  File "/bin/ceph", line 1041, in main
    prefix='get_command_descriptions')
  File "/usr/lib/python2.7/site-packages/ceph_argparse.py", line 1346, in json_command
    raise RuntimeError('"{0}": exception {1}'.format(argdict, e))
RuntimeError: "None": exception "['{"prefix": "get_command_descriptions"}']": exception [Errno 2] error calling conf_read_file

Version-Release number of selected component (if applicable):
ceph: 12.2.0-1.el7cp (b661348f156f148d764b998b65b90451f096cb27) luminous (rc)

How reproducible:
10/10

Steps to Reproduce:
NA

Actual results:
NA

Expected results:
NA

Additional info:
NA

Comment 2 Patrick Donnelly 2017-09-13 16:00:37 UTC
<pre>
$ sudo ceph tell mds.1 injectargs "--debug-mds 20" --cluster qetest
Traceback (most recent call last):
  File "/bin/ceph", line 1121, in <module>
    retval = main()
  File "/bin/ceph", line 1041, in main
    prefix='get_command_descriptions')
  File "/usr/lib/python2.7/site-packages/ceph_argparse.py", line 1346, in json_command
    raise RuntimeError('"{0}": exception {1}'.format(argdict, e))
RuntimeError: "None": exception "['{"prefix": "get_command_descriptions"}']": exception [Errno 2] error calling conf_read_file
</pre>

The "--cluster qetest" argument should go before "tell". You see this error when the ceph executable can't find the config file associated with that cluster name.

Comment 4 Patrick Donnelly 2017-09-15 19:17:29 UTC
Ya that's a bug. Thanks!

http://tracker.ceph.com/issues/21406

Comment 5 Ram Raja 2017-09-21 10:56:31 UTC
As Patrick says in http://tracker.ceph.com/issues/21406
it's only the --cluster argument that `tell mds` doesn't understand. To use `tell mds` with cluster having a non-default name, you can get around the issue for now by passing the corresponding conf file path argument,  --conf <conf file path>

Comment 16 Ramakrishnan Periyasamy 2018-02-05 05:45:35 UTC
Moving this bug to verified state.

Verified in ceph build: 12.2.1-44.el7cp (5bef20c3d60a27005cbe5e814d833cb18aa2335f) luminous (stable)

command output:

[root@host1 ~]# ceph tell mds.0 injectargs "--debug-mds 20" 
2018-02-05 00:41:41.127707 7fbb457fa700  0 client.4248 ms_handle_reset on 10.70.39.9:6800/1535821975
2018-02-05 00:41:41.136793 7fbb467fc700  0 client.4249 ms_handle_reset on 10.70.39.9:6800/1535821975
debug_mds=20/20 


[root@host ~]# ceph tell mds.0 client ls
2018-02-05 00:42:26.328886 7ff68cff9700  0 client.4253 ms_handle_reset on 10.70.39.9:6800/1535821975
2018-02-05 00:42:26.336165 7ff68dffb700  0 client.4254 ms_handle_reset on 10.70.39.9:6800/1535821975
[
    {
        "id": 4232,
        "num_leases": 0,
        "num_caps": 2,
        "state": "open",
        "replay_requests": 0,
        "completed_requests": 0,
        "reconnecting": false,
        "inst": "client.4232 10.xx.xx.xx:0/2124990151",
        "client_metadata": {
            "ceph_sha1": "5bef20c3d60a27005cbe5e814d833cb18aa2335f",
            "ceph_version": "ceph version 12.2.1-44.el7cp (5bef20c3d60a27005cbe5e814d833cb18aa2335f) luminous (stable)",
            "entity_id": "admin",
            "hostname": "host.lab.eng.blr.redhat.com",
            "mount_point": "/mnt/fuse",
            "pid": "28234",
            "root": "/"
        }
    },
    {
        "id": 4234,
        "num_leases": 0,
        "num_caps": 3,
        "state": "open",
        "replay_requests": 0,
        "completed_requests": 0,
        "reconnecting": false,
        "inst": "client.4234 10.xx.xx.xx:0/3709878253",
        "client_metadata": {
            "entity_id": "admin",
            "hostname": "host.lab.eng.blr.redhat.com",
            "kernel_version": "3.10.0-693.17.1.el7.x86_64"
        }
    }
]

Comment 19 errata-xmlrpc 2018-03-08 15:51:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0474


Note You need to log in before you can comment on or make changes to this bug.