Bug 1559026 - [RFE] cephfs add information about active "scrub_path" commands to "ceph -s " or similar
Summary: [RFE] cephfs add information about active "scrub_path" commands to "ceph -s "...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: CephFS
Version: 3.0
Hardware: All
OS: All
low
medium
Target Milestone: rc
: 4.0
Assignee: Venky Shankar
QA Contact: Hemanth Kumar
Erin Donnelly
URL:
Whiteboard:
Depends On:
Blocks: 1730176
TreeView+ depends on / blocked
 
Reported: 2018-03-21 14:27 UTC by Tomas Petr
Modified: 2021-03-11 17:17 UTC (History)
10 users (show)

Fixed In Version: ceph-14.2.4-7.el8cp, ceph-14.2.4-2.el7cp
Doc Type: Enhancement
Doc Text:
.`ceph -w` now shows information about CephFS scrubs Previously, it was not possible to check the ongoing Ceph File System (CephFS) scrubs status aside from checking the Metadata server (MDS) logs. With this update, the `ceph -w` command, shows information about active CephFS scrubs to better understand the status.
Clone Of:
Environment:
Last Closed: 2020-01-31 12:44:52 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 41508 0 None None None 2019-10-02 11:04:36 UTC
Red Hat Product Errata RHBA-2020:0312 0 None None None 2020-01-31 12:45:25 UTC

Description Tomas Petr 2018-03-21 14:27:50 UTC
Description of problem:
When you run command for scrub_path on mds, like:
# ceph daemon mds.mdss-1 scrub_path / recursive repair

In the logs we will get  two lines about starting the command above:
2018-03-21 08:08:07.947869 7fe4b616f700  1 mds.mdss-1 asok_command: scrub_path (starting...)
2018-03-21 08:08:07.948528 7fe4b616f700  1 mds.mdss-1 asok_command: scrub_path (complete)

 ... # messages about scrubbed objects

- and final message when scrub_path finishes:
2018-03-21 08:08:09.634098 7fe4af962700  0 log_channel(cluster) log [INF] : scrub complete with tag ''

In this case the command finished in matter of seconds, but on large clusters it may take hours to finish.
Also you may run the command for multiple sub paths, would be good to actually track which command is running or has ended in the logs.

In "ceph -s" the output could look like:
MDS.<daemonID> scrubbing in progress: scrub_path <path_that_is_scrubbed> recursive repair
MDS.<daemonID> scrubbing in progress: scrub_path <different_path_that_is_scrubbed> recursive repair

This would especially simplified situations when multiple scrub_path commands were started or with large cephfs.

Version-Release number of selected component (if applicable):
RHCS 3.0

Comment 5 John Spray 2018-03-22 09:03:47 UTC
See also: "Progress/abort/pause interface for ongoing scrubs
" http://tracker.ceph.com/issues/12282

Comment 18 Giridhar Ramaraju 2019-08-05 13:06:40 UTC
Updating the QA Contact to a Hemant. Hemant will be rerouting them to the appropriate QE Associate. 

Regards,
Giri

Comment 19 Giridhar Ramaraju 2019-08-05 13:09:17 UTC
Updating the QA Contact to a Hemant. Hemant will be rerouting them to the appropriate QE Associate. 

Regards,
Giri

Comment 30 Hemanth Kumar 2020-01-29 09:26:16 UTC
The scrub progress can now be seen with "ceph -w"

2020-01-29 09:13:00.815430 mds.magna127 [INF] scrub summary: active
2020-01-29 09:13:00.815851 mds.magna127 [INF] scrub queued for path: /
2020-01-29 09:13:00.815857 mds.magna127 [INF] scrub summary: active [paths:/]
2020-01-29 09:13:11.606764 mds.magna127 [INF] scrub summary: idle
2020-01-29 09:13:11.607558 mds.magna127 [INF] scrub complete with tag '6dd9cc95-215c-4e40-aa6d-c1a2698f2523'
2020-01-29 09:13:11.607565 mds.magna127 [INF] scrub completed for path: /
2020-01-29 09:13:11.607568 mds.magna127 [INF] scrub summary: idle
2020-01-29 09:21:45.688361 mds.magna127 [INF] scrub summary: active
2020-01-29 09:21:45.688449 mds.magna127 [INF] scrub summary: idle
2020-01-29 09:21:45.688459 mds.magna127 [INF] scrub queued for path: /
2020-01-29 09:21:45.688464 mds.magna127 [INF] scrub summary: idle
2020-01-29 09:21:45.689540 mds.magna127 [INF] scrub complete with tag 'a01e2b9e-fd3c-4607-8abf-d3bef34399f5'
2020-01-29 09:21:45.689547 mds.magna127 [INF] scrub completed for path: /
2020-01-29 09:21:45.689551 mds.magna127 [INF] scrub summary: idle


Moving to verified.

Comment 33 errata-xmlrpc 2020-01-31 12:44:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0312


Note You need to log in before you can comment on or make changes to this bug.