Bug 1559026

Summary:	[RFE] cephfs add information about active "scrub_path" commands to "ceph -s " or similar
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Tomas Petr <tpetr>
Component:	CephFS	Assignee:	Venky Shankar <vshankar>
Status:	CLOSED ERRATA	QA Contact:	Hemanth Kumar <hyelloji>
Severity:	medium	Docs Contact:	Erin Donnelly <edonnell>
Priority:	low
Version:	3.0	CC:	anharris, ceph-eng-bugs, ceph-qe-bugs, edonnell, hyelloji, pasik, pdonnell, tchandra, tserlin, vshankar
Target Milestone:	rc	Keywords:	FutureFeature
Target Release:	4.0
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:	ceph-14.2.4-7.el8cp, ceph-14.2.4-2.el7cp	Doc Type:	Enhancement
Doc Text:	.`ceph -w` now shows information about CephFS scrubs Previously, it was not possible to check the ongoing Ceph File System (CephFS) scrubs status aside from checking the Metadata server (MDS) logs. With this update, the `ceph -w` command, shows information about active CephFS scrubs to better understand the status.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-01-31 12:44:52 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1730176

Description Tomas Petr 2018-03-21 14:27:50 UTC

Description of problem:
When you run command for scrub_path on mds, like:
# ceph daemon mds.mdss-1 scrub_path / recursive repair

In the logs we will get  two lines about starting the command above:
2018-03-21 08:08:07.947869 7fe4b616f700  1 mds.mdss-1 asok_command: scrub_path (starting...)
2018-03-21 08:08:07.948528 7fe4b616f700  1 mds.mdss-1 asok_command: scrub_path (complete)

 ... # messages about scrubbed objects

- and final message when scrub_path finishes:
2018-03-21 08:08:09.634098 7fe4af962700  0 log_channel(cluster) log [INF] : scrub complete with tag ''

In this case the command finished in matter of seconds, but on large clusters it may take hours to finish.
Also you may run the command for multiple sub paths, would be good to actually track which command is running or has ended in the logs.

In "ceph -s" the output could look like:
MDS.<daemonID> scrubbing in progress: scrub_path <path_that_is_scrubbed> recursive repair
MDS.<daemonID> scrubbing in progress: scrub_path <different_path_that_is_scrubbed> recursive repair

This would especially simplified situations when multiple scrub_path commands were started or with large cephfs.

Version-Release number of selected component (if applicable):
RHCS 3.0

Comment 5 John Spray 2018-03-22 09:03:47 UTC

See also: "Progress/abort/pause interface for ongoing scrubs
" http://tracker.ceph.com/issues/12282

Comment 18 Giridhar Ramaraju 2019-08-05 13:06:40 UTC

Updating the QA Contact to a Hemant. Hemant will be rerouting them to the appropriate QE Associate. 

Regards,
Giri

Comment 19 Giridhar Ramaraju 2019-08-05 13:09:17 UTC

Updating the QA Contact to a Hemant. Hemant will be rerouting them to the appropriate QE Associate. 

Regards,
Giri

Comment 30 Hemanth Kumar 2020-01-29 09:26:16 UTC

The scrub progress can now be seen with "ceph -w"

2020-01-29 09:13:00.815430 mds.magna127 [INF] scrub summary: active
2020-01-29 09:13:00.815851 mds.magna127 [INF] scrub queued for path: /
2020-01-29 09:13:00.815857 mds.magna127 [INF] scrub summary: active [paths:/]
2020-01-29 09:13:11.606764 mds.magna127 [INF] scrub summary: idle
2020-01-29 09:13:11.607558 mds.magna127 [INF] scrub complete with tag '6dd9cc95-215c-4e40-aa6d-c1a2698f2523'
2020-01-29 09:13:11.607565 mds.magna127 [INF] scrub completed for path: /
2020-01-29 09:13:11.607568 mds.magna127 [INF] scrub summary: idle
2020-01-29 09:21:45.688361 mds.magna127 [INF] scrub summary: active
2020-01-29 09:21:45.688449 mds.magna127 [INF] scrub summary: idle
2020-01-29 09:21:45.688459 mds.magna127 [INF] scrub queued for path: /
2020-01-29 09:21:45.688464 mds.magna127 [INF] scrub summary: idle
2020-01-29 09:21:45.689540 mds.magna127 [INF] scrub complete with tag 'a01e2b9e-fd3c-4607-8abf-d3bef34399f5'
2020-01-29 09:21:45.689547 mds.magna127 [INF] scrub completed for path: /
2020-01-29 09:21:45.689551 mds.magna127 [INF] scrub summary: idle


Moving to verified.

Comment 33 errata-xmlrpc 2020-01-31 12:44:52 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0312