Bug 1461527

Summary:	[RFE] store longer dup op detection information
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Josh Durgin <jdurgin>
Component:	RADOS	Assignee:	J. Eric Ivancich <ivancich>
Status:	CLOSED ERRATA	QA Contact:	ceph-qe-bugs <ceph-qe-bugs>
Severity:	medium	Docs Contact:	Bara Ancincova <bancinco>
Priority:	unspecified
Version:	2.2	CC:	anharris, ceph-eng-bugs, dzafman, hnallurv, ivancich, jdurgin, kchai, mhackett, shmohan, tserlin, uboppana, vakulkar
Target Milestone:	rc	Keywords:	FutureFeature
Target Release:	2.4
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	RHEL: ceph-10.2.7-31.el7cp Ubuntu: ceph_10.2.7-32redhat1	Doc Type:	Enhancement
Doc Text:	.A new structure for detecting duplicate operations Under certain circumstances, it is better to recover by using the backfill process instead of log-based recovery. The most direct way to force backfilling is to reduce the size of the placement group log. This setting, however, undermines the detection of duplicate operations. This update introduces a separate structure to detect duplicate operations beyond the placement group log entries. As a result, duplicate operations are detected as expected.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-09-05 20:49:52 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1479701

Description Josh Durgin 2017-06-14 16:58:47 UTC

Under some workloads, it is beneficial to avoid log-based recovery and use backfill instead. This lets I/O proceed on objects being backfilled, since it occurs in the background. This is implemented by setting the min/max pg log size to 1 and 2.

This has a side effect of eliminating our duplicate op detection. To make this configuration robust, we need to keep a separate structure to detect duplicate ops.

Comment 3 J. Eric Ivancich 2017-08-02 14:35:48 UTC

This PR should likely go into this BZ as it fixes a bug in the dup ops code:

https://github.com/ceph/ceph/pull/16744

At the time of this writing it hasn't been merged upstream yet.

Comment 5 J. Eric Ivancich 2017-08-02 15:19:52 UTC

Josh, my understanding is that this won't make it in 2.4 unless it's labeled a blocker. I can do more analysis to see it could create problems other than putting items in the dup index when it should not, but do you have a sense?

Comment 6 Josh Durgin 2017-08-02 15:55:51 UTC

(In reply to Eric Ivancich from comment #5)
> Josh, my understanding is that this won't make it in 2.4 unless it's labeled
> a blocker. I can do more analysis to see it could create problems other than
> putting items in the dup index when it should not, but do you have a sense?

It just saves a tiny amount of cpu/memory in the rare case when we don't index anything. It's not necessary to backport the fix.

Comment 12 shylesh 2017-09-05 14:07:30 UTC

Marking this bug as verified as planned regression testing and bug verfication is done.

Verified on 10.2.7-32.el7cp.x86_64

Comment 14 errata-xmlrpc 2017-09-05 20:49:52 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2640