Bug 1461527 - [RFE] store longer dup op detection information
Summary: [RFE] store longer dup op detection information
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RADOS
Version: 2.2
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: rc
: 2.4
Assignee: J. Eric Ivancich
QA Contact: ceph-qe-bugs
Bara Ancincova
URL:
Whiteboard:
Depends On:
Blocks: 1479701
TreeView+ depends on / blocked
 
Reported: 2017-06-14 16:58 UTC by Josh Durgin
Modified: 2022-02-21 18:20 UTC (History)
12 users (show)

Fixed In Version: RHEL: ceph-10.2.7-31.el7cp Ubuntu: ceph_10.2.7-32redhat1
Doc Type: Enhancement
Doc Text:
.A new structure for detecting duplicate operations Under certain circumstances, it is better to recover by using the backfill process instead of log-based recovery. The most direct way to force backfilling is to reduce the size of the placement group log. This setting, however, undermines the detection of duplicate operations. This update introduces a separate structure to detect duplicate operations beyond the placement group log entries. As a result, duplicate operations are detected as expected.
Clone Of:
Environment:
Last Closed: 2017-09-05 20:49:52 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 20298 0 None None None 2017-06-14 16:58:46 UTC
Red Hat Product Errata RHBA-2017:2640 0 normal SHIPPED_LIVE Red Hat Ceph Storage 2.4 bug fix and enhancement update 2017-10-10 20:18:35 UTC

Description Josh Durgin 2017-06-14 16:58:47 UTC
Under some workloads, it is beneficial to avoid log-based recovery and use backfill instead. This lets I/O proceed on objects being backfilled, since it occurs in the background. This is implemented by setting the min/max pg log size to 1 and 2.

This has a side effect of eliminating our duplicate op detection. To make this configuration robust, we need to keep a separate structure to detect duplicate ops.

Comment 3 J. Eric Ivancich 2017-08-02 14:35:48 UTC
This PR should likely go into this BZ as it fixes a bug in the dup ops code:

https://github.com/ceph/ceph/pull/16744

At the time of this writing it hasn't been merged upstream yet.

Comment 5 J. Eric Ivancich 2017-08-02 15:19:52 UTC
Josh, my understanding is that this won't make it in 2.4 unless it's labeled a blocker. I can do more analysis to see it could create problems other than putting items in the dup index when it should not, but do you have a sense?

Comment 6 Josh Durgin 2017-08-02 15:55:51 UTC
(In reply to Eric Ivancich from comment #5)
> Josh, my understanding is that this won't make it in 2.4 unless it's labeled
> a blocker. I can do more analysis to see it could create problems other than
> putting items in the dup index when it should not, but do you have a sense?

It just saves a tiny amount of cpu/memory in the rare case when we don't index anything. It's not necessary to backport the fix.

Comment 12 shylesh 2017-09-05 14:07:30 UTC
Marking this bug as verified as planned regression testing and bug verfication is done.

Verified on 10.2.7-32.el7cp.x86_64

Comment 14 errata-xmlrpc 2017-09-05 20:49:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2640


Note You need to log in before you can comment on or make changes to this bug.