Bug 1461527 - [RFE] store longer dup op detection information
[RFE] store longer dup op detection information
Status: CLOSED ERRATA
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RADOS (Show other bugs)
2.2
Unspecified Unspecified
unspecified Severity unspecified
: rc
: 2.4
Assigned To: Eric Ivancich
ceph-qe-bugs
Bara Ancincova
: FutureFeature
Depends On:
Blocks: 1479701
  Show dependency treegraph
 
Reported: 2017-06-14 12:58 EDT by Josh Durgin
Modified: 2017-09-05 16:49 EDT (History)
12 users (show)

See Also:
Fixed In Version: RHEL: ceph-10.2.7-31.el7cp Ubuntu: ceph_10.2.7-32redhat1
Doc Type: Enhancement
Doc Text:
.A new structure for detecting duplicate operations Under certain circumstances, it is better to recover by using the backfill process instead of log-based recovery. The most direct way to force backfilling is to reduce the size of the placement group log. This setting, however, undermines the detection of duplicate operations. This update introduces a separate structure to detect duplicate operations beyond the placement group log entries. As a result, duplicate operations are detected as expected.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-09-05 16:49:52 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Ceph Project Bug Tracker 20298 None None None 2017-06-14 12:58 EDT

  None (edit)
Description Josh Durgin 2017-06-14 12:58:47 EDT
Under some workloads, it is beneficial to avoid log-based recovery and use backfill instead. This lets I/O proceed on objects being backfilled, since it occurs in the background. This is implemented by setting the min/max pg log size to 1 and 2.

This has a side effect of eliminating our duplicate op detection. To make this configuration robust, we need to keep a separate structure to detect duplicate ops.
Comment 3 Eric Ivancich 2017-08-02 10:35:48 EDT
This PR should likely go into this BZ as it fixes a bug in the dup ops code:

https://github.com/ceph/ceph/pull/16744

At the time of this writing it hasn't been merged upstream yet.
Comment 5 Eric Ivancich 2017-08-02 11:19:52 EDT
Josh, my understanding is that this won't make it in 2.4 unless it's labeled a blocker. I can do more analysis to see it could create problems other than putting items in the dup index when it should not, but do you have a sense?
Comment 6 Josh Durgin 2017-08-02 11:55:51 EDT
(In reply to Eric Ivancich from comment #5)
> Josh, my understanding is that this won't make it in 2.4 unless it's labeled
> a blocker. I can do more analysis to see it could create problems other than
> putting items in the dup index when it should not, but do you have a sense?

It just saves a tiny amount of cpu/memory in the rare case when we don't index anything. It's not necessary to backport the fix.
Comment 12 shylesh 2017-09-05 10:07:30 EDT
Marking this bug as verified as planned regression testing and bug verfication is done.

Verified on 10.2.7-32.el7cp.x86_64
Comment 14 errata-xmlrpc 2017-09-05 16:49:52 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2640

Note You need to log in before you can comment on or make changes to this bug.