Bug 2101735 - [rbd-mirror] : Miss in alternate snapshot schedule for some of the images which were observed through snap ls --all
Summary: [rbd-mirror] : Miss in alternate snapshot schedule for some of the images whi...
Keywords:
Status: NEW
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RBD-Mirror
Version: 5.1
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 7.0
Assignee: Ilya Dryomov
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-06-28 09:33 UTC by Vasishta
Modified: 2023-07-31 21:50 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-4629 0 None None None 2022-06-28 09:52:07 UTC

Description Vasishta 2022-06-28 09:33:09 UTC
Description of problem:
snapshot schedule missed thrice when observed with snap ls --all.

Version-Release number of selected component (if applicable):
16.2.7-126.el8cp

How reproducible:
Observed thrice on same image in a span of 1-1:30 hours

Steps to Reproduce:
1. Configure two clusters.
2. Configure mirroring on 150 images
3. add snapshot scheduling 

Actual results:
Miss in alternate snapshot schedule for some of the images were observed.

Expected results:
Mirror Snapshots should get created (only with possible minimal delay)

Additional info:
Observation made on some of the images
Links have cli output of machine's current time, snapshot schedule status, snap ls -
http://pastebin.test.redhat.com/1061270
http://pastebin.test.redhat.com/1061268
http://pastebin.test.redhat.com/1061267
http://pastebin.test.redhat.com/1061265
http://pastebin.test.redhat.com/1061270

Comment 6 Vasishta 2023-03-24 17:28:51 UTC
While scrubbing BZs in 6.1 today, we decided to re-iterate through this BZ and highlight particulars that can be used to begin with to analyze.

Scrubbed available logs to cherry-pick particulars on issue that was tried to track earlier -

(In reply to Vasishta from comment #0)
> Additional info:
> Observation made on some of the images
> Links have cli output of machine's current time, snapshot schedule status,
> snap ls -
> http://pastebin.test.redhat.com/1061270
> http://pastebin.test.redhat.com/1061268
> http://pastebin.test.redhat.com/1061267
> http://pastebin.test.redhat.com/1061265
> http://pastebin.test.redhat.com/1061270

--------------------------------------------------------------------------------------
In above links 

1) http://pastebin.test.redhat.com/1061265 has `snapshot schedule status` and `snap ls --all` of image create_delete_magna006_143

In 3rd line (o/p of snapshot schedule status) it can be observed that there should have been a primary snapshot created at 04:27:00 (with acceptable delay)
But the actual next snapshot after  04:24:51 was 04:30:55



2) http://pastebin.test.redhat.com/1061267 has `snapshot schedule status` and `snap ls --all` of image create_delete_magna006_1

In 3rd (o/p of snapshot schedule status) it can be observed that there should have been a primary snapshot created at 05:57:00
But the next snap was created at 06:00:57 2022 (Last line)

3) http://pastebin.test.redhat.com/1061268

Expected - 05:33:00 (pastebin snippet doesn't have this recorded but following the pattern it can be observed)
Actual - 05:37:02 (expected is 5:36:00)

----------------------------------------------------------------------------------------


Note You need to log in before you can comment on or make changes to this bug.