Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1354156 - RGW process crash during multisite sync operation
RGW process crash during multisite sync operation
Status: CLOSED ERRATA
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RGW (Show other bugs)
2.0
Unspecified Unspecified
unspecified Severity high
: rc
: 2.0
Assigned To: Casey Bodley
shilpa
:
Depends On:
Blocks: 1353972
  Show dependency treegraph
 
Reported: 2016-07-10 03:08 EDT by shilpa
Modified: 2017-07-31 16:59 EDT (History)
9 users (show)

See Also:
Fixed In Version: RHEL: ceph-10.2.2-23.el7cp Ubuntu: ceph_10.2.2-18redhat1xenial
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-08-23 15:43:52 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Ceph Project Bug Tracker 16665 None None None 2016-07-12 15:51 EDT
Red Hat Product Errata RHBA-2016:1755 normal SHIPPED_LIVE Red Hat Ceph Storage 2.0 bug fix and enhancement update 2016-08-23 19:23:52 EDT

  None (edit)
Description shilpa 2016-07-10 03:08:54 EDT
Description of problem:
While syncing data from master to non-master zone, the non-master rgw process crashed with a Segfault.

Version-Release number of selected component (if applicable):
ceph-radosgw-10.2.2-15.el7cp.x86_64

Steps to Reproduce:
1. Configure rgw1 as master zone and upload around 300G of data to master zone.
2. Configure rgw2 as the second zone and start syncing data from rgw1. 
3. Add a delay of about 150ms between the two nodes.


Actual results:
Around 300G of data was uploaded on rgw1 before starting the sync and rgw2 had no data in it. Once the sync started, another 300G were added to rgw1. Process crashed on rgw2 after syncing 300G of data. 

2016-07-09 22:06:38.136383 7fdd437fe700 -1 *** Caught signal (Segmentation fault) **
 in thread 7fdd437fe700 thread_name:radosgw

 ceph version 10.2.2-15.el7cp (60cd52496ca02bdde9c2f4191e617f75166d87b6)
 1: (()+0x54e22a) [0x7fde5db4822a]
 2: (()+0xf100) [0x7fde5cf79100]
 3: (RGWCoroutinesStack::wakeup()+0xa) [0x7fde5d8c44aa]
 4: (RGWBucketShardFullSyncCR::operate()+0x8a1) [0x7fde5d971531]
 5: (RGWCoroutinesStack::operate(RGWCoroutinesEnv*)+0x7e) [0x7fde5d8c3a2e]
 6: (RGWCoroutinesManager::run(std::list<RGWCoroutinesStack*, std::allocator<RGWCoroutinesStack*> >&)+0x3f1) [0x7fde5d8c59d1]
 7: (RGWCoroutinesManager::run(RGWCoroutine*)+0x70) [0x7fde5d8c6590]
 8: (RGWRemoteDataLog::run_sync(int, rgw_data_sync_status&)+0x352) [0x7fde5d96d912]
 9: (RGWDataSyncProcessorThread::process()+0x49) [0x7fde5da30289]
 10: (RGWRadosThread::Worker::entry()+0x133) [0x7fde5d9d5083]
 11: (()+0x7dc5) [0x7fde5cf71dc5]
 12: (clone()+0x6d) [0x7fde5c57bced]
Comment 3 Casey Bodley 2016-07-18 13:17:39 EDT
The fix has been cherry-picked to ceph-2-rhel-patches.
Comment 7 shilpa 2016-07-28 12:37:33 EDT
Not seen the crash since ceph-10.2.2-23. Moving to verified.
Comment 9 errata-xmlrpc 2016-08-23 15:43:52 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1755.html

Note You need to log in before you can comment on or make changes to this bug.