Bug 1881985 - Mon crash observed at std::_Rb_tree_rebalance_for_erase
Summary: Mon crash observed at std::_Rb_tree_rebalance_for_erase
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RADOS
Version: 4.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.*
Assignee: Neha Ojha
QA Contact: Manohar Murthy
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-23 14:24 UTC by Tejas
Modified: 2020-12-07 05:34 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-12-07 05:34:26 UTC
Embargoed:


Attachments (Terms of Use)

Description Tejas 2020-09-23 14:24:52 UTC
Description of problem:
  
Mon crashed on changing Cluster Ip subnet on the OSDs. The Cluster Ip was wrongly configured on the OSDs, so the ceph.conf on the MoN abd the OSDs were having different Cluster IP subnets.
On changing the Cluster IP subnet on the OSDs to match that on the Mon , and restarting all OSDs, the primary MON crashed.
No other MONs crashed.

Version-Release number of selected component (if applicable):
ceph version 14.2.8-50.el7cp

How reproducible:
observed only once



 ~]# ceph crash info 2020-09-23_06:19:50.288218Z_967aefe7-4aec-440e-9176-e0d6cb588206
{
    "os_version_id": "7.8", 
    "utsname_release": "3.10.0-1127.19.1.el7.x86_64", 
    "os_name": "Red Hat Enterprise Linux Server", 
    "entity_name": "mon.f03-h02-000-r620", 
    "timestamp": "2020-09-23 06:19:50.288218Z", 
    "process_name": "ceph-mon", 
    "utsname_machine": "x86_64", 
    "utsname_sysname": "Linux", 
    "os_version": "7.8 (Maipo)", 
    "os_id": "rhel", 
    "utsname_version": "#1 SMP Tue Aug 11 19:12:04 EDT 2020", 
    "backtrace": [
        "(()+0xf630) [0x7f05da68c630]", 
        "(std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)+0x184) [0x7f05d9da55a4]", 
        "(std::_Rb_tree<int, std::pair<int const, failure_reporter_t>, std::_Select1st<std::pair<int const, failure_reporter_t> >, std::less<int>, std::allocator<std::pair<int const, failure_reporter_t> > >::_M_erase_aux(std::_Rb_tree_const_iterator<std::pair<int const, failure_reporter_t> >)+0x15) [0x555967fc3a25]", 
        "(OSDMonitor::check_failure(utime_t, int, failure_info_t&)+0x951) [0x555967f51101]", 
        "(OSDMonitor::check_failures(utime_t)+0x61) [0x555967f516c1]", 
        "(OSDMonitor::tick()+0x1aa) [0x555967f7a3ea]", 
        "(Monitor::tick()+0x120) [0x555967e437e0]", 
        "(C_MonContext::finish(int)+0x39) [0x555967e1f3b9]", 
        "(Context::complete(int)+0x9) [0x555967e59e99]", 
        "(SafeTimer::timer_thread()+0x180) [0x7f05dd969290]", 
        "(SafeTimerThread::entry()+0xd) [0x7f05dd96aafd]", 
        "(()+0x7ea5) [0x7f05da684ea5]", 
        "(clone()+0x6d) [0x7f05d95488dd]"
    ], 
    "utsname_hostname": "f03-h02-000-r620.rdu2.scalelab.redhat.com", 
    "crash_id": "2020-09-23_06:19:50.288218Z_967aefe7-4aec-440e-9176-e0d6cb588206", 
    "ceph_version": "14.2.8-50.el7cp"
}


Note You need to log in before you can comment on or make changes to this bug.