Bug 1296518

Summary: Mon listing API does not provide details if mon is leader or not
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Shubhendu Tripathi <shtripat>
Component: CalamariAssignee: Christina Meno <gmeno>
Calamari sub component: Back-end QA Contact: Harish NV Rao <hnallurv>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: ceph-eng-bugs, flucifre, gmeno, hnallurv, icolle, jquinn, kdreyer, sankarshan
Version: 1.3.1   
Target Milestone: rc   
Target Release: 2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: calamari-server-1.4.1-1.el7cp Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-23 19:29:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1291304, 1343229    

Description Shubhendu Tripathi 2016-01-07 12:23:50 UTC
Description of problem:
Currently the mon listing api "/api/v2/cluster/<fsid>/mon" shows the below details

[
    {
        "name": "a", 
        "rank": 0, 
        "in_quorum": true, 
        "server": "<server>", 
        "addr": "<addr>"
    }
]

We need to make out if the mon is leader in the cluster. So add a field to mark the leader in the mons list something line below -

[
    {
        "name": "a", 
        "rank": 0, 
        "in_quorum": true, 
        "server": "<server>", 
        "addr": "<addr>",
        "leader": true/false
    }
]

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Create a ceph cluster with one MON and 3 OSD
2. Start calamari-light on the MON node
3. Access the ceph-api using URL "http://<node>:8002/api/v2/cluster/<fsid>/mon"

Actual results:
The mons listing does not mark is a mon is leader

Expected results:
The mons listing should mark the leader from the mons list

Additional info:

Comment 3 Christina Meno 2016-01-29 18:55:47 UTC
This can be had as a workaround from http://calamari/api/v2/cluster/<fsid>/sync_object/mon_status

adding it to the mon endpoint should take 2 days

Comment 4 Christina Meno 2016-04-14 13:16:07 UTC
Shubhendu

I am going to implement this as a filed called state like we get from ceph
http://docs.ceph.com/docs/master/rados/troubleshooting/troubleshooting-mon/#most-common-monitor-issues

state can be one of leader, peon, electing, probing, synchronizing

Comment 5 Christina Meno 2016-04-16 17:06:37 UTC
disregard comment#4 I was wrong, this has been implemented as described in initial request.

Comment 7 Harish NV Rao 2016-05-30 09:51:26 UTC
The fix is not working on 3 mons system.

calamari.log shows:

2016-05-30 04:47:10,610 - ERROR - django.request Internal Server Error: /api/v2/cluster/e9948b6a-cfb9-41d2-a032-662f3809e1c0/mon
Traceback (most recent call last):
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 115, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/rest_framework/viewsets.py", line 78, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/calamari_rest_api-0.1-py2.7.egg/calamari_rest/views/rpc_view.py", line 91, in dispatch
    return super(RPCViewSet, self).dispatch(request, *args, **kwargs)
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 77, in wrapped_view
    return view_func(*args, **kwargs)
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/rest_framework/views.py", line 399, in dispatch
    response = self.handle_exception(exc)
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/calamari_rest_api-0.1-py2.7.egg/calamari_rest/views/rpc_view.py", line 108, in handle_exception
    return super(RPCViewSet, self).handle_exception(exc)
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/rest_framework/views.py", line 396, in dispatch
    response = handler(request, *args, **kwargs)
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/calamari_rest_api-0.1-py2.7.egg/calamari_rest/views/v2.py", line 1142, in list
    return Response(self.serializer_class([DataObject(m) for m in self._get_mons(fsid)], many=True).data)
  File "/opt/calamari/venv/local/lib/python2.7/site-packages/calamari_rest_api-0.1-py2.7.egg/calamari_rest/views/v2.py", line 1060, in _get_mons
    local_epoch = service_info['status']['election_epoch']
TypeError: string indices must be integers, not str

Comment 8 Christina Meno 2016-06-09 04:15:55 UTC
https://github.com/ceph/calamari/pull/459

Comment 9 Ken Dreyer (Red Hat) 2016-06-14 16:24:55 UTC
What version of Calamari will contain this change?

Comment 10 Christina Meno 2016-06-14 17:45:09 UTC
v1.4.1

Comment 11 Christina Meno 2016-06-15 16:33:31 UTC
Didn't attach to the advisory because it was already there.

Comment 12 Shubhendu Tripathi 2016-06-17 12:18:15 UTC
Find it working with build calamari-server-1.4.2-1.el7cp.x86_64

Comment 13 Harish NV Rao 2016-06-21 17:32:45 UTC
Found working. Moving to verified state.

Tested on:
 calamari-server-1.4.2-1.el7cp.x86_64
 ceph version 10.2.2-5.el7cp 

Logs:
HTTP 200 OK
Vary: Accept
Content-Type: text/html; charset=utf-8
Allow: GET, HEAD, OPTIONS

[
    {
        "name": "magna055", 
        "rank": 0, 
        "in_quorum": true, 
        "server": "magna055.ceph.redhat.com", 
        "addr": "10.8.128.55:6789/0", 
        "leader": true
    }, 
    {
        "name": "magna068", 
        "rank": 1, 
        "in_quorum": true, 
        "server": "magna068.ceph.redhat.com", 
        "addr": "10.8.128.68:6789/0", 
        "leader": false
    }, 
    {
        "name": "magna071", 
        "rank": 2, 
        "in_quorum": true, 
        "server": "magna071.ceph.redhat.com", 
        "addr": "10.8.128.71:6789/0", 
        "leader": false
    }
]

Comment 15 errata-xmlrpc 2016-08-23 19:29:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1755