Bug 1339486 - Start calamari post mon configure on a single mon does not work
Summary: Start calamari post mon configure on a single mon does not work
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Calamari
Version: 2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: 2.0
Assignee: Christina Meno
QA Contact: Rachana Patel
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-25 07:47 UTC by Shubhendu Tripathi
Modified: 2016-08-23 19:39 UTC (History)
8 users (show)

Fixed In Version: RHEL: calamari-server-1.4.0-0.12.rc15.el7cp Ubuntu: calamari-server_1.4.0~rc15-2redhat1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-23 19:39:26 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1755 0 normal SHIPPED_LIVE Red Hat Ceph Storage 2.0 bug fix and enhancement update 2016-08-23 23:23:52 UTC

Description Shubhendu Tripathi 2016-05-25 07:47:55 UTC
Description of problem:
Post enablenment of https for calamari, and as suggested we are trying to start calamari on single mon using the command "calamari-ctl initialize --admin-username <admin> --admin-password <admin> --admin-email <skyring@redhat.com>".

After running this command, the command "supervisorctl status" shows calamari running on the mon node, but URL https://<mon>:8002/api/v2/cluster does not respond and login page does not prompt for login.

Version-Release number of selected component (if applicable):


How reproducible:
Alway

Steps to Reproduce:
1. 
2.
3.

Actual results:
Even if calamari shows running on the node, the URL is not accessible.

Expected results:
After start, the calamari should be working fine and all the URLs should be accessible.

Additional info:

From USM code, the flow of code invocation is as below
1. Install mon with calamari=true (installs calamari on the mon node)
2. Configure mon with calamari=false (because we want to start calaamri explicitly on only one mon as suggested)
3. Invoke calamari starting code (salt SLS file) to start calamari

The content of the SLS file is as below

--------------------------
start_ceph_calamari:
  cmd.run:
    - name: calamari-ctl initialize --admin-username admin --admin-password admin --admin-email skyring@redhat.com; service supervisord restart; supervisorctl restart all

Comment 2 Christina Meno 2016-05-25 22:26:55 UTC
[admin@gmeno-clot-2 ~]$ sudo salt-call --local state.template start-cala.sls 
[INFO    ] Loading fresh modules for state activity
[INFO    ] Running state [calamari-ctl initialize --admin-username admin --admin-password admin --admin-email skyring@redhat.com;] at time 18:18:22.149650
[INFO    ] Executing state cmd.run for calamari-ctl initialize --admin-username admin --admin-password admin --admin-email skyring@redhat.com;
[INFO    ] Executing command 'calamari-ctl initialize --admin-username admin --admin-password admin --admin-email skyring@redhat.com;' in directory '/root'
[INFO    ] {'pid': 36214, 'retcode': 0, 'stderr': '[INFO] Loading configuration..\n[INFO] Starting/enabling postgres...\n[INFO] Updating database...', 'stdout': 'GMENO/opt/calamari/salt-local/postgres.sls\nGMENO local:\n    Data failed to compile:\n----------\n    The function "state.template" is running as PID 36170 and was started at 2016, May 25 18:18:21.710346 with jid 20160525181821710346\n[DEBUG   ] Reading configuration from /etc/salt/minion\n[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: gmeno-clot-2.os1.phx2.redhat.com\n[DEBUG   ] Configuration file path: /etc/salt/minion\n[DEBUG   ] LazyLoaded jinja.render\n[DEBUG   ] LazyLoaded yaml.render\n[DEBUG   ] LazyLoaded jinja.render\n[DEBUG   ] LazyLoaded yaml.render\n[DEBUG   ] LazyLoaded state.template\n[DEBUG   ] LazyLoaded saltutil.is_running\n[DEBUG   ] LazyLoaded highstate.output'}
[INFO    ] Completed state [calamari-ctl initialize --admin-username admin --admin-password admin --admin-email skyring@redhat.com;] at time 18:18:27.283471
[INFO    ] Running state [service supervisord restart; supervisorctl restart all] at time 18:18:27.285278
[INFO    ] Executing state cmd.run for service supervisord restart; supervisorctl restart all
[INFO    ] Executing command 'service supervisord restart; supervisorctl restart all' in directory '/root'
[INFO    ] {'pid': 36239, 'retcode': 0, 'stderr': 'Redirecting to /bin/systemctl restart  supervisord.service', 'stdout': 'calamari-lite: started'}
[INFO    ] Completed state [service supervisord restart; supervisorctl restart all] at time 18:18:29.531969
local:
----------
          ID: start_ceph_calamari
    Function: cmd.run
        Name: calamari-ctl initialize --admin-username admin --admin-password admin --admin-email skyring@redhat.com;
      Result: True
     Comment: Command "calamari-ctl initialize --admin-username admin --admin-password admin --admin-email skyring@redhat.com;" run
     Started: 18:18:22.149650
    Duration: 5133.821 ms
     Changes:   
              ----------
              pid:
                  36214
              retcode:
                  0
              stderr:
                  [INFO] Loading configuration..
                  [INFO] Starting/enabling postgres...
                  [INFO] Updating database...
              stdout:
                  GMENO/opt/calamari/salt-local/postgres.sls
                  GMENO local:
                      Data failed to compile:
                  ----------
                      The function "state.template" is running as PID 36170 and was started at 2016, May 25 18:18:21.710346 with jid 20160525181821710346
                  [DEBUG   ] Reading configuration from /etc/salt/minion
                  [DEBUG   ] Using cached minion ID from /etc/salt/minion_id: gmeno-clot-2.os1.phx2.redhat.com
                  [DEBUG   ] Configuration file path: /etc/salt/minion
                  [DEBUG   ] LazyLoaded jinja.render
                  [DEBUG   ] LazyLoaded yaml.render
                  [DEBUG   ] LazyLoaded jinja.render
                  [DEBUG   ] LazyLoaded yaml.render
                  [DEBUG   ] LazyLoaded state.template
                  [DEBUG   ] LazyLoaded saltutil.is_running
                  [DEBUG   ] LazyLoaded highstate.output
----------
          ID: restart_things
    Function: cmd.run
        Name: service supervisord restart; supervisorctl restart all
      Result: True
     Comment: Command "service supervisord restart; supervisorctl restart all" run
     Started: 18:18:27.285278
    Duration: 2246.691 ms
     Changes:   
              ----------
              pid:
                  36239
              retcode:
                  0
              stderr:
                  Redirecting to /bin/systemctl restart  supervisord.service
              stdout:
                  calamari-lite: started

Summary
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
[admin@gmeno-clot-2 ~]$

Comment 3 Christina Meno 2016-05-25 22:29:40 UTC
seems like salt calling salt results in "Data failed to compile:" for /opt/calamari/salt-local/postgres.sls

two options present themselves:
1. call the salt-gurus for magic that will cause this to work
2. yank the things I need out of salt and make the calls in python

Comment 4 Christina Meno 2016-05-26 21:52:16 UTC
tagged upstream as v1.4.0-rc14
Would you please build downstream?

Comment 9 Rachana Patel 2016-07-07 18:33:52 UTC
version:-
ceph version 10.2.2-9.el7cp (ece2b6d4849364cdca71fb66895409b0f34c199f)
ceph-installer-1.0.12-3.el7scon.noarch
calamari-server-1.4.4-1.el7cp.x86_64


Steps:-
1. did pre installation tasks on all nodes
2. run setup, setup/agent, setup/key and api/agent APIs.
3. run mon/install with calamari=true
4. run mon/configure with calamari=false from 2 mon nodes
5. on first mon executed below commands

[root@magna078 ubuntu]# ps auxww | grep calamari
root     28932  0.0  0.0 112648   972 pts/0    S+   18:09   0:00 grep --color=auto calamari
[root@magna078 ubuntu]# calamari-ctl initialize --admin-username ceph --admin-password admin --admin-email skyring@redhat.com; service supervisord restart; supervisorctl restart all
[INFO] Loading configuration..
[INFO] Starting/enabling postgres...
[INFO] Initializing database...
Generating a 2048 bit RSA private key
......................................................................................+++
...+++
writing new private key to '/etc/calamari/ssl/private/calamari-lite.key'
-----
Redirecting to /bin/systemctl restart  supervisord.service
calamari-lite: stopped
calamari-lite: started
[root@magna078 ubuntu]# ps auxww | grep calamari
root     29548  5.6  0.1 1658408 48248 ?       Sl   18:10   0:00 /opt/calamari/venv/bin/python /opt/calamari/venv/bin/calamari-lite
root     29734  0.0  0.0 112648   976 pts/0    S+   18:10   0:00 grep --color=auto calamari
[root@magna078 ubuntu]# supervisorctl status
calamari-lite                    RUNNING    pid 29548, uptime 0:02:47


6. try to access Web UI

http://magna078.xx.xx.com/
http://magna078.xx.xx.com:8002/api/v2/cluster

unable to access web UI


Let me know If I am missing something  else will move this bug to Failed_QA .

Comment 11 Christina Meno 2016-07-19 21:51:08 UTC
This is not a bug, calamari no longer responds to http requests.
It will only answer on https://

Rachana, please re-test. I am sorry that I missed this AND I'm sure I would have heard about it from the console team if it was failing.

Comment 12 Rachana Patel 2016-08-07 15:44:08 UTC
Considering comment#11 from Gregory, verified as below

Steps:-
1. did pre installation tasks on all nodes
2. run setup, setup/agent, setup/key and api/agent APIs.
3. run mon/install with calamari=true
4. run mon/configure with calamari=false from 2 mon nodes
5. on first mon executed below commands

[root@magna094 ubuntu]# calamari-ctl initialize --admin-username ceph --admin-password admin --admin-email skyring@redhat.com; service supervisord restart; supervisorctl restart all
[INFO] Loading configuration..
[INFO] Starting/enabling postgres...
[INFO] Initializing database...
Generating a 2048 bit RSA private key
......................................................................................+++
...+++
writing new private key to '/etc/calamari/ssl/private/calamari-lite.key'
-----
Redirecting to /bin/systemctl restart  supervisord.service
calamari-lite: stopped
calamari-lite: started

[root@magna094 ubuntu]# supervisorctl status
calamari-lite                    RUNNING    pid 2948, uptime 0:01:24


Hence moving to Verified

Comment 14 errata-xmlrpc 2016-08-23 19:39:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1755.html


Note You need to log in before you can comment on or make changes to this bug.