Bug 1494450

Summary: [ceph-iscsi-cli]: gwcli should fail cleanly if new GW config does not match existing GW
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Tejas <tchandra>
Component: iSCSIAssignee: Paul Cuzner <pcuzner>
Status: CLOSED ERRATA QA Contact: Tejas <tchandra>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.0CC: ceph-eng-bugs, ceph-qe-bugs, jdillama, tchandra
Target Milestone: rc   
Target Release: 3.0   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-05 23:44:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tejas 2017-09-22 09:53:09 UTC
Description of problem:
       I am creating a iSCSI setup from scratch. Created a GW using gwcli. While adding a new GW seeing a JSON error.
The reason is the iscsi-gateway.cfg  of both GWs differ in 1 parameter.

/iscsi-target...ceph/gateways> create nighthawk 10.70.39.17
CMD: ../gateways/ create nighthawk 10.70.39.17 nosync=False
Adding gateway, sync'ing 0 disk(s) and 0 client(s)
Traceback (most recent call last):
  File "/usr/bin/gwcli", line 187, in <module>
    main()
  File "/usr/bin/gwcli", line 119, in main
    shell.run_interactive()
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 905, in run_interactive
    self._cli_loop()
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 734, in _cli_loop
    self.run_cmdline(cmdline)
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 848, in run_cmdline
    self._execute_command(path, command, pparams, kparams)
  File "/usr/lib/python2.7/site-packages/configshell_fb/shell.py", line 823, in _execute_command
    result = target.execute_command(command, pparams, kparams)
  File "/usr/lib/python2.7/site-packages/configshell_fb/node.py", line 1406, in execute_command
    return method(*pparams, **kparams)
  File "/usr/lib/python2.7/site-packages/gwcli/gateway.py", line 519, in ui_command_create
    msg = api.response.json()['message']
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded



oot@nighthawk ~]# cat /etc/ceph/iscsi-gateway.cfg 
# This is seed configuration used by the ceph_iscsi_config modules
# when handling configuration tasks for iscsi gateway(s)
#
# Please do not change this file directly since it is managed by Ansible and will be overwritten

[config]
cluster_name = ceph
gateway_keyring = ceph.client.admin.keyring


# Optional settings related to the CLI/API service
#api_user = admin
#api_password = admin
#api_port = 5001
api_secure = true      <---------- differs from the other GW
#loop_delay = .5
trusted_ip_list = 10.70.39.14,10.70.39.17

The api_secure was false on the already added GW, and it was true on this node.


Trying from the local node :
/iscsi-target...ceph/gateways> create nighthawk 10.70.39.17
CMD: ../gateways/ create nighthawk 10.70.39.17 nosync=False
Adding gateway, sync'ing 0 disk(s) and 0 client(s)
Failed : Gateway creation failed, gateway(s) unavailable:10.70.39.14(Down),10.70.39.17(Up)


rbd-target-api logs:

2017-09-22 14:04:38,424     INFO [_internal.py:87:_log()] - 127.0.0.1 - - [22/Sep/2017 14:04:38] "PUT /api/gateway/nighthawk HTTP/1.1" 500 -
2017-09-22 14:07:00,660    ERROR [_internal.py:87:_log()] - 10.70.39.17 - - [22/Sep/2017 14:07:00] code 400, message Bad HTTP/0.9 request type ('\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03\x05qW\xb2\x1f\x8eM\x19s\x04\x7f\xdf#\xe5I')
2017-09-22 14:07:00,660     INFO [_internal.py:87:_log()] - 10.70.39.17 - - [22/Sep/2017 14:07:00] "^V^C^A^B^@^A^@^Aü^C^C^EqW²^_<8e>M^Ys^D^?ß#åI ú^Eæ<83>¢+]<89>¥cZY<8d>P§<8e>^@^@<9e>À0À,À2À.À/À+À1À-^@¥^@£^@¡^@<9f>^@¤^@¢^@ ^@<9e>À(À$À^TÀ" 400 -
2017-09-22 14:09:08,198     INFO [_internal.py:87:_log()] - 127.0.0.1 - - [22/Sep/2017 14:09:08] "GET /api/config HTTP/1.1" 200 -
2017-09-22 14:09:18,959     INFO [_internal.py:87:_log()] - 127.0.0.1 - - [22/Sep/2017 14:09:18] "GET /api/config HTTP/1.1" 200 -
2017-09-22 14:09:18,973     INFO [_internal.py:87:_log()] - 127.0.0.1 - - [22/Sep/2017 14:09:18] "PUT /api/gateway/nighthawk HTTP/1.1" 500 -





Version-Release number of selected component (if applicable):

ceph-iscsi-cli-2.5-6.el7cp.noarch
ceph-iscsi-config-2.3-7.el7cp.noarch

Comment 2 Jason Dillaman 2017-09-24 15:24:24 UTC
@Tejas: how did you get into this state where the configs were different on each node?

Comment 3 Tejas 2017-09-25 03:17:06 UTC
Jason,

   I was trying to add the 2nd GW node with the api_secure parameter different. 1 GW was present already.

- Tejas

Comment 6 Tejas 2017-09-29 08:59:22 UTC
Verified on ceph-iscsi-cli-2.5-8.el7cp.noarch

Comment 9 errata-xmlrpc 2017-12-05 23:44:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3387