Bug 1640029 - [ceph-ansible] - rolling update failing in osd creation task saying 'No JSON object could be decoded'
Summary: [ceph-ansible] - rolling update failing in osd creation task saying 'No JSON ...
Status: CLOSED DUPLICATE of bug 1640086
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Ceph-Ansible
Version: 3.2
Hardware: Unspecified
OS: Unspecified
Target Milestone: rc
: 3.2
Assignee: leseb
QA Contact: Vasishta
Depends On:
TreeView+ depends on / blocked
Reported: 2018-10-17 07:46 UTC by Vasishta
Modified: 2018-10-17 12:58 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2018-10-17 12:58:27 UTC
Target Upstream Version:

Attachments (Terms of Use)
File contans rolling update log snippet (976.65 KB, text/plain)
2018-10-17 07:46 UTC, Vasishta
no flags Details

Description Vasishta 2018-10-17 07:46:13 UTC
Created attachment 1494763 [details]
File contans rolling update log snippet

Description of problem:
When it was tried to upgrade cepph cluster from one build of 3.2 to latest 3.2 build, rollling updated failed in task [ceph-osd : use ceph-volume to create bluestore osds] saying  raise ValueError(\"No JSON object could be decoded

Version-Release number of selected component (if applicable):

How reproducible:
Always (5/5)

Steps to Reproduce:
1. Bring up 3.2 cluster with n-1 build with osd_scenario lvm and providing devices for lvm_volumes.
2. Upgrade ceph-ansible and run upgrade to update cluster.

Actual results:
failed: [xxxxx] (item={u'data': u'/dev/sdd'}) => {
    "changed": false, 
    "item": {
        "data": "/dev/sdd"
    "module_stderr": "OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 8: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 72956\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\nTraceback (most recent call last):\n  File \"/tmp/ansible_dbAugM/ansible_module_ceph_volume.py\", line 625, in <module>\n    main()\n  File \"/tmp/ansible_dbAugM/ansible_module_ceph_volume.py\", line 621, in main\n    run_module()\n  File \"/tmp/ansible_dbAugM/ansible_module_ceph_volume.py\", line 521, in run_module\n    out_dict = json.loads(out)\n  File \"/usr/lib64/python2.7/json/__init__.py\", line 338, in loads\n    return _default_decoder.decode(s)\n  File \"/usr/lib64/python2.7/json/decoder.py\", line 366, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/usr/lib64/python2.7/json/decoder.py\", line 384, in raw_decode\n    raise ValueError(\"No JSON object could be decoded\")\nValueError: No JSON object could be decoded\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 1

Expected results:
CLuster must get updated without any issues

Additional info:

$ cat group_vars/all.yml | egrep -v ^# | grep -v ^$
fetch_directory: ~/ceph-ansible-keys
ceph_origin: distro
ceph_repository: rhcs
ceph_rhcs_version: 3
monitor_interface: eno1

$  cat group_vars/osds.yml | egrep -v ^# | grep -v ^$
osd_scenario: lvm
   - data: /dev/sdb
   - data: /dev/sdc
   - data: /dev/sdd

Comment 3 leseb 2018-10-17 12:58:27 UTC

*** This bug has been marked as a duplicate of bug 1640086 ***

Note You need to log in before you can comment on or make changes to this bug.