Bug 1655350 - containerized deployment using osd_scenario lvm fails
Summary: containerized deployment using osd_scenario lvm fails
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Ceph-Ansible
Version: 3.1
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: rc
: 3.*
Assignee: Sébastien Han
QA Contact: ceph-qe-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-03 00:35 UTC by Heðin
Modified: 2018-12-03 22:03 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-12-03 22:03:39 UTC
Embargoed:


Attachments (Terms of Use)

Description Heðin 2018-12-03 00:35:19 UTC
Description of problem:
rhcs deployment fails on osd deployment, when running ansible-playbook -i hosts.yml site-docker.yml after configuring ceph-ansible according to the RHCS 3 container guide dated 2018-11-06

Version-Release number of selected component (if applicable):
rhcs-3.1
ceph-anisble-3.1.10 from downstream repository.
rhel-7.6

How reproducible:
100%

Steps to Reproduce:
1. Spin up 3 rhel-7.6 minimal instances with at least one osd disk.
2. Install ceph-ansible.
3. Follow the above mentioned documentation.
4. Run the playbook: ansible-playbook -i hosts.yml site-docker.yml

Actual results:
This result is from the ceph-ansible-3.2 playbook.
2018-12-03 00:08:43,379 p=28549 u=cephadmin |  TASK [ceph-osd : use ceph-volume to create bluestore osds] *********************************************************
2018-12-03 00:08:43,379 p=28549 u=cephadmin |  Monday 03 December 2018  00:08:43 +0000 (0:00:00.086)       0:08:00.784 *******
2018-12-03 00:08:49,605 p=28549 u=cephadmin |  failed: [satv-rheltest01] (item={u'data_vg': u'ceph_vg_sdc', u'data': u'ceph_lv_sdc'}) => {"changed": false, "failed": true, "item": {"data": "ceph_lv_sdc", "data_vg": "ceph_vg_sdc"}, "msg": "Could not decode json output:  stdout: /dev/sda2\";\"\";\"CeZ2Pn-QOOv-72tw-HlVr-5MTc-JAKi-f8dxpr\";\"rhel\n stdout: /dev/sdb\";\"\";\"iEqNzZ-p5jz-GtAR-9pA4-2nf7-1kqR-vFYlxC\";\"ceph_vg_nvme_b\n stdout: /dev/sdc\";\"\";\"XcdhJo-h9VA-Hgqw-B7yH-EUSp-Hdpz-2un7fW\";\"ceph_vg_sdc\n stdout: /dev/sdd\";\"\";\"v64kkf-M6Xe-g2Ou-QeeK-2nMK-1PD8-SreewW\";\"ceph_vg_sdd\n stdout: /dev/sde\";\"\";\"wI6Ay7-aU5W-0uFl-KOzF-Nk0O-7J1r-aNM12M\";\"ceph_vg_sde\n stderr: Operation prohibited while global/metadata_read_only is set.\n stderr: Recovery of standalone physical volumes failed.\n stderr: Cannot process standalone physical volumes\n stderr: Operation prohibited while global/metadata_read_only is set.\n stderr: Recovery of standalone physical volumes failed.\n stderr: Cannot process standalone physical volumes\n stderr: Operation prohibited while global/metadata_read_only is set.\n stderr: Recovery of standalone physical volumes failed.\n stderr: Cannot process standalone physical volumes\n{}\n from the command ['docker', 'run', '--rm', '--privileged', '--net=host', '-v', '/run/lock/lvm:/run/lock/lvm:z', '-v', '/dev:/dev', '-v', '/etc/ceph:/etc/ceph:z', '-v', '/run/lvm/lvmetad.socket:/run/lvm/lvmetad.socket', '-v', '/var/lib/ceph/:/var/lib/ceph/:z', '-v', '/var/log/ceph/:/var/log/ceph/:z', '--entrypoint=ceph-volume', 'registry.access.redhat.com/rhceph/rhceph-3-rhel7:latest', '--cluster', 'ceph', 'lvm', 'list', 'ceph_vg_sdc/ceph_lv_sdc', '--format=json']", "rc": 1}
2018-12-03 00:08:49,609 p=28549 u=cephadmin |  PLAY RECAP *********************************************************


Expected results:
deployed ceph cluster with 1 osd.

Additional info:
I have shared the playbook for 2 weeks: https://cloud.sa.fo/s/88QCLpabt6YsWni

the *3.1.10 files in groupvars are the ones I used in the ceph-anisble-3.1.10 playbook. (I trimmed the disk count down to 1 when i tried cpeh-anisble-3.2)

Comment 3 seb 2018-12-03 16:05:31 UTC
The error looks a bit familiar, thanks for sharing the logs.
Alfredo do you know if the patch that was fixing the output of the list in json format made it in 3.0?

Thanks.

Comment 4 Alfredo Deza 2018-12-03 21:51:53 UTC
Ceph 3.0 did not have the --readonly flag.

Comment 5 seb 2018-12-03 22:03:39 UTC
Thanks, I'm closing this then. If you're looking at using ceph-volume please wait for the 3.2 release. Thanks.


Note You need to log in before you can comment on or make changes to this bug.