Bug 1662655 - Using default Ceph environment causes the deployment failure
Summary: Using default Ceph environment causes the deployment failure
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: ceph-ansible
Version: unspecified
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: ---
Assignee: John Fulton
QA Contact: Yogev Rabl
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-31 09:52 UTC by Amit Ugol
Modified: 2019-01-15 08:40 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-09 14:39:02 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Amit Ugol 2018-12-31 09:52:02 UTC
Description of problem:
Using ceph-ansible to deploy an environment with ceph storage will not work OOB without providing extra parameters.

Version-Release number of selected component (if applicable):
ceph-ansible-3.2.0-1.el7cp.noarch

How reproducible:
100%

Steps to Reproduce:
deploy in the following way:

openstack overcloud deploy \
    --templates \
    --libvirt-type kvm \
    --ntp-server clock.redhat.com \
    -e /home/stack/environments/node_scale.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \
    -e /home/stack/environments/ceph_devices.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/net-multiple-nics.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
    -e /home/stack/containers-prepare-parameter.yaml

ceph_devices.yaml is the only configuration that I expect to have to supply here because I used 5 OSDs:

parameter_defaults:
    CephAnsibleDisksConfig:
        devices:
            - '/dev/vdb'
            - '/dev/vdc'
            - '/dev/vdd'
            - '/dev/vde'
            - '/dev/vdf'

Actual results:
see bug https://bugzilla.redhat.com/show_bug.cgi?id=1586155
see bug https://bugzilla.redhat.com/show_bug.cgi?id=1502878

Comment 1 John Fulton 2019-01-02 13:07:52 UTC
Please provide (In reply to Amit Ugol from comment #0)
> Description of problem:
> Using ceph-ansible to deploy an environment with ceph storage will not work
> OOB without providing extra parameters.

That's unavoidable and not a bug. You need to appropriately plan a Ceph deployment as a function of your hardware in all current versions of Ceph.

> ceph_devices.yaml is the only configuration that I expect to have to supply
> here because I used 5 OSDs:

Why do you have that expectation? How many OSD nodes do you have?

The docs do mention that you need to set your PG count and refer you to the PG count calculator so that's one parameter you will have to override in addition to your devices list. I specifically mention this because you mentioned bz 1502878.

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html-single/deploying_an_overcloud_with_containerized_red_hat_ceph/#custom-ceph-pools

> Actual results:
> see bug https://bugzilla.redhat.com/show_bug.cgi?id=1586155

That's specific to deploying a version N overcloud with an N-1 undercloud. I don't think it clarifies whatever issue you hit.

> see bug https://bugzilla.redhat.com/show_bug.cgi?id=1502878

If you got something along the lines of ["Error ERANGE:  pg_num 128 size 3 would mean 768 total pgs, which exceeds max 600 (mon_max_pg_per_osd 200 * num_in_osds 3)"] then please confirm so we can troubleshoot that issue.

In order to proceed I will need your ceph-ansible run log. For OSP14 by default, config-download will place that in /var/lib/mistral/overcloud/ceph-ansible/

If you would please tar that up and attach a reference to it in this BZ, then I'd be happy to try to figure out what happened.

Comment 4 John Fulton 2019-01-09 14:39:02 UTC
We haven't gotten your logs. Feel free to re-open and if you can provide the requested information.


Note You need to log in before you can comment on or make changes to this bug.