Description of problem: The customized pool parameters set in the environment file are not passed to Ceph ansible. The parameters were set parameter_defaults: CinderEnableIscsiBackend: false CinderEnableRbdBackend: true CinderEnableNfsBackend: false NovaEnableRbdBackend: true GlanceBackend: rbd CinderRbdPoolName: "volumes" NovaRbdPoolName: "vms" GlanceRbdPoolName: "images" CephPoolDefaultPgNum: 32 CephAnsibleDisksConfig: devices: - '/dev/vdb' - '/dev/vdc' - '/dev/vdd' - '/dev/vde' - '/dev/vdf' journal_size: 512 osd_scenario: collocated CephPools: - {"name": volumes, "size": 3, "pg_num": 128, "rule_name": ""} - {"name": vms, "size": 1, "pg_num": 128, "rule_name": ""} The ceph-install-workflow.log shows: 2018-06-18 12:08:53,227 p=5065 u=mistral | ok: [192.168.24.15 -> 192.168.24.11] => (item=[{u'rule_name': u'', u'pg_num': 32, u'name': u'vms'}, {'_ansible_parsed': True, 'stderr_lines': [u"Error ENOENT: unrecognized pool 'vms'"], u'cmd': [u'docker', u'exec', u'ceph-mon-controller-0', u'ceph', u'--cluster', u'ceph', u'osd', u'pool', u'get', u'vms', u'size'], u'end': u'2018-06-18 16:08:46.940216', '_ansible_no_log': False, '_ansible_delegated_vars': {'ansible_delegated_host': u'192.168.24.11', 'ansible_host': u'192.168.24.11'}, '_ansible_item_result': True, u'changed': True, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': False, u'_raw_params': u'docker exec ceph-mon-controller-0 ceph --cluster ceph osd pool get vms size', u'removes': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'stdout': u'', u'start': u'2018-06-18 16:08:46.624634', u'delta': u'0:00:00.315582', 'item': {u'rule_name': u'', u'pg_num': 32, u'name': u'vms'}, u'rc': 2, u'msg': u'non-zero return code', 'stdout_lines': [], 'failed_when_result': False, u'stderr': u"Error ENOENT: unrecognized pool 'vms'", '_ansible_ignore_errors': None, u'failed': False}]) => {"changed": false, "cmd": ["docker", "exec", "ceph-mon-controller-0", "ceph", "--cluster", "ceph", "osd", "pool", "create", "vms", "32", "32", "replicated_rule", "1"], "delta": "0:00:01.142553", "end": "2018-06-18 16:08:53.428036", "item": [{"name": "vms", "pg_num": 32, "rule_name": ""}, {"_ansible_delegated_vars": {"ansible_delegated_host": "192.168.24.11", "ansible_host": "192.168.24.11"}, "_ansible_ignore_errors": null, "_ansible_item_result": true, "_ansible_no_log": false, "_ansible_parsed": true, "changed": true, "cmd": ["docker", "exec", "ceph-mon-controller-0", "ceph", "--cluster", "ceph", "osd", "pool", "get", "vms", "size"], "delta": "0:00:00.315582", "end": "2018-06-18 16:08:46.940216", "failed": false, "failed_when_result": false, "invocation": {"module_args": {"_raw_params": "docker exec ceph-mon-controller-0 ceph --cluster ceph osd pool get vms size", "_uses_shell": false, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "warn": true}}, "item": {"name": "vms", "pg_num": 32, "rule_name": ""}, "msg": "non-zero return code", "rc": 2, "start": "2018-06-18 16:08:46.624634", "stderr": "Error ENOENT: unrecognized pool 'vms'", "stderr_lines": ["Error ENOENT: unrecognized pool 'vms'"], "stdout": "", "stdout_lines": []}], "rc": 0, "start": "2018-06-18 16:08:52.285483", "stderr": "pool 'vms' created", "stderr_lines": ["pool 'vms' created"], "stdout": "", "stdout_lines": []} 2018-06-18 12:08:54,935 p=5065 u=mistral | ok: [192.168.24.15 -> 192.168.24.11] => (item=[{u'rule_name': u'', u'pg_num': 32, u'name': u'volumes'}, {'_ansible_parsed': True, 'stderr_lines': [u"Error ENOENT: unrecognized pool 'volumes'"], u'cmd': [u'docker', u'exec', u'ceph-mon-controller-0', u'ceph', u'--cluster', u'ceph', u'osd', u'pool', u'get', u'volumes', u'size'], u'end': u'2018-06-18 16:08:47.472141', '_ansible_no_log': False, '_ansible_delegated_vars': {'ansible_delegated_host': u'192.168.24.11', 'ansible_host': u'192.168.24.11'}, '_ansible_item_result': True, u'changed': True, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': False, u'_raw_params': u'docker exec ceph-mon-controller-0 ceph --cluster ceph osd pool get volumes size', u'removes': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'stdout': u'', u'start': u'2018-06-18 16:08:47.148688', u'delta': u'0:00:00.323453', 'item': {u'rule_name': u'', u'pg_num': 32, u'name': u'volumes'}, u'rc': 2, u'msg': u'non-zero return code', 'stdout_lines': [], 'failed_when_result': False, u'stderr': u"Error ENOENT: unrecognized pool 'volumes'", '_ansible_ignore_errors': None, u'failed': False}]) => {"changed": false, "cmd": ["docker", "exec", "ceph-mon-controller-0", "ceph", "--cluster", "ceph", "osd", "pool", "create", "volumes", "32", "32", "replicated_rule", "1"], "delta": "0:00:01.143402", "end": "2018-06-18 16:08:55.137711", "item": [{"name": "volumes", "pg_num": 32, "rule_name": ""}, {"_ansible_delegated_vars": {"ansible_delegated_host": "192.168.24.11", "ansible_host": "192.168.24.11"}, "_ansible_ignore_errors": null, "_ansible_item_result": true, "_ansible_no_log": false, "_ansible_parsed": true, "changed": true, "cmd": ["docker", "exec", "ceph-mon-controller-0", "ceph", "--cluster", "ceph", "osd", "pool", "get", "volumes", "size"], "delta": "0:00:00.323453", "end": "2018-06-18 16:08:47.472141", "failed": false, "failed_when_result": false, "invocation": {"module_args": {"_raw_params": "docker exec ceph-mon-controller-0 ceph --cluster ceph osd pool get volumes size", "_uses_shell": false, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "warn": true}}, "item": {"name": "volumes", "pg_num": 32, "rule_name": ""}, "msg": "non-zero return code", "rc": 2, "start": "2018-06-18 16:08:47.148688", "stderr": "Error ENOENT: unrecognized pool 'volumes'", "stderr_lines": ["Error ENOENT: unrecognized pool 'volumes'"], "stdout": "", "stdout_lines": []}], "rc": 0, "start": "2018-06-18 16:08:53.994309", "stderr": "pool 'volumes' created", "stderr_lines": ["pool 'volumes' created"], "stdout": "", "stdout_lines": []} Version-Release number of selected component (if applicable): openstack-tripleo-common-8.6.1-20.el7ost.noarch openstack-tripleo-common-containers-8.6.1-20.el7ost.noarch puppet-tripleo-8.3.2-8.el7ost.noarch openstack-tripleo-puppet-elements-8.0.0-2.el7ost.noarch openstack-tripleo-heat-templates-8.0.2-35.el7ost.noarch openstack-tripleo-validations-8.4.1-5.el7ost.noarch openstack-tripleo-image-elements-8.0.1-1.el7ost.noarch ceph-ansible-3.1.0-0.1.rc9.el7cp.noarch puppet-ceph-2.5.0-1.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy an overcloud with customize pool parameters 2. after the deployment is successful, validate the values are set in the pools Actual results: [root@controller-0 ~]# ceph --cluster ceph osd pool get vms size size: 3 [root@controller-0 ~]# ceph --cluster ceph osd pool get vms pg_num pg_num: 32 [root@controller-0 ~]# ceph --cluster ceph osd pool get volumes pg_num pg_num: 32 [root@controller-0 ~]# ceph --cluster ceph osd pool get volumes size size: 3 Expected results: [root@controller-0 ~]# ceph --cluster ceph osd pool get vms size size: 1 [root@controller-0 ~]# ceph --cluster ceph osd pool get vms pg_num pg_num: 128 [root@controller-0 ~]# ceph --cluster ceph osd pool get volumes pg_num pg_num: 128 [root@controller-0 ~]# ceph --cluster ceph osd pool get volumes size size: 3 Additional info:
Yogev, I think the custom env file is wrongly formatted. CephPool parameter should not be nested within CephAnsibleDisksConfig.