Hide Forgot
Created attachment 1242024 [details] playbook log Description of problem: The ceph-client role fails to take the default pg num value. The default_pg_num has a default value of 128. TASK [ceph-client : create pools] ********************************************** fatal: [magna061]: FAILED! => {"failed": true, "msg": "[{u'name': u'test2', u'pgs': u'{{ pool_default_pg_num }}'}]: 'pool_default_pg_num' is undefined"} to retry, use: --limit @/usr/share/ceph-ansible/site.retry Version-Release number of selected component (if applicable): ceph-ansible-2.1.3-1.el7scon.noarch ansible-2.2.1.0-1.el7.noarch How reproducible: Always Steps to Reproduce: 1. Install a ceph-client with a user config defined. Additional info: client.yml: user_config: true pools: # - { name: test, pgs: "{{ pool_default_pg_num }}" } - { name: test2, pgs: "{{ pool_default_pg_num }}" } keys: # - { name: client.test, value: "mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=test'" } - { name: client.test2, value: "mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=test2'" } ceph-ansible]# grep -r 'pool_default_pg_num' group_vars/clients.yml:# - { name: test, pgs: "{{ pool_default_pg_num }}" } group_vars/clients.yml: - { name: test2, pgs: "{{ pool_default_pg_num }}" } group_vars/clients.yml.sample:# - { name: test, pgs: "{{ pool_default_pg_num }}" } group_vars/clients.yml.sample:# - { name: test2, pgs: "{{ pool_default_pg_num }}" } group_vars/mons.yml:pool_default_pg_num: 128 <-------------- the default is defined here group_vars/mons.yml:# pg_num: "{{ pool_default_pg_num }}" group_vars/mons.yml:# pg_num: "{{ pool_default_pg_num }}" group_vars/mons.yml:# pg_num: "{{ pool_default_pg_num }}" group_vars/mons.yml:# pg_num: "{{ pool_default_pg_num }}" Attached the playbook log here.
It works when the pg number is defined in the clients.yml: pools: # - { name: test, pgs: "{{ pool_default_pg_num }}" } - { name: test2, pgs: "128" } Is it supposed to take this value from a default location such as group_vars/mons.yml:pool_default_pg_num: 128 Thanks, Tejas
It looks like pool_default_pg_num is actually defaulted in the ceph-mon role here: https://github.com/ceph/ceph-ansible/blob/stable-2.1/roles/ceph-mon/defaults/main.yml#L18 Because the variable is set only in the ceph-mon role the ceph-client role does not have access to it. As a temporary workaround could you please try setting pool_default_pg_num in group_vars/all.yml instead of group_vars/mon.yml? I'll make an upstream PR to address this as well.
PR opened upstream: https://github.com/ceph/ceph-ansible/pull/1236
Already addressed in this PR: https://github.com/ceph/ceph-ansible/pull/1146 I'll finalize this ASAP
Hi Andrew, The workaround where we define the "pool_default_pg_num' in all.yml works. Thanks, Tejas
Sounds like this fix could possibly make it in time for RH Ceph Storage 2.2 - re-targeting so we track this.
Opened a PR to backport the fix to the stable-2.1 upstream branch. https://github.com/ceph/ceph-ansible/pull/1245
Created attachment 1245908 [details] ansible playbook log
If you don't declare "ceph_conf_overrides.global.osd_pool_default_pg_num" the task will be skipped. I just logged on the machine and ceph_conf_overrides.global.osd_pool_default_pg_num is not declared, you have to set a value.
Hi Seb,Andrew, Thanks for the info. After defining the ceph_conf_overrides the pool creation succeeds. I will be moving this bug to Doc to document this procedure to use the ceph-client role. Thanks, Tejas
Sure, np!
Tejas, Maybe I'm confused here -- Should this BZ be in the NEW state instead? per https://mojo.redhat.com/docs/DOC-1126123 Anjana, Do you prefer code BZs to get repurposed as doc bugs? Or would you rather have a separate BZ that tracks the code BZ? cheers, G
Anjana, Please make it so.
Ken, Anjana, I think we need this as part of 2.2 release. I have changed the summary to reflect the purposee of this bug. Could we add this back to the 2.2 release? Thanks, Tejas
LGTM
This looks good, thanks. Seb, If you could take a look at this, it would be good. Thanks, Tejas
@Tejas, yes we can declare this in client.yml.
Bara, Seb, Since we have tested this with group_vars/all.yml, lets keep it that way. I hope thats okay. Thanks, Tejas