Bug 1414213 - [ceph-client] Need documentation about ceph client role in ceph-ansible
Summary: [ceph-client] Need documentation about ceph client role in ceph-ansible
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Documentation
Version: 2.2
Hardware: Unspecified
OS: Unspecified
Target Milestone: rc
: 2.2
Assignee: Bara Ancincova
QA Contact: Tejas
Depends On:
TreeView+ depends on / blocked
Reported: 2017-01-18 03:50 UTC by Tejas
Modified: 2017-03-21 23:50 UTC (History)
12 users (show)

Fixed In Version: ceph-ansible-2.1.4-1.el7scon
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2017-03-21 23:50:21 UTC
Target Upstream Version:

Attachments (Terms of Use)
playbook log (161.55 KB, text/plain)
2017-01-18 03:50 UTC, Tejas
no flags Details
ansible playbook log (227.69 KB, text/plain)
2017-01-30 14:36 UTC, Tejas
no flags Details

Description Tejas 2017-01-18 03:50:25 UTC
Created attachment 1242024 [details]
playbook log

Description of problem:
      The ceph-client role fails to take the default pg num value. The default_pg_num has a default value of 128.

TASK [ceph-client : create pools] **********************************************
fatal: [magna061]: FAILED! => {"failed": true, "msg": "[{u'name': u'test2', u'pgs': u'{{ pool_default_pg_num }}'}]: 'pool_default_pg_num' is undefined"}
	to retry, use: --limit @/usr/share/ceph-ansible/site.retry

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Install a ceph-client with a user config defined.

Additional info:

user_config: true
#  - { name: test, pgs: "{{ pool_default_pg_num }}" }
  - { name: test2, pgs: "{{ pool_default_pg_num }}" }

#  - { name: client.test, value: "mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=test'" }
  - { name: client.test2, value: "mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=test2'" }

ceph-ansible]# grep -r 'pool_default_pg_num'

group_vars/clients.yml:#  - { name: test, pgs: "{{ pool_default_pg_num }}" }
group_vars/clients.yml:  - { name: test2, pgs: "{{ pool_default_pg_num }}" }
group_vars/clients.yml.sample:#  - { name: test, pgs: "{{ pool_default_pg_num }}" }
group_vars/clients.yml.sample:#  - { name: test2, pgs: "{{ pool_default_pg_num }}" }
group_vars/mons.yml:pool_default_pg_num: 128                            <-------------- the default is defined here
group_vars/mons.yml:#  pg_num: "{{ pool_default_pg_num }}"
group_vars/mons.yml:#  pg_num: "{{ pool_default_pg_num }}"
group_vars/mons.yml:#  pg_num: "{{ pool_default_pg_num }}"
group_vars/mons.yml:#  pg_num: "{{ pool_default_pg_num }}"

Attached the playbook log here.

Comment 2 Tejas 2017-01-18 05:18:40 UTC
It works when the pg number is defined in the clients.yml:
#  - { name: test, pgs: "{{ pool_default_pg_num }}" }
  - { name: test2, pgs: "128" }

Is it supposed to take this value from a default location such as 
group_vars/mons.yml:pool_default_pg_num: 128


Comment 3 Andrew Schoen 2017-01-18 12:24:35 UTC
It looks like pool_default_pg_num is actually defaulted in the ceph-mon role here: https://github.com/ceph/ceph-ansible/blob/stable-2.1/roles/ceph-mon/defaults/main.yml#L18

Because the variable is set only in the ceph-mon role the ceph-client role does not have access to it. As a temporary workaround could you please try setting pool_default_pg_num in group_vars/all.yml instead of group_vars/mon.yml?

I'll make an upstream PR to address this as well.

Comment 4 Andrew Schoen 2017-01-18 13:16:20 UTC
PR opened upstream: https://github.com/ceph/ceph-ansible/pull/1236

Comment 5 seb 2017-01-18 13:44:10 UTC
Already addressed in this PR: https://github.com/ceph/ceph-ansible/pull/1146
I'll finalize this ASAP

Comment 6 Tejas 2017-01-19 11:47:50 UTC
Hi Andrew, 

     The workaround where we define the "pool_default_pg_num' in all.yml works.


Comment 7 Ken Dreyer (Red Hat) 2017-01-20 04:32:38 UTC
Sounds like this fix could possibly make it in time for RH Ceph Storage 2.2 - re-targeting so we track this.

Comment 8 Andrew Schoen 2017-01-23 16:23:46 UTC
Opened a PR to backport the fix to the stable-2.1 upstream branch.


Comment 13 Tejas 2017-01-30 14:36:18 UTC
Created attachment 1245908 [details]
ansible playbook  log

Comment 14 seb 2017-01-30 14:45:47 UTC
If you don't declare "ceph_conf_overrides.global.osd_pool_default_pg_num" the task will be skipped. I just logged on the machine and ceph_conf_overrides.global.osd_pool_default_pg_num is not declared, you have to set a value.

Comment 16 Tejas 2017-01-31 03:47:24 UTC
Hi Seb,Andrew,

    Thanks for the info. After defining the ceph_conf_overrides the pool creation succeeds.

I will be moving this bug to Doc to document this procedure to use the ceph-client role.


Comment 17 seb 2017-01-31 07:56:11 UTC
Sure, np!

Comment 19 Christina Meno 2017-02-02 17:33:23 UTC

Maybe I'm confused here -- Should this BZ be in the NEW state instead?
per https://mojo.redhat.com/docs/DOC-1126123


Do you prefer code BZs to get repurposed as doc bugs?
Or would you rather have a separate BZ that tracks the code BZ?


Comment 21 Christina Meno 2017-02-15 19:03:11 UTC
Please make it so.

Comment 23 Tejas 2017-02-20 13:40:19 UTC
Ken, Anjana,

    I think we need this as part of 2.2 release. I have changed the summary to reflect the purposee of this bug. Could we add this back to the 2.2 release?


Comment 25 seb 2017-02-28 14:32:25 UTC

Comment 29 Tejas 2017-03-01 12:00:41 UTC
This looks good, thanks. 

   If you could take a look at this, it would be good.


Comment 30 seb 2017-03-01 14:11:34 UTC
@Tejas, yes we can declare this in client.yml.

Comment 32 Tejas 2017-03-02 11:05:58 UTC
Bara, Seb,

   Since we have tested this with group_vars/all.yml, lets keep it that way. I hope thats okay.


Note You need to log in before you can comment on or make changes to this bug.