Bug 1593866
| Summary: | ceph-ansible fails to restart services after ceph_conf_overrides is changed and rerun | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Vasu Kulkarni <vakulkar> |
| Component: | Ceph-Ansible | Assignee: | Sébastien Han <shan> |
| Status: | CLOSED NOTABUG | QA Contact: | ceph-qe-bugs <ceph-qe-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 3.1 | CC: | aschoen, ceph-eng-bugs, dfuller, gabrioux, gmeno, jharriga, nthomas, sankarshan, vakulkar |
| Target Milestone: | rc | ||
| Target Release: | 3.1 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-07-11 14:14:41 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1581350 | ||
I ran the sequence of commands again and now see that one of the values
(rgw_thread_pool_size) gets applied while the other (objecter_inflight_ops)
does not:
1) edit 'group_vars/all.yml' as such:
# JTH apply RECOMMENDED settings
global:
objecter_inflight_ops: 24576 # default 1024
rgw_thread_pool_size: 512 # default 100
2) run 'ansible-playbook site.yml'
3) look at ceph.conf confirming that new values are in there
[global]
objecter_inflight_ops = 24576
rgw_thread_pool_size = 512
3) probe for values
[OSDnode1]# ceph daemon osd.105 config get objecter_inflight_ops
{
"objecter_inflight_ops": "1024" <-- NOT CHANGED
}
[OSDnode1]# ceph daemon client.rgw.c04-h01-6048r config get rgw_thread_pool_size
{
"rgw_thread_pool_size": "512" <-- CHANGED
}
4) restart daemons
# ansible all -a “systemctl restart ceph.target”
5) probe again for objecter_inflight_ops value
# ceph daemon osd.105 config get objecter_inflight_ops{
"objecter_inflight_ops": "24576" <-- CHANGED
}
# ceph daemon client.rgw.c04-h01-6048r config get rgw_thread_pool_size
{
"rgw_thread_pool_size": "512" <-- persists as expected
}
Any ideas on one value is applied while the other is not?
here is a transcript provided Guillaume Abrioux which indicates that this works correctly https://docs.google.com/document/d/1Nas4b-ZgmI4pGFtYwaMvCpWY_zrlFesL_3PPol7ERcc/edit?usp=sharing My experience differs. Vasu, what do you think about Johns comment in #c4? I plan to close this as WORKSFORME tomorrow lacking further detail from you. The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days |
Description of problem: This issue was discovered by John during DFG workload, after applying config changes to ceph_conf_overrides and rerunning ceph-ansible, the services were not restarted and the setting didn't apply properly From John's email thread. 1) verify current settings [root@c04-h01-6048r ~]# ceph daemon osd.100 config get filestore_merge_threshold (10) [root@c04-h01-6048r ~]# ceph daemon osd.100 config get objecter_inflight_ops (1024) [root@c04-h01-6048r ~]# ceph daemon ceph-rgw.c04-h01-6048r config rgw_thread_pool_size ← FAILS 2) Edit all.yml with recommended settings # ln -s all.ymlRECOMMENDED all.yml # vi group_vars/all.yml # JTH apply RECOMMENDED settings global: objecter_inflight_ops: 24576 # default 1024 rgw_thread_pool_size: 512 # default 50 (unable to verify) osd: filestore_merge_threshold: -1 # default 10 3) run ceph-ansible # ansible-playbook site.yml 2>&1 | tee recommendedJun17.txt 4) view settings # cat /etc/ceph/ceph.conf objecter_inflight_ops = 24576 rgw_thread_pool_size = 512 filestore_merge_threshold = -1 5) probe with 'ceph daemon' (run on OSD node) # ceph daemon osd.100 config get filestore_merge_threshold <-- still 10 # ceph daemon osd.100 config get objecter_inflight_ops <-- still 1024