Bug 1232797
| Summary: | Ceilometer redis url not showing up on all controllers in HA deployment | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Marius Cornea <mcornea> | ||||
| Component: | openstack-tripleo-heat-templates | Assignee: | Giulio Fidente <gfidente> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Marius Cornea <mcornea> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 7.0 (Kilo) | CC: | dmacpher, gfidente, jguiditt, kbasil, mandreou, mburns, rhel-osp-director-maint | ||||
| Target Milestone: | ga | ||||||
| Target Release: | Director | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | openstack-tripleo-heat-templates-0.8.6-11.el7ost | Doc Type: | Bug Fix | ||||
| Doc Text: |
Ceilometer uses an incorrect redis VIP for its backend_url. This fix sets the Ceilometer backend_url to one that Heat provides and not constructed during deployment. Ceilometer now uses the correct IP address.
|
Story Points: | --- | ||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-08-05 13:54:18 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Marius Cornea
2015-06-17 13:48:26 UTC
This might need updates to make it more similar to what we have in quickstack: https://github.com/redhat-openstack/astapor/blob/master/puppet/modules/quickstack/manifests/pacemaker/ceilometer.pp#L35 https://github.com/redhat-openstack/astapor/blob/master/puppet/modules/quickstack/manifests/pacemaker/redis.pp#L77 https://github.com/redhat-openstack/astapor/blob/master/puppet/modules/quickstack/manifests/load_balancer/redis.pp neutron port-list | grep redis_virtual_ip
| b6240296-0968-4c3d-a6be-2fb2e2371c90 | redis_virtual_ip | fa:16:3e:10:2e:26 | {"subnet_id": "34f46be5-855f-42d7-96cb-0533d916def0", "ip_address": "192.0.2.13"} |
The Redis VIP seems to be set in the ceilometer conf file only on one of the controllers. Nevertheles the node where the Redis VIP is set is running Redis on the local IP address not on the VIP:
[stack@instack ~]$ ssh heat-admin.2.13 'sudo lsof -i :6379 -n'
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
redis-ser 23585 redis 4u IPv4 194002 0t0 TCP 192.0.2.14:6379 (LISTEN)
redis-ser 23585 redis 6u IPv4 197905 0t0 TCP 192.0.2.14:38999->192.0.2.15:6379 (ESTABLISHED)
hi Marisu, redis is meant to bind on the local ip address, the clients have to reach it via the VIP can you please attach log from ceilometer-central ? Created attachment 1040023 [details]
central.log
Attached, it's showing connection refused for the VIP address.
Balancing for Redis is false by default [1] we miss to enable it! 1. https://github.com/openstack/puppet-tripleo/blob/master/manifests/loadbalancer.pp#L217 On the non-0 controllers backend_url appear to be misconfigured as well: backend_url=redis://:6379 waiting for envs to come up but setting [1] is trivial may just start a review for that while waiting and then dig at the other issue giulio mentions in #8 above. [1] https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/manifests/overcloud_controller_pacemaker.pp#L58 https://review.openstack.org/#/c/193048/ Enable loadbalancing of the Redis VIP, defaults to False upstream review out as a placeholder fow now to see if it fixes the problem and what else might be needed. so confirmed by gfidente and also my latest run with downstream tht + the review above shows correct backend_url on controller-1 for example: [root@overcloud-controller-1 heat-admin]# sudo grep backend_url=redis /etc/ceilometer/ceilometer.conf backend_url=redis://192.0.2.14:6379 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2015:1549 |