Bug 1083414

Summary: ceilometer qpid reconnection delay must be more accurate
Product: Red Hat OpenStack Reporter: Flavio Percoco <fpercoco>
Component: openstack-ceilometerAssignee: Flavio Percoco <fpercoco>
Status: CLOSED ERRATA QA Contact: Shai Revivo <srevivo>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.0CC: ajeain, apevec, breeler, eglynn, fdinitto, fpercoco, jruzicka, pbrady, sclewis, scohen, sradvan, yeylon
Target Milestone: z4Keywords: OtherQA, Rebase, ZStream
Target Release: 4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-ceilometer-2013.2.3-1.el6ost Doc Type: Bug Fix
Doc Text:
Cause: reconnection to the Qpid message broker was delayed by a minimum of 60 seconds. Consequence: broker reconnect operations from ceilometer agents were not triggered in a timely fashion, so the flow of samples would not resume soon enough once the message broker was restored. Fix: the minimum reconnection delay has been reduced from 60 seconds to 5 seconds. Result: after the Qpid broker is restarted, the flow of samples is quickly resumed.
 Story Points: ---
Clone Of: 1060689 Environment:
Last Closed: 2014-05-29 19:56:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 2 Alan Pevec 2014-04-04 11:40:08 UTC 
Included in 2013.2.3
Comment 5 Eoghan Glynn 2014-05-29 10:45:23 UTC 
OtherQA Process
===============

0. Install packstack --allinone in the usual way

1. Artificially accelerate the cadence of polling for an individual meter by adding the following entry to /etc/ceilometer/pipeline.yaml:

     name: image_pipeline
     interval: 10
     meters:
         - "image.size"
     transformers:
     publishers:
         - rpc://

2. Restart the agent responsible for collecting that accelerated meter:

   $ sudo service openstack-ceilometer-central restart

3. Note the 10 second cadence of samples for that meter:

   $ ceilometer sample-list -m image.size | awk '/image.size/ {print $(NF-1)}'

4. Stop the qpidd broker:

   $ sudo service qpidd stop

5. Note that the flow of samples dries up:

   $ while true; do ceilometer sample-list -m image.size | awk '/image.size/ {print $(NF-1)}' | head -1 ; echo '---' ; sleep 1 ; done

6. Restart the qpidd broker:

   $ sudo service qpidd stop

7. The flow of sample should resume within 5s:

   $ while true; do ceilometer sample-list -m image.size | awk '/image.size/ {print $(NF-1)}' | head -1 ; echo '---' ; sleep 1 ; done

Marking as verified due to the expected rapid resumption of message flow being observed.
Comment 6 Eoghan Glynn 2014-05-29 13:12:57 UTC 
Correction to OtherQA steps above, step #6 should read:

6. Restart the qpidd broker:

   $ sudo service qpidd start
Comment 8 errata-xmlrpc 2014-05-29 19:56:24 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0577.html