Hide Forgot
Description of problem: cinder volume service is down . service restart brings it up for few mins . No events noticed on cinder logs . Version-Release number of selected component (if applicable): RHOS6 python-oslo-messaging-1.4.1-3.el7ost.noarch rabbitmq-server is provided by 3rd party (Pivotal) {rabbit,"RabbitMQ","3.3.1"} How reproducible: Only on customer end . Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: rabbitmq-server logs : =ERROR REPORT==== 29-Nov-2016::04:19:02 === connection <0.14981.69>, channel 1 - soft error: {amqp_error,precondition_failed, "parameters for queue 'engine' in vhost '/' not equivalent", 'queue.declare'}
(In reply to John Eckersberg from comment #7) > I've seen countless times in the past where the ceph code blocks inside of > the eventloop, which prevents the periodic job running causing the service > to be marked as down. Usually if you strace the process, you will see it > spinning on futex() with FUTEX_WAIT. Thanks a lot John . It seems this could be the issue . The issue was resolved after all ceph concerns were resolved . As soon as all PGs were active+clean cinder service came back and all Openstack Services were back online . Closing as not a bug . I think it would be a good idea to create a doc with information on what all needs to be checked for each service , when it is seen as down . I will start working on it . Thanks Victor , Petr for the helping out on this . Regards, Jaison R