Bug 1563389

Summary: notifications.info keeps growing even after passing 'driver=noop'
Product: Red Hat OpenStack Reporter: Irina Petrova <ipetrova>
Component: puppet-osloAssignee: RHOS Maint <rhos-maint>
Status: CLOSED DUPLICATE QA Contact: nlevinki <nlevinki>
Severity: high Docs Contact:
Priority: high    
Version: 10.0 (Newton)CC: apevec, chjones, dhill, ipetrova, jeckersb, jjoyce, jschluet, lhh, michele, slinaber, srevivo, tvignaud, vstinner
Target Milestone: ---Keywords: Reopened, Triaged, ZStream
Target Release: ---   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-23 13:23:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Irina Petrova 2018-04-03 20:07:20 UTC
Description of problem:

In RHOS-10 w/ disabled Telemetry we're pushing the noop notification driver to all controllers, for all services with oslo_messaging_notifications config SECTION:

[root@controller-01 ~]# for i in $(grep -ri oslo_messaging_notifications /etc/* 2>/dev/null |awk -F: '{print $1}'); do printf "%-40s" $i; crudini --get $i oslo_messaging_notifications driver; done &> ~/oslo_messaging_notifications_$HOSTNAME.txt

cat os  [root@controller-01 ~]#
[root@controller-01 ~]# cat oslo_messaging_notifications_controller-01.openstack.local.txt  | grep "conf "
/etc/aodh/aodh.conf                     noop
/etc/ceilometer/ceilometer.conf         noop
/etc/cinder/cinder.conf                 noop
/etc/cinder/cinder-cesano.conf          noop
/etc/cinder/cinder-rozzano.conf         noop
/etc/glance/glance-registry.conf        noop
/etc/glance/glance-api.conf             noop
/etc/heat/heat.conf                     noop
/etc/keystone/keystone.conf             noop
/etc/manila/manila.conf                 noop
/etc/neutron/neutron.conf               noop
/etc/nova/nova.conf                     noop
/etc/sahara/sahara.conf                 noop


[root@controller-02 ~]#  for i in $(grep -ri oslo_messaging_notifications /etc/* 2>/dev/null |awk -F: '{print $1}'); do printf "%-40s" $i; crudini --get $i oslo_messaging_notifications driver; done &> ~/oslo_messaging_notifications_$HOSTNAME.txt

[root@controller-02 ~]# cat oslo_messaging_notifications_controller-02.openstack.local.txt  | grep "conf "
/etc/aodh/aodh.conf                     noop
/etc/ceilometer/ceilometer.conf         noop
/etc/cinder/cinder.conf                 noop
/etc/cinder/cinder-cesano.conf          noop
/etc/cinder/cinder-rozzano.conf         noop
/etc/glance/glance-registry.conf        noop
/etc/glance/glance-api.conf             noop
/etc/heat/heat.conf                     noop
/etc/keystone/keystone.conf             noop
/etc/manila/manila.conf                 noop
/etc/neutron/neutron.conf               noop
/etc/nova/nova.conf                     noop
/etc/sahara/sahara.conf                 noop


[root@controller-03 ~]# for i in $(grep -ri oslo_messaging_notifications /etc/* 2>/dev/null |awk -F: '{print $1}'); do printf "%-40s" $i; crudini --get $i oslo_messaging_notifications driver; done &> ~/oslo_messaging_notifications_$HOSTNAME.txt

[root@controller-03 ~]# cat oslo_messaging_notifications_controller-03.openstack.local.txt | grep "conf "
/etc/aodh/aodh.conf                     noop
/etc/ceilometer/ceilometer.conf         noop
/etc/cinder/cinder.conf                 noop
/etc/cinder/cinder-cesano.conf          noop
/etc/cinder/cinder-rozzano.conf         noop
/etc/glance/glance-registry.conf        noop
/etc/glance/glance-api.conf             noop
/etc/heat/heat.conf                     noop
/etc/keystone/keystone.conf             noop
/etc/manila/manila.conf                 noop
/etc/neutron/neutron.conf               noop
/etc/nova/nova.conf                     noop
/etc/sahara/sahara.conf                 noop


<< ALL Controller nodes get rebooted. >>


[root@controller-03 ~]# rabbitmqctl purge_queue notifications.info
Purging queue 'notifications.info' in vhost '/' ...
[root@controller-03 ~]# rabbitmqctl list_queues | grep notifications
notifications.error     0
notifications.warn      0
notifications.debug     0
notifications.critical  0
notifications.sample    0
notifications.audit     0
versioned_notifications.error   0
versioned_notifications.info    0
notifications.info      0


But after 10 minutes:

[root@controller-03 ~]# rabbitmqctl list_queues | grep notifications
notifications.error     0
notifications.warn      0
notifications.debug     0
notifications.critical  0
notifications.sample    0
notifications.audit     0
versioned_notifications.error   0
versioned_notifications.info    0
notifications.info      68


>> notifications.info still grows...

Comment 1 Peter Lemenkov 2018-04-04 15:25:44 UTC
Not sure what we can do from the RabbitMQ side. Someone should stop either creating these notifications.info queues or better stop pushing data into it. Perhaps it should be reassigned to oslo-messaging? 

Also I can see sign of a bug 1436784, but it mostly a cosmetic and does not affect cluster in this particular case.

Comment 2 Irina Petrova 2018-04-04 16:16:11 UTC
Switching to oslo as per Peter's recommendation.

Comment 3 Victor Stinner 2018-04-16 13:43:36 UTC
"In RHOS-10 w/ disabled Telemetry we're pushing the noop notification driver to all controllers, for all services with oslo_messaging_notifications config SECTION: (...)"

Did you follow the following documentation to disable Telemetry?
https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/disable_telemetry.html

It suggests to use this command:

"openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml"

If you disabled notifications manually, are you sure that you did not forget a service? Maybe there is one remaining service which still sends notifications?

Comment 5 Chris Jones 2018-06-20 18:22:21 UTC
We are closing this issue because we do not have all of the information required to make further progress.

Please re-open if you are able to provide the required information.

Comment 6 David Hill 2018-07-15 19:38:45 UTC
The problem is present in RHOSP 10 and later:

args='/usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/share/neutron/server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-server --log-file /var/log/neutron/server.log'.split(' ')
from oslo_config import cfg
from neutron.common import config
from neutron.common import rpc as n_rpc
config.init(args[2:])

for k, v in cfg.CONF.oslo_messaging_notifications.iteritems(): print k, v
transport_url None
topics ['notifications']
driver ['neutron.openstack.common.notifier.rpc_notifier', 'noop']    

The problem is originating in /usr/share/neutron/neutron-dist.conf and commenting the notification_driver solves the problem:

[DEFAULT]
verbose = True
lock_path = $state_path/lock
#notification_driver = neutron.openstack.common.notifier.rpc_notifier
allow_overlapping_ips = True
use_stderr = False
api_paste_config = /usr/share/neutron/api-paste.ini

[agent]
root_helper = sudo neutron-rootwrap /etc/neutron/rootwrap.conf
root_helper_daemon = sudo neutron-rootwrap-daemon /etc/neutron/rootwrap.conf

Comment 7 David Hill 2018-07-15 19:39:49 UTC
When commenting notification_driver in /usr/share/neutron/neutron-dist.conf, the notification driver is properly set to noop:

transport_url None
topics ['notifications']
driver ['noop']

Comment 8 David Hill 2018-07-15 20:28:38 UTC
And with heat.conf, the problem is that even by setting heat::notification_driver  in the environment file (network-environment.yaml ) , it never lands in /etc/heat/heat.conf.   I guess we have 2 bugs here .  Should I open another one or clone this one ?

Comment 15 Victor Stinner 2018-07-23 13:13:44 UTC
Oh wow, I didn't know that the option accepted multiple values :-) I guess that there is an use case to have multiple notifier drivers.

But here, we want to not send notifications, so we must have exactly one driver which is: "noop". I'm not sure which option is the best to get exactly one driver.

https://github.com/openstack/tripleo-heat-templates/blob/master/environments/disable-telemetry.yaml

In the YAML, I see a single value:

parameter_defaults:
  NotificationDriver: 'noop'

Comment 16 Michele Baldessari 2018-07-23 13:23:34 UTC
Ack, so I think we're all in agreement that we should just remove it from neutron-dist.conf. Thanks, everyone

*** This bug has been marked as a duplicate of bug 1601593 ***