Bug 1370242
| Summary: | Keystone overwhelms Ceilometer with Identity Events | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | david.costakos |
| Component: | openstack-ceilometer | Assignee: | Pradeep Kilambi <pkilambi> |
| Status: | CLOSED NOTABUG | QA Contact: | Yurii Prokulevych <yprokule> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 8.0 (Liberty) | CC: | david.costakos, fbaudin, gblomqui, jdennis, jmarc, jruzicka, kbasil, kmorey, nkinder, nstephan, pkilambi, rduartes, srevivo |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | 11.0 (Ocata) | Flags: | david.costakos:
needinfo-
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-01-18 20:24:07 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This "works as designed" as these events are necssary for any auditing. They are not SPAM. Each of these has security implications. The error seems to be on the CloudForms side in trying to process events it should be ignoring. This is not configurable in Keystone. Do you really think that Keystone needs to change here? Note that you can reduce the number events produced by Keystone using a configuration option. According to the Keystone documentation, http://git.openstack.org/cgit/openstack/keystone/tree/etc/keystone.conf.sample?h=9.0.2#n105 [DEFAULT] notification_opt_out=identity.user.create notification_opt_out=identity.authenticate.success This would prevent keystone from sending notifications out on successful authentications or on user creates. According to http://docs.openstack.org/developer/ceilometer/events.html#events-from-notifications "The general philosophy of notifications in OpenStack is to emit any and all data someone might need, and let the consumer filter out what they are not interested in." According to http://docs.openstack.org/developer/ceilometer/events.html#converting-notifications-to-events "the conversion from Notifications to Events is driven by a configuration file (specified by the flag definitions_cfg_file in ceilometer.conf)." In the overcloud, you can find the specified file at: /etc/ceilometer/event_definitions.yaml This has a stanza that starts like this: - event_type: identity.authenticate Removing that stanza should prevent Ceilometer from recording the event. This is likely fixed by https://bugzilla.redhat.com/show_bug.cgi?id=1389848 |
Description of problem: When configuring OpenStack from OSP director, keystone is enabled to produce ceilometer events. These events spam Ceilometer, and any CloudForms instance managing the Overcloud with "identity.authenticate" events. These events cause unneeded processing on CloudForms and unneeded data storage in ceilometer as they have no practical use Version-Release number of selected component (if applicable): openstack-keystone-8.0.1-1.el7ost.noarch python-tripleoclient-0.3.4-6.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy Overcloud with ceilometer Events parameter_defaults: CeilometerStoreEvents: true 2. login to controller 3. sudo openstack-config --get /etc/keystone/keystone.conf DEFAULT notification driver messagin Actual results: literally nearly 100,000 identity events get created per day. Here is a sample of about 22 hours from an unused Cloud. grep /ManageIQ/System/Event/EmsEvent/OPENSTACK evm.log | awk '{ print $10 }' | sort | uniq -c ... 86317 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.authenticate] 473 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.created.role_assignment] 2 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.domain.created] 54 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.endpoint.created] 23 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.OS-TRUST:trust.created] 21 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.OS-TRUST:trust.deleted] 29 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.project.created] 21 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.project.deleted] 2 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.region.created] 473 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.role_assignment.created] 8 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.role.created] 18 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.service.created] 467 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.user.created] 404 [/ManageIQ/System/Event/EmsEvent/OPENSTACK/identity.user.deleted] ... NOTE: the 86,317 identity.authenticate events produced by the Overcloud Expected results: identity events need not be produced by default for CloudForms to do its thing. these are essentially SPAM events that use valuable resources Additional info: Suggest setting notification_driver to either log or noop in /etc/keystone/keystone.conf