Bug 1146433 - E-Mail Spamming from Node during HA state changes
Summary: E-Mail Spamming from Node during HA state changes
Status: CLOSED CURRENTRELEASE
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-hosted-engine-ha
Version: 3.5
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: 3.5.0
Assignee: Martin Sivák
QA Contact: Artyom
URL:
Whiteboard: sla
Keywords: Triaged
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-09-25 09:09 UTC by Zordrak
Modified: 2016-02-10 19:41 UTC (History)
10 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2014-10-17 12:38:58 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 33394 master MERGED Do not emit email events when the state was not changed Never
oVirt gerrit 33518 ovirt-hosted-engine-ha-1.2 MERGED Do not emit email events when the state was not changed Never

Description Zordrak 2014-09-25 09:09:49 UTC
Description of problem:

When nodes are placed into maintenance, instead of one e-mail per state change, it seems to generate e-mails regularly during any state other than up and running.

Version-Release number of selected component (if applicable):
3.5-rc3

How reproducible:
Always

Steps to Reproduce:
1. Run HostedEngine HA across multiple nodes
2. Place a node into Maintenance
3. Perform maintenance
4. Activate node

Actual results:

ovirt-hosted-engine\ state\ transition\ EngineDown-EngineStart\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1200.eml
ovirt-hosted-engine\ state\ transition\ EngineDown-EngineStart\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253.eml
ovirt-hosted-engine\ state\ transition\ EngineDown-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1141.eml
ovirt-hosted-engine\ state\ transition\ EngineStart-EngineStarting\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1200.eml
ovirt-hosted-engine\ state\ transition\ EngineStart-EngineStarting\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253.eml
ovirt-hosted-engine\ state\ transition\ EngineStarting-EngineStarting\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1200.eml
ovirt-hosted-engine\ state\ transition\ EngineStarting-EngineStarting\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253.eml
ovirt-hosted-engine\ state\ transition\ EngineStarting-EngineStarting\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ EngineStarting-EngineStarting\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254-2-3.eml
ovirt-hosted-engine\ state\ transition\ EngineStarting-EngineStarting\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254-2.eml
ovirt-hosted-engine\ state\ transition\ EngineStarting-EngineStarting\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254.eml
ovirt-hosted-engine\ state\ transition\ EngineStarting-EngineUp\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1200.eml
ovirt-hosted-engine\ state\ transition\ EngineStarting-EngineUp\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254.eml
ovirt-hosted-engine\ state\ transition\ EngineUp-LocalMaintenanceMigrateVm\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1251.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1142-2-3-4-5-6.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1142-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1142-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1142-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1142-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1142.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1143-2-3-4-5-6.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1143-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1143-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1143-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1143-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1143.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1144-2-3-4-5-6.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1144-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1144-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1144-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1144-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1144.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1145-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1145-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1145-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1145.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1251.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1252-2-3-4-5-6.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1252-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1252-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1252-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1252-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1252.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253-2-3-4-5-6.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1253.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254-2-3-4-5-6.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1254.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1255-2-3-4-5-6.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1255-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1255-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1255-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1255-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1255.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1256-2-3-4-5-6.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1256-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1256-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1256-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1256-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1256.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1257-2-3-4-5.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1257-2-3-4.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1257-2-3.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1257-2.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenance-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1257.eml
ovirt-hosted-engine\ state\ transition\ LocalMaintenanceMigrateVm-EngineMigratingAway\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1251.eml
ovirt-hosted-engine\ state\ transition\ ReinitializeFSM-LocalMaintenance\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1251.eml
ovirt-hosted-engine\ state\ transition\ StartState-ReinitializeFSM\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1136.eml
ovirt-hosted-engine\ state\ transition\ StartState-ReinitializeFSM\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1140.eml
ovirt-hosted-engine\ state\ transition\ StartState-ReinitializeFSM\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1204.eml
ovirt-hosted-engine\ state\ transition\ StartState-ReinitializeFSM\ -\ root\@ovirt.example.org\ -\ 2014-09-24\ 1317.eml


Expected results:

<10 e-mails

Comment 1 Martin Sivák 2014-09-26 09:14:08 UTC
Hi,

please see the /etc/ovirt-hosted-engine-ha/broker.conf file, there is a notification section:

[notify]
state_transition = maintenance|start|stop|migrate|on

The logic takes the state transition info (eg. StartState-ReinitializeFSM) and tries to match the state_transition regular expression on it.

If it matches, email is sent.

We might change the default rule a bit, but you can fix this by changing the configuration.

Comment 2 Martin Sivák 2014-09-26 09:17:46 UTC
Looking at the list of emails more closely.. we might reconsider if emitting no change transitions is a good idea.

Comment 3 Artyom 2014-10-06 11:04:07 UTC
Checked on ovirt-hosted-engine-ha-1.2.1-1.el6ev.noarch
Hosted-engine still notify not only when state changed, for example I have notification about local maintenance each time I have line about local maintenance in log. Also I not receive notification when host change status to up.
1) Put host to local maintenance(hosted-engine --set-maintenance --mode=local)
2) Deactivate local maintenance(hosted-engine --set-maintenance --mode=none)
Mails:
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:52 PM
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:53 PM
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:53 PM
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:53 PM
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:53 PM
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:54 PM
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:54 PM
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:54 PM
ovirt-hosted-engine state transition EngineDown-LocalMaintenance October 6, 2014 1:54 PM
ovirt-hosted-engine state transition LocalMaintenance-ReinitializeFSM October 6, 2014 1:54 PM

Comment 4 Martin Sivák 2014-10-06 11:23:31 UTC
Please test with proper version. Fixed in ovirt-hosted-engine-ha-1.2.1-1.el6ev.noarch

Comment 5 Martin Sivák 2014-10-06 11:25:48 UTC
I ment ovirt-hosted-engine-ha-1.2.2-1.el6ev.noarch

Comment 6 Artyom 2014-10-06 12:07:59 UTC
ok vt4 still have just ovirt-hosted-engine-ha-1.2.1-1.el6ev.noarch version, when I will have newer version will check

Comment 7 Artyom 2014-10-07 10:45:05 UTC
Verified on ovirt-hosted-engine-ha-1.2.2-2.el6ev.noarch

Comment 8 Sandro Bonazzola 2014-10-17 12:38:58 UTC
oVirt 3.5 has been released and should include the fix for this issue.


Note You need to log in before you can comment on or make changes to this bug.