Description of problem: It can happen that the JBoss instance is imported in the JON (using CLI) but it does not have Alert definitions specified when checking JON UI -> JBoss instance -> Alerts -> Definitions. Version-Release number of selected component (if applicable): JON 3.1.1 How reproducible: Always Steps to Reproduce: 1. Start JON Server; 2. From JON UI navigate to Administration -> Alert Definition Templates -> Servers -> press Edit for JBossAS5 plugin -> add Alert templates; 3. Start JON Agent; 4. Start JBoss instance; 5. Let JON Agent discover JBoss instance (so it's visible in the Discovery Queue); 6. Shut down JON Agent; 7. Execute addJBoss.js script [1] from the JON CLI to inventory all JBoss instances that are in the discovery queue; 8. Confirm that JON UI has got in its inventory JBoss instance with availability UNKNOWN (as it's Agent is DOWN); 9. From JON UI navigate to JBoss instance that should have alerts definitions defioned -> Alerts -> Definitions and confirm that they are not there. 10. Start JON Agent with --cleanconfig option; 11. Once the Agent is up and running, check the availability of the JBoss instance and confirm that it is GREEN. 12. From JON UI navigate to JBoss instance -> Alerts -> Definitions and confirm that this page is still empty. Actual results: Alert definitions page for imported JBoss instance is empty. Expected results: Alert Definitions page for imported JBoss instance contains alert definitions defined by the Alert definitions Template for JBossAS5. Additional info: If JON agent is re-started without --cleanconfig option, JBoss instance that was discovered and imported would become GREEN, but also it would have alert definitions added. [1] https://access.redhat.com/knowledge/solutions/68975
Does this really require the use of the API? Couldn't we have simply reproduced this by using the UI to import? Instead of running addJBoss.js in step 7, simply import the AS server from the discovery queue? Additionally, is the --cleanconfig really needed here?
Answering my own questions: (In reply to comment #1) > Does this really require the use of the API? Couldn't we have simply > reproduced this by using the UI to import? Instead of running addJBoss.js in > step 7, simply import the AS server from the discovery queue? The API is not required. This can be reproduced from the UI. > Additionally, is the --cleanconfig really needed here? No and yes. --cleanconfig is not needed but what is needed is a purge of the agent's persisted inventory. --cleanconfig does this. However, you could easily do this with --purgedata or simply by deleting the agent's inventory.dat file. Here are my updated steps to reproduce this issue: 1. Start EAP server 2. Start JBoss ON system (server and agent) 3. From ON UI, add new alert definition template for JBossAS Server from JBossAS5 plug-in From Administration > Configuration / Alert Definition Templates >> Servers / JBossAS Server [JBossAS5] select the edit icon (pencil) and create a new alert definition of any condition. 4. Verity EAP server appears in discovery queue. DO NOT import it yet 5. Shut down the ON agent 6. Import EAP server from discovery queue 7. Confirm newly imported EAP server resource appears under Inventory > Resources / Servers - Top Level Imports with an *Availability* of `UNKNOWN` 8. Confirm no alert definitions appear under the EAP server resources Alerts >> Definitions page 9. Purge the agent's persisted inventory and start the agent This can be done by simply deleting the <RHQ_AGENT_HOME>/data/inventory.dat before starting the agent normally or starting the agent with the --purgedata argument. 10. Wait a few seconds for the agent to finish starting and refreshing its inventory and availability 11. Confirm the availability status of the EAP server resource is now `UP` instead of `UNKNOWN` 12. View the EAP server resource's Alerts >> Definitions page for the alert definition created as part of the alert definition template I am not sure that the confirmation in step 8. matters from an "expected" behavior standpoint. In other words, perhaps it would be completely valid for this alert definition to already appear at this point. And if it does, it would be expected that it would remain for the following steps. The important step is 9. If the agent is allowed to start back up with its persisted inventory, the alert definition gets applied to the newly imported EAP server resource. However, if the agent must gets it inventory from the server, the alert definition does not get applied. In the end, I am setting the severity of this to medium as its occurrence would require something that would not be normal within a production system (or even development for that matter).
Clearing the NEEDINFO flag.
This bug appears to be a result of bug 887411. Basically, due to the agent dropping its inventory and re-syncing with the server, the alert definitions from the resource type's template do not get applied. This bug has also been captured as bug 884598 in the upstream RHQ project.
My mistake. This is not a direct result of bug 887411. The workaround for this issue was not to restart the agent or force an inventory sync at the time a resource is imported. That workaround revealed bug 887411 but this issue is directly dependent on 887411.
Please see my comment in the upstream, Bug 884598.
Steps from comment 2 are what we should pay attention to: 1. Start EAP server 2. Start JBoss ON system (server and agent) 3. From ON UI, add new alert definition template for JBossAS Server from JBossAS5 plug-in From Administration > Configuration / Alert Definition Templates >> Servers / JBossAS Server [JBossAS5] select the edit icon (pencil) and create a new alert definition of any condition. 4. Verity EAP server appears in discovery queue. DO NOT import it yet 5. Shut down the ON agent 6. Import EAP server from discovery queue 7. Confirm newly imported EAP server resource appears under Inventory > Resources / Servers - Top Level Imports with an *Availability* of `UNKNOWN` 8. Confirm no alert definitions appear under the EAP server resources Alerts >> Definitions page 9. Purge the agent's persisted inventory and start the agent This can be done by simply deleting the <RHQ_AGENT_HOME>/data/inventory.dat before starting the agent normally or starting the agent with the --purgedata argument. 10. Wait a few seconds for the agent to finish starting and refreshing its inventory and availability 11. Confirm the availability status of the EAP server resource is now `UP` instead of `UNKNOWN` 12. View the EAP server resource's Alerts >> Definitions page for the alert definition created as part of the alert definition template In these steps the resource isn't being removed or modified. Specifically, if a resource is in inventory as NEW and the agent is shutdown and the resource is imported, the alert templates won't get applied if the agent was started with --cleanconfig. Although the user did not intend to purge inventory, they did intend to make a configuration change. This is resulting in this bug. This is a pretty big issue if the discovery queue has many resources in a new state that get imported while the agent is busy, down, or unable to reach the server. This will result in the resource templates not getting applied.
The upstream has been closed as NotABug which I think is very reasonable. I suggest the same be done here.
I am not sure why the upstream closed this as NOTABUG considering that it is very clear that this is a bug as described in the steps to reproduce the issue -- which are very common -- and the comments on the upstream. I am re-targeting this for consideration in a future minor/major release. At this time, the only viable workaround to this is not to purge the agent's inventory or data. In the event that this happens, the user will need to make an arbitrary and non-necessary modification to the alert template and re-apply it to all resources to get the template to get properly applied to the target resource types.
JBoss ON is coming to the end of its product life cycle. For more information regarding this transition, see https://access.redhat.com/articles/3827121. This bug report/request is being closed. If you feel this issue should not be closed or requires further review, please create a new bug report against the latest supported JBoss ON 3.3 version.