Bug 1450201

Summary: EAP7 Events Not Displayed On Provider Timeline
Product: Red Hat CloudForms Management Engine Reporter: Matt Mahoney <mmahoney>
Component: ProvidersAssignee: Jay Shaughnessy <jshaughn>
Status: CLOSED CURRENTRELEASE QA Contact: Matt Mahoney <mmahoney>
Severity: medium Docs Contact:
Priority: high    
Version: unspecifiedCC: abonas, cpelland, izapolsk, jdoyle, jfrey, jhardy, jshaughn, mmahoney, obarenbo, simaishi
Target Milestone: GAKeywords: Triaged
Target Release: cfme-future   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: :middleware
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-01-10 15:03:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: Middleware Target Upstream Version:
Embargoed:
Attachments:
Description Flags
evm.log
none
hawkular.log
none
eap7.log
none
Servers List
none
Timelines View none

Description Matt Mahoney 2017-05-11 19:36:43 UTC 
Created attachment 1278003 [details]
evm.log

Description of problem:
After Inventorying multiple EAP7 Standalone Servers via CFME, the Middleware Provider Timeline does not show any EAP7 Events.

JMAN4: https://issues.jboss.org/browse/JMAN4-163

This is a regression issue since ER1, when this feature was last tested.

Version-Release number of selected component (if applicable):
CR1
CFME: 5.8.0.14-rc3.20170509175238_5dbf87a

How reproducible:


Steps to Reproduce:
1. Start Middleware Provider, and then add as CFME provider.
2. Refresh MW Provider Inventory
3. Check Provider Timelines for EAP Events
4. Start one or more EAP7 Standalone servers
Validate that EAP log contains "HAWKMONITOR010034: Opened feed WebSocket connection to endpoint..."
5. CFME MW Provider Refresh
   Validate that Middleware Provider->Servers list contains the started EAP7 Servers
6. Nav to MW Provider Timeslines:
    - adjust filters to include Applications 
    - Check "Show Detailed Events"
    - Adjust End date to be one day beyond todays date, click Apply
  There should be an event for each EAP inventoried

Actual results:
Timelines should display NO EAP events

Expected results:
Timelines should display an event for each EAP

Additional info:
Comment 2 Matt Mahoney 2017-05-11 19:37:14 UTC 
Created attachment 1278004 [details]
hawkular.log
Comment 3 Matt Mahoney 2017-05-11 19:38:10 UTC 
Created attachment 1278005 [details]
eap7.log
Comment 4 Matt Mahoney 2017-05-11 19:39:44 UTC 
Created attachment 1278006 [details]
Servers List
Comment 5 Matt Mahoney 2017-05-11 19:40:14 UTC 
Created attachment 1278007 [details]
Timelines View
Comment 6 Alissa 2017-05-14 12:52:51 UTC 
Jay, could this be because of the move to the new inventory impl?
Comment 7 Jay Shaughnessy 2017-05-15 20:34:55 UTC 
Matt, are you saying we use to have timeline events for started servers?  I thought we had them only for new/added servers.  I'm not sure if we have a feature gap here. Or are these new servers?  Depending on the version of the hawkular services backend maybe there is an issue.  The feature was no reinstated until 0.37 of services (only ADD server, REMOVE server no longer generates an event.
Comment 11 Jay Shaughnessy 2017-05-24 19:25:40 UTC 
Without knowing the services backend version it's hard to know if this was tested during the time things were broken due to the inventory migration.  Please confirm the services version or please test again using services 0.37 or higher.
Comment 13 Jay Shaughnessy 2017-05-30 20:28:07 UTC 
It's usually normal to not see a server-added event for the Hawkular server itself.  This is because the Hawkular server is started before it is added as a provider in MIQ.  Assuming the agent is deployed and started with the Hawkular server, the agent has already discovered and added the server before we add it as a MIQ provider.  The "server-added" event therefore predates the provider's event catcher.

I confirmed locally that the event was in fact generated, and by playing with some dates I was able to see it on the timeline, so the server-side event was generated and the catcher does seem to be catching events generated after the provider is added and the catcher is running.

What I can't explain yet is why there are no events seen for the other servers added in the reproduction steps. If I read the steps correctly, after the provider is created, more EAPs were added, I presume these were added using the EAP+agent docker image, and configured to report to the hawkular server?  All I can think of is that the EAP images have agents that pre-date the feature re-implementation, because now those events are initiated by the agent.  If we could crack open the image and see what the agent config looked like we could tell if it was an older agent.

The other possibility is to inspect hAlerts via REST to see what events have actually been generated for consumption by MIQ.

Is is possible to determine the EAP image version?
Comment 15 Jay Shaughnessy 2017-06-05 20:44:24 UTC 
After investigating a repeated failure using the latest EAP+javaagent image, it seems that the EAP image may have an issue, possibly using an outdated javaagent config (yaml) that lacks the new entries necessary for having the agent notify the hwk server of added servers.  I'm pursuing with release engineering to see if they can confirm a build issue but there is definitely something fishy here...

My feeling is that this BZ will be resolved given a fixed EAP image.