Bug 1428797

Summary: some of events appear in timelines w/o "source vm"
Product: Red Hat CloudForms Management Engine Reporter: Ievgen Zapolskyi <izapolsk>
Component: ProvidersAssignee: Adam Grare <agrare>
Status: CLOSED ERRATA QA Contact: Ievgen Zapolskyi <izapolsk>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.7.0CC: cpelland, gblomqui, jfrey, jhardy, obarenbo, simaishi
Target Milestone: GA   
Target Release: 5.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: ui:timeline:event
Fixed In Version: 5.10.0.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-02-07 23:02:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Bug
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core Target Upstream Version:
Embargoed:
Attachments:
Description Flags
screenshot
none
event_streams table
none
vms table
none
screenshot none

Description Ievgen Zapolskyi 2017-03-03 11:36:19 UTC 
Description of problem:
I found at least 2 VM events received from vsphere6-nested provider which appear w/o source vm in provider's Timelines. Those events don't exist in vm's Timelines accordingly.
Those events have empty vm_or_template_id and aren't assosiated with appropriate vm. As far as I'm concerned it happens because this vm was added to vms table after receiving those messages.

Version-Release number of selected component (if applicable):
5.7.1.3

How reproducible:
100%

Steps to Reproduce:
This issue happens when provider obtains information about new vm when it already received some of vm's events

Actual results:
such events don't belong to any vm in provider's Timelines and are absent in appropriate vm's Timelines

Expected results:
Such events should either be updated later when vm shows up in vms table
or shouldn't be inserted.

Additional info:
I've attached screenshot, dump of event_streams table and vms table.
Comment 2 Ievgen Zapolskyi 2017-03-03 11:40:09 UTC 
Created attachment 1259519 [details]
screenshot

glance at messages and pay attention on source vm field
Comment 3 Ievgen Zapolskyi 2017-03-03 11:45:22 UTC 
Created attachment 1259521 [details]
event_streams table
Comment 4 Ievgen Zapolskyi 2017-03-03 11:47:12 UTC 
Created attachment 1259523 [details]
vms table
Comment 5 Ievgen Zapolskyi 2017-03-03 12:09:55 UTC 
Created attachment 1259526 [details]
screenshot
Comment 6 Adam Grare 2017-03-27 14:50:21 UTC 
After an ems_refresh we run a number of "post_refresh_ems" tasks, one of which is to connect events to new VMs that came in before the VM was in our DB, https://github.com/ManageIQ/manageiq/blob/master/app/models/vm_or_template.rb#L804-L830

I was able to reproduce what you describe and from looking at that query and my events that don't have a vm_or_template_id set my guess is that the vm_location being nil in the event is causing reconnect_events to fail.  I'll investigate if there is another way to find the events that will work with more event types.
Comment 7 Adam Grare 2018-03-13 13:57:02 UTC 
From my testing I don't see events linked up to new VMs when those events were delivered prior to the cached VM having a location.  In this case we have no context to reconnect the VMs with.

The only thing we could do to more reliably connect these events is to use the ems_ref as that is populated earlier than the location.  This will require a schema change to add the vm_ems_ref to the ems_events table.
Comment 8 CFME Bot 2018-03-13 15:10:19 UTC 
https://github.com/ManageIQ/manageiq-schema/pull/176
Comment 9 CFME Bot 2018-03-13 15:16:07 UTC 
https://github.com/ManageIQ/manageiq/pull/17145
Comment 10 CFME Bot 2018-03-15 21:03:31 UTC 
New commit detected on ManageIQ/manageiq-schema/master:

https://github.com/ManageIQ/manageiq-schema/commit/664d9899f8ad61eb140ab7bc2b14c7bc7f3d9be3
commit 664d9899f8ad61eb140ab7bc2b14c7bc7f3d9be3
Author:     Adam Grare <agrare>
AuthorDate: Tue Mar 13 10:00:13 2018 -0400
Commit:     Adam Grare <agrare>
CommitDate: Tue Mar 13 10:00:13 2018 -0400

    Add vm_ems_ref to event_streams

    Early events in a VM's lifecycle don't contain the vmPathName yet which
    is used to connect events which were processed before the VM gets
    created by EmsRefresh.  These events do however contain the ems_ref of
    the VM which can be used for more reliable event -> vm association.

    https://bugzilla.redhat.com/show_bug.cgi?id=1428797

 db/migrate/20180313135925_add_vm_ems_ref_to_event_streams.rb | 6 +
 1 file changed, 6 insertions(+)
Comment 11 CFME Bot 2018-03-19 18:01:08 UTC 
New commit detected on ManageIQ/manageiq/master:

https://github.com/ManageIQ/manageiq/commit/1cfd5ebaca03cd3f6e888f3f1be57424f67d6d56
commit 1cfd5ebaca03cd3f6e888f3f1be57424f67d6d56
Author:     Adam Grare <agrare>
AuthorDate: Tue Mar 13 10:55:33 2018 -0400
Commit:     Adam Grare <agrare>
CommitDate: Tue Mar 13 10:55:33 2018 -0400

    Use vm_ems_ref to reconnect events

    Events delivered early in a VM's lifecycle often do not contain the
    vmPathName yet and therefore cannot be connected to VMs which are saved
    after the events are processed.  It is much more reliable to use the
    VM's ems_ref since that is available in all tasks and events except the
    initial CreateVM_Task/CloneVM_Task (since the target VM hasn't been
    created yet it isn't possible to link these to a VM anyway).

    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1428797

 app/models/ems_event.rb | 9 +-
 app/models/vm_or_template.rb | 6 +-
 2 files changed, 11 insertions(+), 4 deletions(-)
Comment 12 Ievgen Zapolskyi 2018-06-26 12:10:24 UTC 
VERIFIED in 5.10.0.1
Comment 14 errata-xmlrpc 2019-02-07 23:02:25 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:0212