Description of problem: I found at least 2 VM events received from vsphere6-nested provider which appear w/o source vm in provider's Timelines. Those events don't exist in vm's Timelines accordingly. Those events have empty vm_or_template_id and aren't assosiated with appropriate vm. As far as I'm concerned it happens because this vm was added to vms table after receiving those messages. Version-Release number of selected component (if applicable): 5.7.1.3 How reproducible: 100% Steps to Reproduce: This issue happens when provider obtains information about new vm when it already received some of vm's events Actual results: such events don't belong to any vm in provider's Timelines and are absent in appropriate vm's Timelines Expected results: Such events should either be updated later when vm shows up in vms table or shouldn't be inserted. Additional info: I've attached screenshot, dump of event_streams table and vms table.
Created attachment 1259519 [details] screenshot glance at messages and pay attention on source vm field
Created attachment 1259521 [details] event_streams table
Created attachment 1259523 [details] vms table
Created attachment 1259526 [details] screenshot
After an ems_refresh we run a number of "post_refresh_ems" tasks, one of which is to connect events to new VMs that came in before the VM was in our DB, https://github.com/ManageIQ/manageiq/blob/master/app/models/vm_or_template.rb#L804-L830 I was able to reproduce what you describe and from looking at that query and my events that don't have a vm_or_template_id set my guess is that the vm_location being nil in the event is causing reconnect_events to fail. I'll investigate if there is another way to find the events that will work with more event types.
From my testing I don't see events linked up to new VMs when those events were delivered prior to the cached VM having a location. In this case we have no context to reconnect the VMs with. The only thing we could do to more reliably connect these events is to use the ems_ref as that is populated earlier than the location. This will require a schema change to add the vm_ems_ref to the ems_events table.
https://github.com/ManageIQ/manageiq-schema/pull/176
https://github.com/ManageIQ/manageiq/pull/17145
New commit detected on ManageIQ/manageiq-schema/master: https://github.com/ManageIQ/manageiq-schema/commit/664d9899f8ad61eb140ab7bc2b14c7bc7f3d9be3 commit 664d9899f8ad61eb140ab7bc2b14c7bc7f3d9be3 Author: Adam Grare <agrare> AuthorDate: Tue Mar 13 10:00:13 2018 -0400 Commit: Adam Grare <agrare> CommitDate: Tue Mar 13 10:00:13 2018 -0400 Add vm_ems_ref to event_streams Early events in a VM's lifecycle don't contain the vmPathName yet which is used to connect events which were processed before the VM gets created by EmsRefresh. These events do however contain the ems_ref of the VM which can be used for more reliable event -> vm association. https://bugzilla.redhat.com/show_bug.cgi?id=1428797 db/migrate/20180313135925_add_vm_ems_ref_to_event_streams.rb | 6 + 1 file changed, 6 insertions(+)
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/1cfd5ebaca03cd3f6e888f3f1be57424f67d6d56 commit 1cfd5ebaca03cd3f6e888f3f1be57424f67d6d56 Author: Adam Grare <agrare> AuthorDate: Tue Mar 13 10:55:33 2018 -0400 Commit: Adam Grare <agrare> CommitDate: Tue Mar 13 10:55:33 2018 -0400 Use vm_ems_ref to reconnect events Events delivered early in a VM's lifecycle often do not contain the vmPathName yet and therefore cannot be connected to VMs which are saved after the events are processed. It is much more reliable to use the VM's ems_ref since that is available in all tasks and events except the initial CreateVM_Task/CloneVM_Task (since the target VM hasn't been created yet it isn't possible to link these to a VM anyway). Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1428797 app/models/ems_event.rb | 9 +- app/models/vm_or_template.rb | 6 +- 2 files changed, 11 insertions(+), 4 deletions(-)
VERIFIED in 5.10.0.1
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:0212