Bug 1352218

Summary: VMWare inventory worker timing out during host save method
Product: Red Hat CloudForms Management Engine Reporter: Josh Carter <jocarter>
Component: ProvidersAssignee: Greg Blomquist <gblomqui>
Status: CLOSED CURRENTRELEASE QA Contact: Dave Johnson <dajohnso>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 5.5.0CC: cpelland, jfrey, jhardy, jprause, obarenbo
Target Milestone: GAKeywords: ZStream
Target Release: 5.7.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1353640 1353641 (view as bug list) Environment:
Last Closed: 2016-07-07 16:04:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1353640, 1353641    

Description Josh Carter 2016-07-02 14:18:18 UTC 
Description of problem:

It looks like the inventory collection process is timing out during the save of a given host. 

[----] I, [2016-07-01T10:32:46.637921 #26657:859990]  INFO -- : MIQ(ManageIQ::Providers::Vmware::InfraManager::RefreshWorker::Runner#get_message_via_drb) Message id: [50000036325533], MiqWorker id: [50000000593828], Zone: [SAC NONPROD WKR1], Role: [ems_inventory], Server: [], Ident: [ems_50000000000003], Target id: [], Instance id: [], Task id: [], Command: [EmsRefresh.refresh], Timeout: [7200], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [[["ManageIQ::Providers::Vmware::InfraManager", 50000000000003]]], Dequeued in: [3.698202472] seconds

[----] I, [2016-07-01T10:34:45.981790 #26657:859990]  INFO -- : MIQ(EmsRefresh.save_hosts_inventory) EMS: [Sacramento NonProd VCenter 2], id: [50000000000003] Updating Host [g31schvne572.ebiz.verizon.com] id: [50000000000405] hostname: [g31schvne572.ebiz.verizon.com] IP
: [113.131.66.148] ems_ref: [host-21427]
[----] I, [2016-07-01T10:34:46.282890 #26657:859990]  INFO -- : MIQ(EmsRefresh.save_hosts_inventory) EMS: [Sacramento NonProd VCenter 2], id: [50000000000003] Updating Host [localhost.ebiz.verizon.com] id: [50000000000406] hostname: [localhost.ebiz.verizon.com] IP: [113
.131.66.147] ems_ref: [host-19552]
[----] I, [2016-07-01T10:34:46.595387 #26657:859990]  INFO -- : MIQ(EmsRefresh.save_hosts_inventory) EMS: [Sacramento NonProd VCenter 2], id: [50000000000003] Updating Host [localhost.ebiz.verizon.com - 7] id: [50000000000407] hostname: [localhost.ebiz.verizon.com] IP:
[113.131.66.146] ems_ref: [host-19548]
[----] E, [2016-07-01T12:32:46.639960 #26657:859990] ERROR -- : MIQ(MiqQueue#deliver) Message id: [50000036325533], timed out after 7200.001486814 seconds.  Timeout threshold [7200]
[----] I, [2016-07-01T12:32:46.640162 #26657:859990]  INFO -- : MIQ(ManageIQ::Providers::Vmware::InfraManager::RefreshWorker::Runner#deliver_queue_message) EMS [vcisc1wnh01.ebiz.verizon.com] as [itcent\svc-cloudforms] Reconnecting to DB after timeout error during queue deliver

Version-Release number of selected component (if applicable): 5.5.3


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 9 CFME Bot 2016-07-07 15:57:01 UTC 
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=1e84440cbce2c3a0bd49653309a04d1bce1b1e6a

commit 1e84440cbce2c3a0bd49653309a04d1bce1b1e6a
Merge: 3f9d373 7a21acf
Author:     Greg Blomquist <gblomqui>
AuthorDate: Thu Jul 7 11:48:56 2016 -0400
Commit:     Greg Blomquist <gblomqui>
CommitDate: Thu Jul 7 11:48:56 2016 -0400

    Merge branch 'bz_1352218_duplicate_hostnames' into '5.5.z'
    
    Bz 1352218 duplicate hostnames
    
    Clean cherry-pick of https://github.com/ManageIQ/manageiq/pull/7146 and https://github.com/ManageIQ/manageiq/pull/8227
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1352218
    
    See merge request !1001

 app/models/ems_refresh/save_inventory_infra.rb       | 12 +++++++++---
 spec/models/ems_refresh/save_inventory_infra_spec.rb | 11 +++++++++++
 2 files changed, 20 insertions(+), 3 deletions(-)
Comment 10 CFME Bot 2016-07-07 15:57:06 UTC 
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=7a21acf340afb9e8e09fb9bb67234b50f273e9da

commit 7a21acf340afb9e8e09fb9bb67234b50f273e9da
Author:     Adam Grare <agrare>
AuthorDate: Mon Apr 25 12:23:49 2016 -0400
Commit:     Adam Grare <agrare>
CommitDate: Thu Jul 7 10:57:26 2016 -0400

    Fix host name collision incrementing every refresh
    
    When saving a host, check to see if the found.name is the current
    host name plus a duplicate host name suffix.  This prevents the
    host name from being constantly incremented every ems_refresh.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1352218

 app/models/ems_refresh/save_inventory_infra.rb       | 12 +++++++++---
 spec/models/ems_refresh/save_inventory_infra_spec.rb | 11 +++++++++++
 2 files changed, 20 insertions(+), 3 deletions(-)