Bug 1311218

Summary: SSA fails for vsphere55's templates
Product: Red Hat CloudForms Management Engine Reporter: Vadim Rutkovsky <vrutkovs>
Component: SmartState AnalysisAssignee: Jerry Keselman <jkeselma>
Status: CLOSED NOTABUG QA Contact: Satyajit Bulage <sbulage>
Severity: high Docs Contact:
Priority: high    
Version: 5.5.0CC: cpelland, jhardy, jteehan, obarenbo, roliveri, sbulage, vrutkovs
Target Milestone: GAKeywords: ZStream
Target Release: 5.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: vmware:smartstate
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1351331 1351334 (view as bug list) Environment:
Last Closed: 2016-07-13 20:01:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1351331, 1351334    
Attachments:
Description Flags
vim.log none

Description Vadim Rutkovsky 2016-02-23 15:40:11 UTC
Created attachment 1129823 [details]
vim.log

Description of problem:
VIX_E_HOST_NETWORK_CONN_REFUSED error appears when template is scanned

Version-Release number of selected component (if applicable):
5.5.2.4

How reproducible:
Always

Steps to Reproduce:
1. Add SmartProxy role to the server
2. Add vsphere55 provider and install vddk
3. Set credentials for hosts
4. Run SSA on template

Actual results:
Error occurs - see vim.log attached

Expected results:
Template is scanned correctly

Additional info:

Comment 1 Vadim Rutkovsky 2016-02-23 17:08:06 UTC
jfyi it seems to work fine on vsphere6

Comment 8 Satyajit Bulage 2016-07-01 07:49:08 UTC
Unable to test this until bug 1351716 is fixed.

Comment 9 Jerry Keselman 2016-07-01 13:23:55 UTC
Why?  What does that have to do with being unable to run SSA on a Template on Vcenter 5.5?

Comment 10 Jerry Keselman 2016-07-01 13:24:07 UTC
Why?  What does that have to do with being unable to run SSA on a Template on Vcenter 5.5?

Comment 12 Satyajit Bulage 2016-07-05 07:00:12 UTC
Hello Jerry,

I sent an email to you with required credentials.
I am able to reproduce it on Vcenter 5.5 using vddk 5.5.4

Thanks
Satyajit Bulage

Comment 13 Jerry Keselman 2016-07-05 11:43:50 UTC
Thank you Satyajit.  I received the email.  Can you also send me the credentials for the appliance on which you are reproducing the issue?  Thank you.

Comment 14 Jerry Keselman 2016-07-05 11:44:00 UTC
Thank you Satyajit.  I received the email.  Can you also send me the credentials for the appliance on which you are reproducing the issue?  Thank you.

Comment 15 Jerry Keselman 2016-07-05 11:51:57 UTC
Satyajit never mind about the appliance.  I just reproduced against the given template using an upstream appliance.  I have run this successfully on a different Vcenter 5.5 provider and template so I wonder if this is an environmental issue.  I will continue to investigate.

Comment 17 Jerry Keselman 2016-07-06 19:17:12 UTC
I received the credentials from Jeff Teehan.

This is an environmental issue.  During refresh the VCenter provider is returning a fully qualified hostname for the ESX hosts used to scan the templates.  However when running SSA on templates, this particular VCenter provider is returning a short hostname to reference the template on the datastore.  This short name is known to the Vcenter server (its pingable from the server) but not from the appliance.  Hence when the appliance attempts to access the template it receives the error in the description above.  When we run VM SSA it does not go through Vcenter by default with scan_via_host set to true so we would not run into this issue.

When we run SSA on templates on a different Vcenter 5.5 provider set up by development we do not run into this issue - the Vcenter server is returning the fully qualified ESX hostname in this case.

Comment 18 Jerry Keselman 2016-07-06 19:18:15 UTC
We do need to determine how the Vcenter server is configured to cause this issue so I will leave this ticket open for now.  Once we get more info about the configuration this BZ will be closed.

Comment 19 Jerry Keselman 2016-07-13 20:01:49 UTC
This Vcenter server has the following /etc/hosts file:
vcenter55u1:~ # cat /etc/hosts
10.8.58.12	cfme-esx-55-01 cfme-esx-55-01.cfme.lab.eng.rdu2.redhat.com
10.8.58.13      cfme-esx-55-02 cfme-esx-55-02.cfme.lab.eng.rdu2.redhat.com
10.8.58.15      cfme-esx-55-03 cfme-esx-55-03.cfme.lab.eng.rdu2.redhat.com
10.8.58.10	cfme-esx-55-04 cfme-esx-55-04.cfme.lab.eng.rdu2.redhat.com

127.0.0.1  vcenter55u1.cfme.lab.eng.rdu2.redhat.com vcenter55u1 localhost
::1     vcenter55u1.cfme.lab.eng.rdu2.redhat.com vcenter55u1 localhost ip6-localhost ip6-loopback

At issue is this - the VCenter server looks up a hostname for the IP address of one of the ESX servers in question, and passes back the first name it finds in the VDDK protocol.  The first names in the file are aliases that are not known by the appliance.  The appliance does not know how to talk to "cfme-esx-55-01 ... 04".  So we get the VIX_E_HOST_NETWORK_CONN_REFUSED error.  If, for instance, I put those aliases in a host file on the appliance, everything works.

On another Vcenter 5.5 server that does NOT put the ESX host entries in the /etc/hosts file, everything works fine also.

First, I would not bother with /etc/hosts - I would let DNS do its job.  Second, if you really want to put the alias in there - then put it *AFTER* the fully qualified name.

I am closing this BZ as an environmental issue.