Bug 800290

Summary: aeolus-audrey-agent fails to resolve host while pushing
Product: [Retired] CloudForms Cloud Engine Reporter: Shveta <ssachdev>
Component: aeolus-audrey-agentAssignee: Greg Blomquist <gblomqui>
Status: CLOSED NOTABUG QA Contact: dgao
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.0.0CC: hbrock, redakkan, whayutin
Target Milestone: beta4   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-03-06 14:37:09 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Shveta 2012-03-06 08:52:39 UTC 
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Added configserver for ec2
2. Built and pushed with this template http://pastebin.test.redhat.com/79326
3. Build suceeded , push failed with reason mentioned here 
    http://pastebin.test.redhat.com/79330
4. Removed audrey agent from template http://pastebin.test.redhat.com/79328
5. Build and push suceeded .
  
Actual results:


Expected results:


Additional info:

===========================http://pastebin.test.redhat.com/79326 =======
<template>
  <name>RHEL5_7 open_vm_tools audrey</name>
  <os>
    <name>RHEL-5</name>
    <version>U7</version>
    <arch>x86_64</arch>
    <install type='url'>
      <url>http://download.lab.bos.redhat.com/released/RHEL-5-Server/U7/x86_64/os/</url>
    </install>
    <rootpw>redhat</rootpw>
  </os>
  <repositories>
    <repository name="rhel">
      <url>http://download.lab.bos.redhat.com/released/RHEL-5-Server/U7/x86_64/os/Server/</url>
      <signed>false</signed>
    </repository>
    <repository name="el5 tools">
      <url>http://download.lab.bos.redhat.com/rel-eng/repos/cf-tools-1.0-rhel-5-candidate/x86_64/</url>
      <signed>false</signed>
    </repository>
  </repositories>
  <packages>
    <package name="open-vm-tools"/>
    <package name="aeolus-audrey-agent"/>
  </packages>
  <description>RHEL 5.7 w/ Open Vm Tools Audrey</description>
</template>

================http://pastebin.test.redhat.com/79330======================
2012-03-06 02:35:35,638 DEBUG imgfac.builders.BaseBuilder.RHEL5_ec2_Builder thread(d1966513) Message: Removed temporary security group (imagefactory-d1966513-9349-498f-9382-0a8fd5bb7b94)
2012-03-06 02:35:35,638 DEBUG imgfac.builders.BaseBuilder.RHEL5_ec2_Builder thread(d1966513) Message: Exception caught in ImageFactory
2012-03-06 02:35:35,640 DEBUG imgfac.builders.BaseBuilder.RHEL5_ec2_Builder thread(d1966513) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_ec2_Builder.py", line 458, in push_image
    credentials)
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_ec2_Builder.py", line 665, in push_image_snapshot_ec2
    self.guest.do_customize(guestaddr)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1110, in do_customize
    tunnels=self.tunnels)
  File "/usr/lib/python2.6/site-packages/imgfac/builders/RHEL5_ec2_Builder.py", line 33, in guest_execute_command
    return super(RHEL5RemoteGuest, self).guest_execute_command(guestaddr, command, timeout, tunnels)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 476, in guest_execute_command
    command, timeout, tunnels)
  File "/usr/lib/python2.6/site-packages/oz/ozutil.py", line 355, in ssh_execute_command
    return subprocess_check_output(cmd)
  File "/usr/lib/python2.6/site-packages/oz/ozutil.py", line 323, in subprocess_check_output
    raise SubprocessException("'%s' failed(%d): %s" % (cmd, retcode, stderr), retcode)
SubprocessException: 'ssh -i /tmp/tmpn29ATw -F /dev/null -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o ConnectTimeout=30 -o UserKnownHostsFile=/dev/null -o PasswordAuthentication=no -R 50000:download.lab.bos.redhat.com:80 root.amazonaws.com yum -y install "open-vm-tools" "aeolus-audrey-agent" ' failed(1): Warning: Permanently added 'ec2-174-129-126-157.compute-1.amazonaws.com,174.129.126.157' (RSA) to the list of known hosts.
http://download.lab.bos.redhat.com/brewroot/packages/open-vm-tools/8.8.0/5.el5/data/signed/897da07a/x86_64/open-vm-tools-8.8.0-5.el5.x86_64.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.
http://download.lab.bos.redhat.com/brewroot/packages/aeolus-audrey-agent/0.4.4/2.el5/data/signed/897da07a/noarch/aeolus-audrey-agent-0.4.4-2.el5.noarch.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.
http://download.lab.bos.redhat.com/brewroot/packages/python-oauth2/1.2.1/4.el5/data/signed/897da07a/noarch/python-oauth2-1.2.1-4.el5.noarch.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.
http://download.lab.bos.redhat.com/brewroot/packages/libdnet/1.12/9.el5/data/signed/897da07a/x86_64/libdnet-1.12-9.el5.x86_64.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.
http://download.lab.bos.redhat.com/brewroot/packages/facter/1.5.2/3.el5/data/signed/897da07a/noarch/facter-1.5.2-3.el5.noarch.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.
http://download.lab.bos.redhat.com/brewroot/packages/python-httplib2/0.4.0/4.el5/data/signed/897da07a/noarch/python-httplib2-0.4.0-4.el5.noarch.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.
http://download.lab.bos.redhat.com/brewroot/packages/python-argparse/1.2.1/2.el5/data/signed/897da07a/noarch/python-argparse-1.2.1-2.el5.noarch.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.
http://download.lab.bos.redhat.com/brewroot/packages/open-vm-tools/8.8.0/5.el5/data/signed/897da07a/x86_64/open-vm-toolsd-8.8.0-5.el5.x86_64.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.
http://download.lab.bos.redhat.com/brewroot/packages/open-vm-tools/8.8.0/5.el5/data/signed/897da07a/x86_64/open-vm-tools-libs-8.8.0-5.el5.x86_64.rpm: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
Trying other mirror.


Error Downloading Packages:
  aeolus-audrey-agent-0.4.4-2.el5.noarch: failed to retrieve packages/aeolus-audrey-agent/0.4.4/2.el5/data/signed/897da07a/noarch/aeolus-audrey-agent-0.4.4-2.el5.noarch.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
  libdnet-1.12-9.el5.x86_64: failed to retrieve packages/libdnet/1.12/9.el5/data/signed/897da07a/x86_64/libdnet-1.12-9.el5.x86_64.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
  open-vm-tools-libs-8.8.0-5.el5.x86_64: failed to retrieve packages/open-vm-tools/8.8.0/5.el5/data/signed/897da07a/x86_64/open-vm-tools-libs-8.8.0-5.el5.x86_64.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
  open-vm-toolsd-8.8.0-5.el5.x86_64: failed to retrieve packages/open-vm-tools/8.8.0/5.el5/data/signed/897da07a/x86_64/open-vm-toolsd-8.8.0-5.el5.x86_64.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
  python-oauth2-1.2.1-4.el5.noarch: failed to retrieve packages/python-oauth2/1.2.1/4.el5/data/signed/897da07a/noarch/python-oauth2-1.2.1-4.el5.noarch.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
  open-vm-tools-8.8.0-5.el5.x86_64: failed to retrieve packages/open-vm-tools/8.8.0/5.el5/data/signed/897da07a/x86_64/open-vm-tools-8.8.0-5.el5.x86_64.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
  python-httplib2-0.4.0-4.el5.noarch: failed to retrieve packages/python-httplib2/0.4.0/4.el5/data/signed/897da07a/noarch/python-httplib2-0.4.0-4.el5.noarch.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
  python-argparse-1.2.1-2.el5.noarch: failed to retrieve packages/python-argparse/1.2.1/2.el5/data/signed/897da07a/noarch/python-argparse-1.2.1-2.el5.noarch.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
  facter-1.5.2-3.el5.noarch: failed to retrieve packages/facter/1.5.2/3.el5/data/signed/897da07a/noarch/facter-1.5.2-3.el5.noarch.rpm from el5_tools
error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
==========================================================================
http://pastebin.test.redhat.com/79328
=============================================================
<template>
  <name>RHEL5_7 open_vm_tools audrey</name>
  <os>
    <name>RHEL-5</name>
    <version>U7</version>
    <arch>x86_64</arch>
    <install type='url'>
      <url>http://download.lab.bos.redhat.com/released/RHEL-5-Server/U7/x86_64/os/</url>
    </install>
    <rootpw>redhat</rootpw>
  </os>
  <repositories>
    <repository name="rhel">
      <url>http://download.lab.bos.redhat.com/released/RHEL-5-Server/U7/x86_64/os/Server/</url>
      <signed>false</signed>
    </repository>
    <repository name="el5 tools">
      <url>http://download.lab.bos.redhat.com/rel-eng/repos/cf-tools-1.0-rhel-5-candidate/x86_64/</url>
      <signed>false</signed>
    </repository>
  </repositories>
  <packages>
    <package name="open-vm-tools"/>
    </packages>
  <description>RHEL 5.7 w/ Open Vm Tools Audrey</description>
</template>
Comment 1 Shveta 2012-03-06 08:53:06 UTC 
rpm -qa|grep aeolus
aeolus-conductor-daemons-0.8.0-40.el6.noarch
aeolus-configure-2.5.0-17.el6.noarch
rubygem-aeolus-image-0.3.0-12.el6.noarch
aeolus-conductor-0.8.0-40.el6.noarch
rubygem-aeolus-cli-0.3.0-12.el6.noarch
aeolus-all-0.8.0-40.el6.noarch
aeolus-conductor-doc-0.8.0-40.el6.noarch
Comment 2 Hugh Brock 2012-03-06 14:32:03 UTC 
David, can you retest?
Comment 3 dgao 2012-03-06 14:37:09 UTC 
This issue is caused by the fact that "el5 tools" is an internal repo. EC2 cannot pick up the audrey-agent pkg unless it resides on a public repo like repos.fedorapeople.org.  

At this moment in time, I am not aware of any working reverse-ssh tunneling
mechanism that would enable ec2 instances to grab packages from internal RH
repos.