Bug 754982

Summary: trouble building on kvm host RHEL 6.2
Product: [Retired] CloudForms Cloud Engine Reporter: wes hayutin <whayutin>
Component: imagefactoryAssignee: Ian McLeod <imcleod>
Status: CLOSED CURRENTRELEASE QA Contact: wes hayutin <whayutin>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 1.0.0CC: akarol, dajohnso, deltacloud-maint, dgao, rjones, ssachdev
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-30 17:15:25 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description wes hayutin 2011-11-18 13:25:51 UTC
Description of problem:

oz builds on a host platform of RHEL 6.2 seem to be failing

ssh root.lab.eng.bos.redhat.com aeolus-cli build --target vsphere --template=/root/RHEL61vmwareTools.tpl
Stdout: 
Image: b4104dea-c6e8-4986-9ed2-4ba7ab18e76b
Build: bde821f1-fb2d-4788-9a09-deac1cb95c7e
Target Image: 6391d0c9-efad-4627-8ac4-7cb2c7b77eb4	 :Status BUILDING
Stderr:
ExitCode: 0


11-11-17 23:06:09,876 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Putting the kickstart in place
2011-11-17 23:06:09,877 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Writing cpio to /home/var/lib/oz/icicletmp/RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4/extra.cpio
2011-11-17 23:06:09,931 DEBUG imgfac.BuildJob.BuildJob pid(11607) Message: Builder (6391d0c9-efad-4627-8ac4-7cb2c7b77eb4) changed percent complete from 0 to 10
2011-11-17 23:06:09,931 INFO oz.Guest.RHEL6Guest pid(11607) Message: Generating 10GB diskimage for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4
2011-11-17 23:06:09,931 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(11607) Message: Doing base install via Oz
2011-11-17 23:06:09,931 INFO oz.Guest.RHEL6Guest pid(11607) Message: Running install for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4
2011-11-17 23:06:09,931 INFO oz.Guest.RHEL6Guest pid(11607) Message: Generate XML for guest RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 with bootdev None
2011-11-17 23:06:09,932 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Generated XML:
<?xml version="1.0"?>
<domain type="kvm">
  <name>RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4</name>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <uuid>5c5a8844-bef7-4e58-bfda-8e8557e8c782</uuid>
  <clock offset="utc"/>
  <vcpu>1</vcpu>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <os>
    <type>hvm</type>
    <kernel>/home/var/lib/imagefactory/images/RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4-kernel</kernel>
    <initrd>/home/var/lib/imagefactory/images/RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4-ramdisk</initrd>
    <cmdline>method=http://download.devel.redhat.com/released/RHEL-6-Server/6.1/x86_64/os/ ks=file:/ks.cfg</cmdline>
  </os>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <console device="pty"/>
    <graphics port="-1" type="vnc"/>
    <interface type="bridge">
      <source bridge="virbr0"/>
      <mac address="52:54:00:7f:89:0c"/>
      <model type="virtio"/>
    </interface>
    <input bus="ps2" type="mouse"/>
    <console type="pty">
      <target port="0"/>
    </console>
    <serial type="tcp">
      <source mode="bind" host="127.0.0.1" service="21545"/>
      <protocol type="raw"/>
      <target port="1"/>
    </serial>
    <disk device="disk" type="file">
      <target dev="vda" bus="virtio"/>
      <source file="/home/var/lib/imagefactory/images/base-image-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4.dsk"/>
    </disk>
  </devices>
</domain>

2011-11-17 23:06:10,444 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3600/3600
2011-11-17 23:06:20,715 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3590/3600
2011-11-17 23:06:31,025 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3580/3600
2011-11-17 23:06:41,324 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3570/3600
2011-11-17 23:06:51,647 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3560/3600
2011-11-17 23:07:01,955 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3550/3600
2011-11-17 23:07:12,578 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3540/3600
2011-11-17 23:07:22,875 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3530/3600
2011-11-17 23:07:33,131 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3520/3600
2011-11-17 23:07:43,464 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3510/3600
2011-11-17 23:07:53,758 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3500/3600
2011-11-17 23:08:03,479 DEBUG paste.httpserver.ThreadPool pid(11607) Message: Added task (0 tasks queued)
2011-11-17 23:08:04,090 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3490/3600
2011-11-17 23:08:14,354 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3480/3600
2011-11-17 23:08:24,952 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3470/3600
2011-11-17 23:08:35,207 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3460/3600
2011-11-17 23:08:45,486 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3450/3600
2011-11-17 23:08:55,882 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3440/3600
2011-11-17 23:09:06,255 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3430/3600
2011-11-17 23:09:16,561 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3420/3600
2011-11-17 23:09:26,875 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3410/3600
2011-11-17 23:09:37,230 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3400/3600
2011-11-17 23:09:47,736 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3390/3600
2011-11-17 23:09:58,000 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3380/3600
2011-11-17 23:10:03,673 DEBUG paste.httpserver.ThreadPool pid(11607) Message: Added task (0 tasks queued)
2011-11-17 23:10:08,296 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3370/3600
2011-11-17 23:10:18,652 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3360/3600
2011-11-17 23:10:28,891 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3350/3600
2011-11-17 23:10:39,230 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3340/3600
2011-11-17 23:10:49,578 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3330/3600
2011-11-17 23:10:59,850 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3320/3600
2011-11-17 23:11:10,172 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3310/3600
2011-11-17 23:11:20,620 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3300/3600
2011-11-17 23:11:30,909 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3290/3600
2011-11-17 23:11:41,264 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3280/3600
2011-11-17 23:11:51,578 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3270/3600
2011-11-17 23:12:01,860 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3260/3600
2011-11-17 23:12:03,866 DEBUG paste.httpserver.ThreadPool pid(11607) Message: Added task (0 tasks queued)
2011-11-17 23:12:12,281 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3250/3600
2011-11-17 23:12:22,543 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3240/3600
2011-11-17 23:12:33,016 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3230/3600
2011-11-17 23:12:43,546 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3220/3600
2011-11-17 23:12:53,811 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3210/3600
2011-11-17 23:13:04,056 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3200/3600
2011-11-17 23:13:14,423 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3190/3600
2011-11-17 23:13:24,728 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Waiting for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 to finish installing, 3180/3600
2011-11-17 23:13:24,730 INFO oz.Guest.RHEL6Guest pid(11607) Message: Install of RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 succeeded
2011-11-17 23:13:24,730 INFO oz.Guest.RHEL6Guest pid(11607) Message: Generate XML for guest RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4 with bootdev hd
2011-11-17 23:13:24,731 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Generated XML:
<?xml version="1.0"?>
<domain type="kvm">
  <name>RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4</name>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <uuid>5c5a8844-bef7-4e58-bfda-8e8557e8c782</uuid>
  <clock offset="utc"/>
  <vcpu>1</vcpu>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <os>
    <type>hvm</type>
    <boot dev="hd"/>
  </os>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <console device="pty"/>
    <graphics port="-1" type="vnc"/>
    <interface type="bridge">
      <source bridge="virbr0"/>
      <mac address="52:54:00:7f:89:0c"/>
      <model type="virtio"/>
    </interface>
    <input bus="ps2" type="mouse"/>
    <console type="pty">
      <target port="0"/>
    </console>
    <serial type="tcp">
      <source mode="bind" host="127.0.0.1" service="21545"/>
      <protocol type="raw"/>
      <target port="1"/>
    </serial>
    <disk device="disk" type="file">
      <target dev="vda" bus="virtio"/>
      <source file="/home/var/lib/imagefactory/images/base-image-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4.dsk"/>
    </disk>
  </devices>
</domain>

2011-11-17 23:13:24,732 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(11607) Message: Base install complete - Doing customization and ICICLE generation
2011-11-17 23:13:24,732 DEBUG imgfac.BuildJob.BuildJob pid(11607) Message: Builder (6391d0c9-efad-4627-8ac4-7cb2c7b77eb4) changed percent complete from 10 to 30
2011-11-17 23:13:24,732 INFO oz.Guest.RHEL6Guest pid(11607) Message: Customizing image
2011-11-17 23:13:24,733 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Generated XML:
<?xml version="1.0"?>
<domain type="kvm">
  <name>RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4</name>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <uuid>5c5a8844-bef7-4e58-bfda-8e8557e8c782</uuid>
  <clock offset="utc"/>
  <vcpu>1</vcpu>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <os>
    <type>hvm</type>
    <boot dev="hd"/>
  </os>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <console device="pty"/>
    <graphics port="-1" type="vnc"/>
    <interface type="bridge">
      <source bridge="virbr0"/>
      <mac address="52:54:00:7f:89:0c"/>
      <model type="virtio"/>
    </interface>
    <input bus="ps2" type="mouse"/>
    <console type="pty">
      <target port="0"/>
    </console>
    
    <disk device="disk" type="file">
      <target dev="vda" bus="virtio"/>
      <source file="/home/var/lib/imagefactory/images/base-image-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4.dsk"/>
    </disk>
  <serial type="tcp"><source mode="bind" host="127.0.0.1" service="21545"/><protocol type="raw"/><target port="1"/></serial></devices>
</domain>

2011-11-17 23:13:24,733 INFO oz.Guest.RHEL6Guest pid(11607) Message: Collection Setup
2011-11-17 23:13:24,734 INFO oz.Guest.RHEL6Guest pid(11607) Message: Setting up guestfs handle for RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4
2011-11-17 23:13:24,735 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Adding disk image /home/var/lib/imagefactory/images/base-image-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4.dsk
2011-11-17 23:13:24,735 DEBUG oz.Guest.RHEL6Guest pid(11607) Message: Launching guestfs
2011-11-17 23:13:36,122 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(11607) Message: Exception caught in ImageFactory
2011-11-17 23:13:36,125 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(11607) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 105, in build_upload
    self.output_descriptor = guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1147, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1112, in _internal_customize
    self._collect_setup(modified_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 427, in _collect_setup
    g_handle = self._guestfs_handle_setup(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 859, in _guestfs_handle_setup
    g.launch()
  File "/usr/lib/python2.6/site-packages/guestfs.py", line 152, in launch
    return libguestfsmod.launch (self._o)
RuntimeError: unexpected end of file when reading from daemon.
This usually means the libguestfs appliance failed to start up.  Please
enable debugging (LIBGUESTFS_DEBUG=1) and rerun the command, then look at
the debug messages output prior to this error.
Or you can run 'libguestfs-test-tool' and post the complete output into
a bug report or message to the libguestfs mailing list.

2011-11-17 23:13:36,125 INFO oz.Guest.RHEL6Guest pid(11607) Message: Cleaning up guest named RHEL_VMWareTools-6391d0c9-efad-4627-8ac4-7cb2c7b77eb4
2011-11-17 23:13:36,364 DEBUG imgfac.BuildJob.BuildJob pid(11607) Message: Builder (6391d0c9-efad-4627-8ac4-7cb2c7b77eb4) changed status from BUILDING to FAILED
2011-11-17 23:13:36,364 INFO oz.Guest.RHEL6Guest pid(11607) Message: Cleaning up after install
2011-11-17 23:13:36,372 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(11607) Message: Exception caught in ImageFactory
2011-11-17 23:13:36,372 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(11607) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 49, in build_image
    self.build_upload(build_id)
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 105, in build_upload
    self.output_descriptor = guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1147, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1112, in _internal_customize
    self._collect_setup(modified_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 427, in _collect_setup
    g_handle = self._guestfs_handle_setup(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 859, in _guestfs_handle_setup
    g.launch()
  File "/usr/lib/python2.6/site-packages/guestfs.py", line 152, in launch
    return libguestfsmod.launch (self._o)
RuntimeError: unexpected end of file when reading from daemon.
This usually means the libguestfs appliance failed to start up.  Please
enable debugging (LIBGUESTFS_DEBUG=1) and rerun the command, then look at
the debug messages output prior to this error.
Or you can run 'libguestfs-test-tool' and post the complete output into
a bug report or message to the libguestfs mailing list.

2011-11-17 23:13:36,373 DEBUG imgfac.BuildJob.BuildJob pid(11607) Message: Builder (6391d0c9-efad-4627-8ac4-7cb2c7b77eb4) changed status from FAILED to FAILED
2011-11-17 23:14:04,067 DEBUG paste.httpserver.ThreadPool pid(11607) Message: Added task (0 tasks queued)
[root@qeblade30 /]# 


[root@qeblade30 /]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 6.2 (Santiago)
[root@qeblade30 /]# 

===== Test starts here =====
LIBGUESTFS_DEBUG=1
library version: 1.7.17
guestfs_get_append: (null)
guestfs_get_autosync: 1
guestfs_get_memsize: 500
guestfs_get_path: /usr/lib64/guestfs
guestfs_get_qemu: /usr/libexec/qemu-kvm
guestfs_get_verbose: 1
Launching appliance, timeout set to 120 seconds.
Guest launched OK.
===== TEST FINISHED OK =====

Comment 1 wes hayutin 2011-11-18 16:21:03 UTC
[root@qeblade30 /]# rpm -qa | grep redhat-release
redhat-release-server-6Server-6.2.0.3.el6.x86_64

Comment 2 wes hayutin 2011-11-30 15:07:20 UTC
need some comments here so we can determine if we need to take this to the RHEL team

Comment 3 wes hayutin 2011-12-01 14:34:41 UTC
So I'm hoping this is a bug that the aeolus team can fix. The need info was removed w/o comment.. Will contact Ian to confirm.

Comment 4 Dave Johnson 2011-12-05 15:28:03 UTC
[root@intel-s3ea2-02 ~]# rpm -qa | egrep 'aeolus|iwhd|imagefactory' | sort
aeolus-all-0.7.0-2.el6.noarch
aeolus-conductor-0.7.0-2.el6.noarch
aeolus-conductor-daemons-0.7.0-2.el6.noarch
aeolus-conductor-doc-0.7.0-2.el6.noarch
aeolus-configure-2.4.0-0.20111129190155gitffc966c.el6.noarch
imagefactory-0.9.1-1.el6.noarch
imagefactory-jeosconf-ec2-fedora-0.9.1-1.el6.noarch
imagefactory-jeosconf-ec2-rhel-0.9.1-1.el6.noarch
iwhd-1.0-1.el6.x86_64
rubygem-aeolus-cli-0.2.0-0.20111129170214gitd721dc0.el6.noarch
rubygem-aeolus-image-0.2.0-0.20111129234803git253239e.el6.noarch
rubygem-arel-2.0.10-0.aeolus.el6.noarch
rubygem-imagefactory-console-0.5.0-4.20110824113238gitd9debef.el6.noarch
rubygem-rack-mount-0.7.1-3.aeolus.el6.noarch
rubygem-ZenTest-4.3.3-2.aeolus.el6.noarch



2011-12-05 10:00:00,790 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Uploading 10485760kB of 10485760kB
2011-12-05 10:00:00,790 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Uploading 10485760kB of 10485760kB
2011-12-05 10:00:00,790 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Uploading 10485760kB of 10485760kB
2011-12-05 10:00:00,790 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Setting metadata ({'icicle': 'd20b2215-6634-46b0-b1d9-7806263ff099', 'uuid': '6f870a3a-cf63-4192-96d4-7adb4103c54e', 'template': '5d373848-9847-4341-8c92-947b01837d14', 'target_parameters': '<?xml version="1.0"?>\n<domain type="kvm">\n  <name>RHEL_westest01-6f870a3a-cf63-4192-96d4-7adb4103c54e</name>\n  <memory>1048576</memory>\n  <currentMemory>1048576</currentMemory>\n  <uuid>6440dffb-2b70-4e17-a769-c2b4242f6fee</uuid>\n  <clock offset="utc"/>\n  <vcpu>1</vcpu>\n  <features>\n    <acpi/>\n    <apic/>\n    <pae/>\n  </features>\n  <os>\n    <type>hvm</type>\n    <boot dev="hd"/>\n  </os>\n  <on_poweroff>destroy</on_poweroff>\n  <on_reboot>destroy</on_reboot>\n  <on_crash>destroy</on_crash>\n  <devices>\n    <console device="pty"/>\n    <graphics port="-1" type="vnc"/>\n    <interface type="bridge">\n      <source bridge="virbr0"/>\n      <mac address="52:54:00:04:4c:6d"/>\n      <model type="virtio"/>\n    </interface>\n    <input bus="ps2" type="mouse"/>\n    <console type="pty">\n      <target port="0"/>\n    </console>\n    <serial type="tcp">\n      <source mode="bind" host="127.0.0.1" service="2699"/>\n      <protocol type="raw"/>\n      <target port="1"/>\n    </serial>\n    <disk device="disk" type="file">\n      <target dev="vda" bus="virtio"/>\n      <source file="/var/lib/imagefactory/images/rhevm-image-6f870a3a-cf63-4192-96d4-7adb4103c54e.dsk"/>\n    </disk>\n  </devices>\n</domain>\n', 'object_type': 'target_image', 'target': 'rhevm', 'build': '75da933e-e200-4472-b5f8-52bdfa5a1c09'}) for http://localhost:9090/target_images/6f870a3a-cf63-4192-96d4-7adb4103c54e
2011-12-05 10:00:01,044 DEBUG imgfac.builders.BaseBuilder.RHEL6_rhevm_Builder pid(23926) Message: Image warehouse storage complete
2011-12-05 10:00:01,044 DEBUG imgfac.BuildJob.BuildJob pid(23926) Message: Builder (6f870a3a-cf63-4192-96d4-7adb4103c54e) changed percent complete from 50 to 100
2011-12-05 10:00:01,045 DEBUG imgfac.BuildJob.BuildJob pid(23926) Message: Builder (6f870a3a-cf63-4192-96d4-7adb4103c54e) changed status from BUILDING to COMPLETED
2011-12-05 10:00:01,045 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Getting metadata (['latest_unpushed']) from http://localhost:9090/images/43743448-71ae-4a50-9f45-2b8be3a57bbe
2011-12-05 10:00:01,047 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Image Warehouse returned status (404) with message: 
2011-12-05 10:00:01,047 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Getting metadata (['latest_build']) from http://localhost:9090/images/43743448-71ae-4a50-9f45-2b8be3a57bbe
2011-12-05 10:00:01,049 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Image Warehouse returned status (404) with message: 
2011-12-05 10:00:01,049 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(23926) Message: Setting metadata ({'latest_unpushed': '75da933e-e200-4472-b5f8-52bdfa5a1c09'}) for http://localhost:9090/images/43743448-71ae-4a50-9f45-2b8be3a57bbe

^C
[root@intel-s3ea2-02 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 6.2 (Santiago)
[root@intel-s3ea2-02 ~]#

Comment 5 wes hayutin 2011-12-07 20:35:33 UTC
2011-12-07 15:25:26,778 DEBUG oz.Guest.RHEL6Guest pid(3055) Message: Launching guestfs
2011-12-07 15:25:28,091 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(3055) Message: Exception caught in ImageFactory
2011-12-07 15:25:28,115 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(3055) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 105, in build_upload
    self.output_descriptor = guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1147, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1112, in _internal_customize
    self._collect_setup(modified_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 427, in _collect_setup
    g_handle = self._guestfs_handle_setup(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 859, in _guestfs_handle_setup
    g.launch()
  File "/usr/lib/python2.6/site-packages/guestfs.py", line 152, in launch
    return libguestfsmod.launch (self._o)
RuntimeError: unexpected end of file when reading from daemon.
This usually means the libguestfs appliance failed to start up.  Please
enable debugging (LIBGUESTFS_DEBUG=1) and rerun the command, then look at
the debug messages output prior to this error.
Or you can run 'libguestfs-test-tool' and post the complete output into
a bug report or message to the libguestfs mailing list.

2011-12-07 15:25:28,115 INFO oz.Guest.RHEL6Guest pid(3055) Message: Cleaning up guest named RHEL6-Basic-d98f7b37-5868-4070-a1b8-4ea2cd783688
2011-12-07 15:25:28,267 DEBUG imgfac.BuildJob.BuildJob pid(3055) Message: Builder (d98f7b37-5868-4070-a1b8-4ea2cd783688) changed status from BUILDING to FAILED
2011-12-07 15:25:28,268 INFO oz.Guest.RHEL6Guest pid(3055) Message: Cleaning up after install
2011-12-07 15:25:28,274 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(3055) Message: Exception caught in ImageFactory
2011-12-07 15:25:28,274 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(3055) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 49, in build_image
    self.build_upload(build_id)
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 105, in build_upload
    self.output_descriptor = guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1147, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1112, in _internal_customize
    self._collect_setup(modified_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 427, in _collect_setup
    g_handle = self._guestfs_handle_setup(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 859, in _guestfs_handle_setup
    g.launch()
  File "/usr/lib/python2.6/site-packages/guestfs.py", line 152, in launch
    return libguestfsmod.launch (self._o)
RuntimeError: unexpected end of file when reading from daemon.
This usually means the libguestfs appliance failed to start up.  Please
enable debugging (LIBGUESTFS_DEBUG=1) and rerun the command, then look at
the debug messages output prior to this error.
Or you can run 'libguestfs-test-tool' and post the complete output into
a bug report or message to the libguestfs mailing list.

2011-12-07 15:25:28,274 DEBUG imgfac.BuildJob.BuildJob pid(3055) Message: Builder (d98f7b37-5868-4070-a1b8-4ea2cd783688) changed status from FAILED to FAILED
2011-12-07 15:26:15,679 DEBUG paste.httpserver.ThreadPool pid(3055) Message: Added task (0 tasks queued)
^C

Comment 6 wes hayutin 2011-12-08 17:41:19 UTC
with downgraded libguestfs 

[root@qeblade30 ~]# rpm -q libguestfs
libguestfs-1.7.17-17.el6.x86_64



<domain type="kvm">
  <name>RHEL6_VMWareTools-6b2c279b-0eae-4192-adee-63476587ef0e</name>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <uuid>501b27f4-82f1-4d96-ac6a-729d9f0afcbd</uuid>
  <clock offset="utc"/>
  <vcpu>1</vcpu>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <os>
    <type>hvm</type>
    <boot dev="hd"/>
  </os>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <console device="pty"/>
    <graphics port="-1" type="vnc"/>
    <interface type="bridge">
      <source bridge="virbr0"/>
      <mac address="52:54:00:f0:c2:63"/>
      <model type="virtio"/>
    </interface>
    <input bus="ps2" type="mouse"/>
    <console type="pty">
      <target port="0"/>
    </console>
    
    <disk device="disk" type="file">
      <target dev="vda" bus="virtio"/>
      <source file="/home/var/lib/imagefactory/images/base-image-6b2c279b-0eae-4192-adee-63476587ef0e.dsk"/>
    </disk>
  <serial type="tcp"><source mode="bind" host="127.0.0.1" service="60195"/><protocol type="raw"/><target port="1"/></serial></devices>
</domain>

2011-12-08 12:14:11,265 INFO oz.Guest.RHEL6Guest pid(24296) Message: Collection Setup
2011-12-08 12:14:11,267 INFO oz.Guest.RHEL6Guest pid(24296) Message: Setting up guestfs handle for RHEL6_VMWareTools-6b2c279b-0eae-4192-adee-63476587ef0e
2011-12-08 12:14:11,267 DEBUG oz.Guest.RHEL6Guest pid(24296) Message: Adding disk image /home/var/lib/imagefactory/images/base-image-6b2c279b-0eae-4192-adee-63476587ef0e.dsk
2011-12-08 12:14:11,267 DEBUG oz.Guest.RHEL6Guest pid(24296) Message: Launching guestfs
2011-12-08 12:14:15,580 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(24296) Message: Exception caught in ImageFactory
2011-12-08 12:14:15,584 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(24296) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 105, in build_upload
    self.output_descriptor = guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1147, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1112, in _internal_customize
    self._collect_setup(modified_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 427, in _collect_setup
    g_handle = self._guestfs_handle_setup(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 859, in _guestfs_handle_setup
    g.launch()
  File "/usr/lib/python2.6/site-packages/guestfs.py", line 151, in launch
    return libguestfsmod.launch (self._o)
RuntimeError: unexpected end of file when reading from daemon

2011-12-08 12:14:15,584 INFO oz.Guest.RHEL6Guest pid(24296) Message: Cleaning up guest named RHEL6_VMWareTools-6b2c279b-0eae-4192-adee-63476587ef0e
2011-12-08 12:14:15,828 DEBUG imgfac.BuildJob.BuildJob pid(24296) Message: Builder (6b2c279b-0eae-4192-adee-63476587ef0e) changed status from BUILDING to FAILED
2011-12-08 12:14:15,828 INFO oz.Guest.RHEL6Guest pid(24296) Message: Cleaning up after install
2011-12-08 12:14:15,836 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(24296) Message: Exception caught in ImageFactory
2011-12-08 12:14:15,836 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(24296) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 49, in build_image
    self.build_upload(build_id)
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 105, in build_upload
    self.output_descriptor = guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1147, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1112, in _internal_customize
    self._collect_setup(modified_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 427, in _collect_setup
    g_handle = self._guestfs_handle_setup(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 859, in _guestfs_handle_setup
    g.launch()
  File "/usr/lib/python2.6/site-packages/guestfs.py", line 151, in launch
    return libguestfsmod.launch (self._o)
RuntimeError: unexpected end of file when reading from daemon

2011-12-08 12:14:15,836 DEBUG imgfac.BuildJob.BuildJob pid(24296) Message: Builder (6b2c279b-0eae-4192-adee-63476587ef0e) changed status from FAILED to FAILED

Comment 7 Richard W.M. Jones 2011-12-08 17:57:11 UTC
What is the error message when you enable debugging?

Comment 8 wes hayutin 2011-12-08 19:19:08 UTC
whayutin is an idiot and changed libvirt and factory dirs w/o updating selinux

Comment 9 wes hayutin 2012-01-03 14:30:37 UTC
These bugs are verified, removing from ce-sprint