772781 – imagefactory does not detect local jeos image, cached modified media builds as running

Bug 772781 - imagefactory does not detect local jeos image, cached modified media builds as running

Summary: imagefactory does not detect local jeos image, cached modified media builds ...

Keywords:
Status:	CLOSED EOL
Alias:	None
Product:	CloudForms Cloud Engine
Classification:	Retired
Component:	oz
Sub Component:
Version:	1.0.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	medium
Target Milestone:	rc
Assignee:	Ian McLeod
QA Contact:	Dave Johnson
Docs Contact:
URL:	na
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2012-01-09 23:22 UTC by wes hayutin
Modified:	2020-03-27 19:35 UTC (History)
CC List:	9 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2020-03-27 19:35:38 UTC
Flags:	hbrock: needinfo+

Attachments	(Terms of Use)
screenshot for point 2 in comment 3 (20.15 KB, image/png) 2012-01-25 09:04 UTC, Martin Kočí	no flags	Details
View All Add an attachment (proposed patch, testcase, etc.)

Description wes hayutin 2012-01-09 23:22:13 UTC

Description of problem:
[root@qeblade31 ~]# cat /etc/oz/oz.cfg 
[paths]
output_dir = /var/lib/libvirt/images
data_dir = /var/lib/oz
screenshot_dir = .

[libvirt]
uri = qemu:///system
# type = kvm
# bridge_name = virbr0

[cache]
original_media = yes
modified_media = yes
jeos = yes

[root@qeblade31 ~]# virsh list
 Id Name                 State
----------------------------------
  6 RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad running

012-01-09 18:14:01,326 DEBUG imgfac.BuildJob.BuildJob pid(16175) Message: Builder (27571ee2-1703-4a3d-be24-8f69ab613cad) changed status from NEW to BUILDING
2012-01-09 18:14:01,328 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(16175) Message: Found neither a call-time config nor a config file - doing nothing
2012-01-09 18:14:01,344 DEBUG paste.httpserver.ThreadPool pid(16175) Message: Added task (0 tasks queued)
2012-01-09 18:14:01,352 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: libvirt bridge name is virbr0
2012-01-09 18:14:01,459 DEBUG paste.httpserver.ThreadPool pid(16175) Message: Added task (0 tasks queued)
2012-01-09 18:14:01,586 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Libvirt type is kvm
2012-01-09 18:14:01,958 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Original URL http://download.devel.redhat.com/released/RHEL-6-Server/6.1/x86_64/os/ resolved to http://download.devel.redhat.com/released/RHEL-6-Server/6.1/x86_64/os/
2012-01-09 18:14:01,958 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Name: RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad, UUID: a99fb9d4-7d35-44b7-acfe-fc6eec023ae7
2012-01-09 18:14:01,958 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: MAC: 52:54:00:e9:1c:2c, distro: RHEL-6
2012-01-09 18:14:01,958 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: update: 1, arch: x86_64, diskimage: /var/lib/imagefactory/images/RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad.dsk
2012-01-09 18:14:01,959 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: nicmodel: virtio, clockoffset: utc
2012-01-09 18:14:01,959 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: mousetype: ps2, disk_bus: virtio, disk_dev: vda
2012-01-09 18:14:01,959 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: icicletmp: /var/lib/oz/icicletmp/RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad, listen_port: 1280
2012-01-09 18:14:01,959 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Original ISO path: /var/lib/oz/isos/RHEL-61x86_64-url.iso
2012-01-09 18:14:01,959 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Modified ISO cache: /var/lib/oz/isos/RHEL-61x86_64-url-oz.iso
2012-01-09 18:14:01,959 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Output ISO path: /var/lib/imagefactory/images/RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad-url-oz.iso
2012-01-09 18:14:01,960 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: ISO content path: /var/lib/oz/isocontent/RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad-url
2012-01-09 18:14:01,960 INFO oz.Guest.RHEL6Guest pid(16175) Message: Cleaning up guest named RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad
2012-01-09 18:14:01,962 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Installtype is URL, trying to do direct kernel boot
2012-01-09 18:14:01,983 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Going to write treeinfo to /var/lib/oz/icicletmp/RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad/treeinfo
2012-01-09 18:14:01,984 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Trying to get treeinfo from http://download.devel.redhat.com/released/RHEL-6-Server/6.1/x86_64/os//.treeinfo
2012-01-09 18:14:02,170 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Got treeinfo, parsing
2012-01-09 18:14:02,171 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Returning kernel images/pxeboot/vmlinuz and initrd images/pxeboot/initrd.img
2012-01-09 18:14:02,171 INFO oz.Guest.RHEL6Guest pid(16175) Message: Fetching the original media
2012-01-09 18:14:02,171 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Attempting to get the lock for /var/lib/oz/kernels/RHEL-61x86_64-kernel
2012-01-09 18:14:02,171 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Got the lock, doing the download
2012-01-09 18:14:02,315 INFO oz.Guest.RHEL6Guest pid(16175) Message: Original install media available, using cached version
2012-01-09 18:14:02,315 INFO oz.Guest.RHEL6Guest pid(16175) Message: Fetching the original media
2012-01-09 18:14:02,315 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Attempting to get the lock for /var/lib/oz/kernels/RHEL-61x86_64-ramdisk
2012-01-09 18:14:02,315 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Got the lock, doing the download
2012-01-09 18:14:02,406 DEBUG paste.httpserver.ThreadPool pid(16175) Message: Added task (0 tasks queued)
2012-01-09 18:14:02,435 INFO oz.Guest.RHEL6Guest pid(16175) Message: Original install media available, using cached version
2012-01-09 18:14:02,448 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Putting the kickstart in place
2012-01-09 18:14:02,449 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Writing cpio to /var/lib/oz/icicletmp/RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad/extra.cpio
2012-01-09 18:14:02,530 DEBUG imgfac.BuildJob.BuildJob pid(16175) Message: Builder (27571ee2-1703-4a3d-be24-8f69ab613cad) changed percent complete from 0 to 10
2012-01-09 18:14:02,531 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(16175) Message: Doing base install via Oz
2012-01-09 18:14:02,531 INFO oz.Guest.RHEL6Guest pid(16175) Message: Found cached JEOS, using it
2012-01-09 18:14:37,400 INFO oz.Guest.RHEL6Guest pid(16175) Message: Generate XML for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad with bootdev hd
2012-01-09 18:14:37,401 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Generated XML:
<?xml version="1.0"?>
<domain type="kvm">
  <name>RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad</name>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <uuid>a99fb9d4-7d35-44b7-acfe-fc6eec023ae7</uuid>
  <clock offset="utc"/>
  <vcpu>1</vcpu>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <os>
    <type>hvm</type>
    <boot dev="hd"/>
  </os>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <console device="pty"/>
    <graphics port="-1" type="vnc"/>
    <interface type="bridge">
      <source bridge="virbr0"/>
      <mac address="52:54:00:e9:1c:2c"/>
      <model type="virtio"/>
    </interface>
    <input bus="ps2" type="mouse"/>
    <console type="pty">
      <target port="0"/>
    </console>
    <serial type="tcp">
      <source mode="bind" host="127.0.0.1" service="1280"/>
      <protocol type="raw"/>
      <target port="1"/>
    </serial>
    <disk device="disk" type="file">
      <target dev="vda" bus="virtio"/>
      <source file="/var/lib/imagefactory/images/base-image-27571ee2-1703-4a3d-be24-8f69ab613cad.dsk"/>
    </disk>
  </devices>
</domain>

2012-01-09 18:14:37,401 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(16175) Message: Base install complete - Doing customization and ICICLE generation
2012-01-09 18:14:37,401 DEBUG imgfac.BuildJob.BuildJob pid(16175) Message: Builder (27571ee2-1703-4a3d-be24-8f69ab613cad) changed percent complete from 10 to 30
2012-01-09 18:14:37,401 INFO oz.Guest.RHEL6Guest pid(16175) Message: Customizing image
2012-01-09 18:14:37,402 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Generated XML:
<?xml version="1.0"?>
<domain type="kvm">
  <name>RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad</name>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <uuid>a99fb9d4-7d35-44b7-acfe-fc6eec023ae7</uuid>
  <clock offset="utc"/>
  <vcpu>1</vcpu>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <os>
    <type>hvm</type>
    <boot dev="hd"/>
  </os>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <console device="pty"/>
    <graphics port="-1" type="vnc"/>
    <interface type="bridge">
      <source bridge="virbr0"/>
      <mac address="52:54:00:e9:1c:2c"/>
      <model type="virtio"/>
    </interface>
    <input bus="ps2" type="mouse"/>
    <console type="pty">
      <target port="0"/>
    </console>
    
    <disk device="disk" type="file">
      <target dev="vda" bus="virtio"/>
      <source file="/var/lib/imagefactory/images/base-image-27571ee2-1703-4a3d-be24-8f69ab613cad.dsk"/>
    </disk>
  <serial type="tcp"><source mode="bind" host="127.0.0.1" service="1280"/><protocol type="raw"/><target port="1"/></serial></devices>
</domain>

2012-01-09 18:14:37,402 INFO oz.Guest.RHEL6Guest pid(16175) Message: Collection Setup
2012-01-09 18:14:37,403 INFO oz.Guest.RHEL6Guest pid(16175) Message: Setting up guestfs handle for RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad
2012-01-09 18:14:37,403 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Adding disk image /var/lib/imagefactory/images/base-image-27571ee2-1703-4a3d-be24-8f69ab613cad.dsk
2012-01-09 18:14:37,403 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Launching guestfs
2012-01-09 18:14:40,910 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Inspecting guest OS
2012-01-09 18:14:41,885 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Getting mountpoints
2012-01-09 18:14:41,886 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Root device: /dev/VolGroup00/LogVol00
2012-01-09 18:14:42,071 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Step 1: Uploading ssh keys
2012-01-09 18:14:42,084 INFO oz.Guest.RHEL6Guest pid(16175) Message: Generating new openssh key
2012-01-09 18:14:42,085 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Step 2: setup sshd
2012-01-09 18:14:42,199 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Step 3: Open up the firewall
2012-01-09 18:14:42,200 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Step 4: Guest announcement
2012-01-09 18:14:42,229 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Step 5: Set SELinux to permissive mode
2012-01-09 18:14:42,251 INFO oz.Guest.RHEL6Guest pid(16175) Message: Cleaning up guestfs handle for RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad
2012-01-09 18:14:42,251 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Syncing
2012-01-09 18:14:42,508 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Unmounting all
2012-01-09 18:14:43,129 INFO oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot
2012-01-09 18:14:43,130 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 300/300
2012-01-09 18:14:53,184 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 290/300
2012-01-09 18:15:03,313 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 280/300
2012-01-09 18:15:14,159 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 270/300
2012-01-09 18:15:24,280 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 260/300
2012-01-09 18:15:34,362 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 250/300
2012-01-09 18:15:44,428 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 240/300
2012-01-09 18:15:54,484 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 230/300
2012-01-09 18:16:03,280 DEBUG paste.httpserver.ThreadPool pid(16175) Message: Added task (0 tasks queued)
2012-01-09 18:16:04,557 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 220/300
2012-01-09 18:16:14,590 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 210/300
2012-01-09 18:16:24,683 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 200/300
2012-01-09 18:16:34,841 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 190/300
2012-01-09 18:16:44,959 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 180/300
2012-01-09 18:16:55,046 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 170/300
2012-01-09 18:17:05,113 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 160/300
2012-01-09 18:17:15,192 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 150/300
2012-01-09 18:17:25,305 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 140/300
2012-01-09 18:17:35,410 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 130/300
2012-01-09 18:17:45,473 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 120/300
2012-01-09 18:17:55,579 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 110/300
2012-01-09 18:18:03,948 DEBUG paste.httpserver.ThreadPool pid(16175) Message: Added task (0 tasks queued)
2012-01-09 18:18:05,677 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 100/300
2012-01-09 18:18:15,771 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 90/300
2012-01-09 18:18:25,894 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 80/300
2012-01-09 18:18:35,991 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 70/300
2012-01-09 18:18:46,078 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 60/300
2012-01-09 18:18:56,139 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 50/300
2012-01-09 18:19:06,270 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 40/300
2012-01-09 18:19:16,377 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 30/300
2012-01-09 18:19:26,473 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 20/300
2012-01-09 18:19:36,562 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Waiting for guest RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad to boot, 10/300
2012-01-09 18:19:47,060 INFO oz.Guest.RHEL6Guest pid(16175) Message: Collection Teardown
2012-01-09 18:19:47,062 INFO oz.Guest.RHEL6Guest pid(16175) Message: Setting up guestfs handle for RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad
2012-01-09 18:19:47,062 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Adding disk image /var/lib/imagefactory/images/base-image-27571ee2-1703-4a3d-be24-8f69ab613cad.dsk
2012-01-09 18:19:47,062 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Launching guestfs
2012-01-09 18:19:50,634 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Inspecting guest OS
2012-01-09 18:19:53,017 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Getting mountpoints
2012-01-09 18:19:53,017 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Root device: /dev/VolGroup00/LogVol00
2012-01-09 18:19:53,152 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Teardown step 1
2012-01-09 18:19:53,152 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Resetting authorized_keys
2012-01-09 18:19:53,187 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Teardown step 2
2012-01-09 18:19:53,187 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Resetting sshd_config
2012-01-09 18:19:53,233 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Resetting sshd service
2012-01-09 18:19:53,282 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Teardown step 3
2012-01-09 18:19:53,282 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Resetting iptables rules
2012-01-09 18:19:53,283 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Teardown step 4
2012-01-09 18:19:53,283 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Resetting announcement to host
2012-01-09 18:19:53,294 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Removing reportip
2012-01-09 18:19:53,295 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Resetting crond service
2012-01-09 18:19:53,304 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Teardown step 5
2012-01-09 18:19:53,318 INFO oz.Guest.RHEL6Guest pid(16175) Message: Cleaning up guestfs handle for RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad
2012-01-09 18:19:53,319 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Syncing
2012-01-09 18:19:53,514 DEBUG oz.Guest.RHEL6Guest pid(16175) Message: Unmounting all
2012-01-09 18:19:53,659 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(16175) Message: Exception caught in ImageFactory
2012-01-09 18:19:53,738 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(16175) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 115, in build_upload
    self.output_descriptor = self.guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1177, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1150, in _internal_customize
    guestaddr = self._wait_for_guest_boot(libvirt_dom)
  File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 1020, in _wait_for_guest_boot
    raise oz.OzException.OzException("Timed out waiting for guest to boot")
OzException: Timed out waiting for guest to boot

2012-01-09 18:19:53,738 INFO oz.Guest.RHEL6Guest pid(16175) Message: Cleaning up guest named RHEL6_VMWareTools-27571ee2-1703-4a3d-be24-8f69ab613cad
2012-01-09 18:19:53,888 DEBUG imgfac.BuildJob.BuildJob pid(16175) Message: Builder (27571ee2-1703-4a3d-be24-8f69ab613cad) changed status from BUILDING to FAILED
2012-01-09 18:19:53,888 DEBUG imgfac.BuildJob.BuildJob pid(16175) Message: 27571ee2-1703-4a3d-be24-8f69ab613cad for vsphere about to exit local queue...
2012-01-09 18:19:53,889 DEBUG imgfac.ReservationManager.ReservationManager pid(16175) Message: EXITING queue: (local)
2012-01-09 18:19:53,889 DEBUG imgfac.ReservationManager.ReservationManager pid(16175) Message: SUCCESS EXITING queue: (local)
2012-01-09 18:19:53,889 INFO oz.Guest.RHEL6Guest pid(16175) Message: Cleaning up after install
2012-01-09 18:19:53,897 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(16175) Message: Exception caught in ImageFactory
2012-01-09 18:19:53,898 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(16175) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 49, in build_image
    self.build_upload(build_id)
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 115, in build_upload
    self.output_descriptor = self.guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1177, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1150, in _internal_customize
    guestaddr = self._wait_for_guest_boot(libvirt_dom)
  File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 1020, in _wait_for_guest_boot
    raise oz.OzException.OzException("Timed out waiting for guest to boot")
OzException: Timed out waiting for guest to boot

2012-01-09 18:19:53,898 DEBUG imgfac.BuildJob.BuildJob pid(16175) Message: Builder (27571ee2-1703-4a3d-be24-8f69ab613cad) changed status from FAILED to FAILED
2012-01-09 18:20:04,628 DEBUG paste.httpserver.ThreadPool

[root@qeblade31 jeos]# rpm -qa | grep factory
imagefactory-jeosconf-ec2-rhel-1.0.0rc2-1.el6.noarch
imagefactory-1.0.0rc2-1.el6.noarch
imagefactory-jeosconf-ec2-fedora-1.0.0rc2-1.el6.noarch
rubygem-imagefactory-console-0.4.0-1.el6.noarch
[root@qeblade31 jeos]# rpm -qa | grep oz
oz-0.7.9-6.el6.noarch
[root@qeblade31 jeos]#

Comment 1 wes hayutin 2012-01-09 23:57:43 UTC

k.. narrowing it down..
[root@qeblade31 ~]# cat /etc/oz/oz.cfg 
[paths]
output_dir = /var/lib/libvirt/images
data_dir = /var/lib/oz
screenshot_dir = .

[libvirt]
uri = qemu:///system
# type = kvm
# bridge_name = virbr0

[cache]
original_media = yes
modified_media = yes
jeos = no
[root@qeblade31 ~]# 

restarted services.. same error

Comment 2 Ian McLeod 2012-01-11 14:37:59 UTC

Changing component to Oz, since that's where all the action is on this.

So, this is really exposing two things.  Only one of which is clearly a bug:

1) Cached RHEL6 JEOS images don't seem to work.  From Oz's perspective, they never check in after being booted and, as a result, cannot be modified to allow further customization.

2) Oz always uses an existing cached copy of install media, modified install media and the JEOS image, regardless of the settings in the [cached] stanza of the config file.

#2 is a long standing behaviour and can be overridden with a "force" option in most of Oz's methods.

#1 is a real problem.  If we can't get to the bottom of it quickly we may want to consider disabling JEOS caching entirely when using Oz through the Factory.

Comment 3 Chris Lalancette 2012-01-11 15:38:07 UTC

Hm.  It looks like Oz is doing most of the correct things here, in that it finds the cached JEOS, modifies it, and then launches it.  I will admit that I haven't extensively tested cached JEOS with customization, so that may well be the issue.  The things that would be helpful here:

1)  "virsh dumpxml <foo>" on the guest that Imagefactory/Oz launches to do the customization
2)  Logging into the guest while Imagefactory is waiting around for it.  I usually do this with "virt-viewer <guestname>" on the build machine, and then log into the console.  In particular, I am interested in the contents of /etc/cron.d/announce and /root/reportip

Comment 4 wes hayutin 2012-01-12 16:31:32 UTC

adding to ce-sprint

Comment 5 wes hayutin 2012-01-12 16:38:48 UTC

removing ce-sprint-next tracker

Comment 6 wes hayutin 2012-01-24 10:53:44 UTC

Martin, please answer Chris's question asap

Comment 7 Martin Kočí 2012-01-24 12:12:44 UTC

OK, I will definitely working on this later today if that is OK ?
Currently I'm quite busy with hiring stuff :(...

Comment 8 Martin Kočí 2012-01-25 09:04:58 UTC

Created attachment 557405 [details]
screenshot for point 2 in comment 3

Comment 9 Martin Kočí 2012-01-25 09:07:54 UTC

(In reply to comment #3)
ad 1) 

# virsh dumpxml factory-build-08aa42fe-29a5-43fe-9cdf-9604f59c2d11
<domain type='kvm' id='5'>
  <name>factory-build-08aa42fe-29a5-43fe-9cdf-9604f59c2d11</name>
  <uuid>7966acc2-e6bd-4a8f-822d-741c54443a57</uuid>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.2.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/imagefactory/images/base-image-08aa42fe-29a5-43fe-9cdf-9604f59c2d11.dsk'/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <interface type='bridge'>
      <mac address='52:54:00:c5:86:da'/>
      <source bridge='virbr0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='tcp'>
      <source mode='bind' host='127.0.0.1' service='27975'/>
      <protocol type='raw'/>
      <target port='1'/>
      <alias name='serial0'/>
    </serial>
    <console type='tcp'>
      <source mode='bind' host='127.0.0.1' service='27975'/>
      <protocol type='raw'/>
      <target type='serial' port='1'/>
      <alias name='serial0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c346,c592</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c346,c592</imagelabel>
  </seclabel>
</domain>

ad 2) see the attached screenshot from the session

Comment 10 Hugh Brock 2012-02-27 17:16:50 UTC

Ian, what's the resolution on this, fix or not? Please either dev ack and fix, or push to 1.1.0.

Comment 11 Ian McLeod 2012-02-27 18:44:52 UTC

This touches on at least two issues and an RFE.  None of them are going to be resolved in the 1.0 timeframe.  I'll push this to 1.1 but then ask that we either clearly specify which issue is being discussed in this particular BZ _or_ close this one and start tracking these things separately.

Issues are:

1) Oz uses cached JEOS if it exists, even if caching jeos images has since been turned off.

2) Caching of RHEL6 JEOS images doesn't work.

RFE is:

1) Use caching or other approaches to minimize duplicated work when building the same OS or same Template for multiple cloud types.

Comment 12 Mike Orazi 2012-08-09 15:30:40 UTC

This is a 2.0 feature.

Note You need to log in before you can comment on or make changes to this bug.