Bug 761254

Summary:	imagefactory only allows one image to be built
Product:	[Retired] CloudForms Cloud Engine	Reporter:	Dave Johnson <dajohnso>
Component:	imagefactory	Assignee:	Steve Loranz <sloranz>
Status:	CLOSED ERRATA	QA Contact:	Martin Kočí <mkoci>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	1.0.0	CC:	akarol, brad, dajohnso, deltacloud-maint, dgao, jrd, nobody, sloranz, ssachdev, whayutin
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-05-15 20:19:05 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Dave Johnson 2011-12-07 22:16:46 UTC

Description of problem:
===========================
With the latest imagefactory and conductor rpms, you can only build a single image.  Subsequent images result in the following being added to the /var/log/imagefactory.log without anything ever being built

imagefactory-1.0.0rc1-1.el6.noarch

2011-12-07 16:44:36,369 DEBUG paste.httpserver.ThreadPool pid(2421) Message: Added task (0 tasks queued)
2011-12-07 16:44:36,413 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(2421) Message: Created Image Warehouse instance http://localhost:9090 - buckets(target_images, templates, icicles, provider_images)
2011-12-07 16:44:36,419 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(2421) Message: Image Warehouse returned status (500) with message: 
2011-12-07 16:44:36,419 INFO imgfac.ImageWarehouse.ImageWarehouse pid(2421) Message: Creating a bucket returned status (500), .
2011-12-07 16:44:36,423 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(2421) Message: Setting metadata ({'timestamp': '1323294276.42', 'image': '677cc554-211c-11e1-8509-5cf3fc1c84a8', 'object_type': 'build', 'uuid': 'ad30fe7c-ead6-43f3-a8aa-b027da62bf92'}) for http://localhost:9090/builds/ad30fe7c-ead6-43f3-a8aa-b027da62bf92
2011-12-07 16:44:36,592 DEBUG imgfac.ImageWarehouse.ImageWarehouse pid(2421) Message: Created Image Warehouse instance http://localhost:9090 - buckets(target_images, templates, icicles, provider_images)
2011-12-07 16:44:36,593 DEBUG imgfac.builders.BaseBuilder.RHEL6_rhevm_Builder pid(2421) Message: build_upload() called on Fedora_rhevm_Builder...
2011-12-07 16:44:36,593 DEBUG imgfac.builders.BaseBuilder.RHEL6_rhevm_Builder pid(2421) Message: Building for target rhevm with warehouse config http://localhost:9090/
2011-12-07 16:44:36,593 DEBUG imgfac.ReservationManager.ReservationManager pid(2421) Message: ENTERING queue: (local)


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Steve Loranz 2011-12-08 15:33:58 UTC

Are you running on RHEL 6.2?

Comment 2 Dave Johnson 2011-12-08 16:14:47 UTC

Yes, I am running on RHEL 6.2.

When I hit this, I reached out to Ian who already knew of the issue.  He asked that I restart imagefactory and try again which up to this point, seems to be working just fine.

Comment 3 Dave Johnson 2011-12-12 15:18:57 UTC

FYI, still seeing this on this morning's latest rpms on top of a fresh RHEL 6.2 install.  This is very sporadic, I do hit this daily but there doesn't seem to be any definitive reproduction steps, smells like a race condition.

Comment 4 Steve Loranz 2011-12-16 21:10:33 UTC

What is the imagefactory RPM version?

Comment 5 wes hayutin 2012-01-03 17:41:39 UTC

adding ce-sprint-next bugs to ce-sprint

Comment 6 Steve Loranz 2012-01-03 18:56:25 UTC

Please check that commit 5d7829b5c0ca8203ae74b3454b8566bbfafbe95a doesn't fix this.

https://github.com/aeolusproject/imagefactory/commit/5d7829b5c0ca8203ae74b3454b8566bbfafbe95

Comment 7 Martin Kočí 2012-01-10 12:44:44 UTC

Just FYI that I'm no longer able to reproduce this issue on RHEL62 nor RHEL61 with this releases: 

#rpm -qa |grep 'aeolus\|imagefactory-\|oz-\|iwhd'
rubygem-imagefactory-console-0.5.0-4.20110824113238gitd9debef.el6.noarch
aeolus-configure-2.6.0-0.20120105171403gitf0043e8.el6.noarch
aeolus-conductor-daemons-0.9.0-0.20120105223209git5e068fe.el6.noarch
imagefactory-jeosconf-ec2-rhel-1.0.0rc1-1.el6.noarch
rubygem-aeolus-cli-0.4.0-0.20120105171410git2c6b457.el6.noarch
aeolus-conductor-0.9.0-0.20120105223209git5e068fe.el6.noarch
iwhd-1.1-2.el6.x86_64
imagefactory-jeosconf-ec2-fedora-1.0.0rc1-1.el6.noarch
rubygem-arel-2.0.10-0.aeolus.el6.noarch
rubygem-rack-mount-0.7.1-3.aeolus.el6.noarch
rubygem-ZenTest-4.3.3-2.aeolus.el6.noarch
aeolus-conductor-doc-0.9.0-0.20120105223209git5e068fe.el6.noarch
oz-0.8.0-0.20111219203204git5775e9d.el6.noarch
aeolus-all-0.9.0-0.20120105223209git5e068fe.el6.noarch
imagefactory-1.0.0rc1-1.el6.noarch
rubygem-aeolus-image-0.4.0-0.20120105171419git8108b42.el6.noarch

I will keep trying to reproduce it.

Comment 8 Martin Kočí 2012-01-13 13:43:11 UTC

Test passed on RHEL61 and F16
#  rpm -qa|grep 'aeolus\|imagefactory-\|oz-\|iwhd'
imagefactory-1.0.0rc1-1.fc16.noarch
aeolus-configure-2.6.0-0.20120112133633git8701011.fc16.noarch
rubygem-aeolus-cli-0.4.0-0.20120111093540git5ec8dc7.fc16.noarch
aeolus-conductor-daemons-0.9.0-0.20120111153517gite55da85.fc16.noarch
aeolus-all-0.9.0-0.20120111153517gite55da85.fc16.noarch
rubygem-aeolus-image-0.4.0-0.20120111093551git67fdf6f.fc16.noarch
aeolus-conductor-doc-0.9.0-0.20120111153517gite55da85.fc16.noarch
imagefactory-jeosconf-ec2-fedora-1.0.0rc1-1.fc16.noarch
rubygem-imagefactory-console-0.4.0-6.fc16.noarch
oz-0.8.0-0.20111219203204git5775e9d.fc16.noarch
aeolus-conductor-0.9.0-0.20120111153517gite55da85.fc16.noarch
iwhd-1.2-1.fc16.x86_64
imagefactory-jeosconf-ec2-rhel-1.0.0rc1-1.fc16.noarch
-----
# rpm -qa|grep 'aeolus\|imagefactory-\|oz-\|iwhd'
aeolus-configure-2.6.0-0.20120111093533gitf3d033e.el6.noarch
aeolus-conductor-0.9.0-0.20120111153517gite55da85.el6.noarch
oz-0.8.0-0.20111219203204git5775e9d.el6.noarch
imagefactory-jeosconf-ec2-rhel-1.0.0rc1-1.el6.noarch
rubygem-arel-2.0.10-0.aeolus.el6.noarch
rubygem-rack-mount-0.7.1-3.aeolus.el6.noarch
rubygem-ZenTest-4.3.3-2.aeolus.el6.noarch
rubygem-imagefactory-console-0.5.0-4.20110824113238gitd9debef.el6.noarch
rubygem-aeolus-cli-0.4.0-0.20120111093540git5ec8dc7.el6.noarch
imagefactory-jeosconf-ec2-fedora-1.0.0rc1-1.el6.noarch
rubygem-aeolus-image-0.4.0-0.20120111093551git67fdf6f.el6.noarch
iwhd-1.1-2.el6.x86_64
aeolus-all-0.9.0-0.20120111153517gite55da85.el6.noarch
imagefactory-1.0.0rc1-1.el6.noarch
aeolus-conductor-doc-0.9.0-0.20120111153517gite55da85.el6.noarch
aeolus-conductor-daemons-0.9.0-0.20120111153517gite55da85.el6.noarch
and RHEL62
rpm -qa + grep 'aeolus\|imagefactory-\|oz-\|iwhd'
rubygem-imagefactory-console-0.5.0-4.20110824113238gitd9debef.el6.noarch aeolus-conductor-daemons-0.9.0-0.20120111153517gite55da85.el6.noarch aeolus-conductor-doc-0.9.0-0.20120111153517gite55da85.el6.noarch aeolus-configure-2.6.0-0.20120112133633git8701011.el6.noarch imagefactory-jeosconf-ec2-rhel-1.0.0rc2.1-1.el6.noarch rubygem-aeolus-image-0.4.0-0.20120111093551git67fdf6f.el6.noarch rubygem-arel-2.0.10-0.aeolus.el6.noarch rubygem-ZenTest-4.3.3-2.aeolus.el6.noarch oz-0.8.0-0.20111219203204git5775e9d.el6.noarch iwhd-1.2-3.el6.x86_64 imagefactory-jeosconf-ec2-fedora-1.0.0rc2.1-1.el6.noarch rubygem-aeolus-cli-0.4.0-0.20120111093540git5ec8dc7.el6.noarch imagefactory-1.0.0rc2.1-1.el6.noarch aeolus-conductor-0.9.0-0.20120111153517gite55da85.el6.noarch aeolus-all-0.9.0-0.20120111153517gite55da85.el6.noarch

Comment 10 errata-xmlrpc 2012-05-15 20:19:05 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2012-0588.html