Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 835005

Summary: aeolus upgrade during push causes deltacloud segfault
Product: [Retired] CloudForms Cloud Engine Reporter: Rehana <redakkan>
Component: aeolus-conductorAssignee: Angus Thomas <athomas>
Status: CLOSED NOTABUG QA Contact: Rehana <aeolus-qa-list>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 1.0.0CC: akarol, dajohnso, jeckersb
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-28 14:47:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
deltacloud.log none

Description Rehana 2012-06-25 08:38:02 UTC
Created attachment 594131 [details]
deltacloud.log

Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:
Not very frequently

Steps to Reproduce:
1.Installed aeolus 1.0.0 
2.imported ec2 image
3.launched ec2 instance
4.when vpshere image push is in progress, i tried updated the aeolus from 1.0.0 to 1.0.1(add a new repo url,executed yum update and /usr/share/aeolus-conductor/script/upgrade)

  
Actual results:
Observed that the instance got disappeard, and all the provider went not available mode in conductor.(PFA:deltacloud.log)(hostname:hp-sl2x160zg6-01.rhts.eng.bos.redhat.com)

Expected results:
all the instance and provider should be avaiable after the update 

Additional info:

Tried to reproduce this observation on new host (hp-sl2x170zg6-01.rhts.eng.bos.redhat.com), the running instace was not affected, but the vsphere provider went to unavailable mode with the below error in dbomatic.log

2012-06-25 03:47:58 INFO Deltacloud realms refresh started
2012-06-25 03:47:59 ERROR Error connecting to framework: 500 : method `hash' called on terminated object (0x7fc8c7a3de68)
2012-06-25 03:47:59 ERROR Backtrace:
/usr/lib/ruby/1.8/set.rb:201:in `[]='
/usr/lib/ruby/1.8/set.rb:201:in `add'
/usr/lib/ruby/1.8/set.rb:263:in `merge'
/usr/lib/ruby/1.8/set.rb:263:in `each'
/usr/lib/ruby/1.8/set.rb:263:in `merge'
/usr/lib/ruby/1.8/set.rb:75:in `initialize'
/usr/lib/ruby/gems/1.8/gems/rbvmomi-1.2.3/lib/rbvmomi/type_loader.rb:27:in `new'
/usr/lib/ruby/gems/1.8/gems/rbvmomi-1.2.3/lib/rbvmomi/type_loader.rb:27:in `init'
/usr/lib/ruby/gems/1.8/gems/rbvmomi-1.2.3/lib/rbvmomi/connection.rb:248:in `load_vmodl'
/usr/lib/ruby/gems/1.8/gems/rbvmomi-1.2.3/lib/rbvmomi/vim.rb:85
2012-06-25 03:47:59 WARN vsphere-default provider's availability changed to false
2012-06-25 03:48:00 INFO Deltacloud realms refresh completed
2012-06-25 03:48:00 INFO Deltacloud refresh completed
2012-06-25 03:49:00 INFO Deltacloud refresh started

on

 rpm -qa | grep aeolus
aeolus-conductor-doc-0.8.31-1.el6cf.noarch
rubygem-aeolus-cli-0.3.3-2.el6_2.noarch
aeolus-conductor-0.8.31-1.el6cf.noarch
rubygem-aeolus-image-0.3.0-12.el6.noarch
aeolus-conductor-daemons-0.8.31-1.el6cf.noarch
aeolus-configure-2.5.9-1.el6cf.noarch
aeolus-all-0.8.31-1.el6cf.noarch

[root@hp-sl2x160zg6-01 ~]# rpm -qa | grep delta*
deltacloud-core-0.5.0-10.el6_2.noarch
deltacloud-core-rhevm-0.5.0-10.el6_2.noarch
deltacloud-core-vsphere-0.5.0-10.el6_2.noarch
rubygem-deltacloud-client-0.5.0-2.el6.noarch
deltacloud-core-ec2-0.5.0-10.el6_2.noarch

Comment 1 Aziza Karol 2012-06-25 11:15:35 UTC
I am also facing the same issue.
All applications went in stopped state and providers are unavailable after upgrading from 1.0.0 to 1.0.1

Comment 2 Aziza Karol 2012-06-25 11:47:08 UTC
deltacloud-core goes dead after and upgrade.

#aeolus-check-services 
Checking mongod ...
 Success: mongod (pid 21437) is running...

Checking iwhd ...
 Success: iwhd (pid  21452) is running...

Checking postgresql ...
 Success: postmaster (pid  21394) is running...

Checking httpd ...
 Success: httpd (pid  21506) is running...

Checking deltacloud-core ...
 FAILURE: deltacloud-core dead but pid file exists

Checking libvirtd ...
 Success: libvirtd (pid  21558) is running...

Checking aeolus-conductor ...
 Success: thin (pid  21606) is running...

Checking conductor-dbomatic ...
 Success: dbomatic (pid  21664) is running...

Checking imagefactory ...
 Success: imagefactory (pid  21681) is running...

Checking ntpd ...
 Success: ntpd (pid  21693) is running...



#ps  -ef | grep 3002
root     24139  6484  0 07:43 pts/0    00:00:00 grep 3002

Comment 3 Dave Johnson 2012-06-25 17:02:39 UTC
Changing the summary, upgrades work when no imagefactory builds/pushes are in progress...

definitly need to add some defensive code for this moving forward

Comment 4 John Eckersberg 2012-06-27 17:39:44 UTC
What version of ruby?

Comment 5 Rehana 2012-06-28 14:47:46 UTC
so, i was NOT able to reproduce the same with the correct rhel6.2-Z repo. 

repo used is

cat /etc/yum.repos.d/aeolus-V1_0_1.repo 
[aeolus-brew-puddle]
name=Aeolus RPMs and Dependencies, in brew
baseurl=http://download.lab.bos.redhat.com/rel-eng/CloudForms/1.0.1/latest/el6-ce/x86_64/
enabled=1
#skip_if_unavailable=1
gpgcheck=0

[aeolus-tools-brew-puddle]
name=Aeolus Tools RPMs and Dependencies, in brew
baseurl=http://download.lab.bos.redhat.com/rel-eng/CloudForms/1.0.1/latest/el6-tools/x86_64/
enabled=1
gpgcheck=0

[rhel62z-updates]
name=rhel62z-updates
baseurl=http://download.lab.bos.redhat.com/brewroot/repos/RHEL-6.2-Z-build/latest/x86_64/
enabled=1
gpgcheck=0


The ruby version is :

ruby-1.8.7.352-7.el6_2.x86_64

hence moving the bug to Closed.