Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1109258

Summary: Rubygem-Staypuft: Can't delete hosts after deployment.
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: rubygem-staypuftAssignee: Jason E. Rist <jrist>
Status: CLOSED ERRATA QA Contact: Alexander Chuzhoy <sasha>
Severity: high Docs Contact:
Priority: urgent    
Version: 5.0 (RHEL 6)CC: aberezin, mburns, mtaylor, ohadlevy, pchalupa, rhos-maint, sclewis, yeylon
Target Milestone: ga   
Target Release: 5.0 (RHEL 7)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: ruby193-rubygem-staypuft-0.1.3.el6ost Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-04 18:34:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
production.log from the staypuft machine. none

Description Alexander Chuzhoy 2014-06-13 13:52:28 UTC
Environment: 
puddle:  2014-06-10.3
foreman-installer-staypuft-0.0.15-1.el6ost.noarch
ruby193-rubygem-staypuft-0.1.1-1.el6ost.noarch
puppet-3.3.2-2.el6.noarch
puppet-server-3.3.2-2.el6.noarch
openstack-puppet-modules-2014.1-14.1.el6ost.noarch


Steps to reproduce:
1. Create and successfuly deploy "Distributed Nova networking" deployment with 1 controller and 3 Nova hosts (total 4 VMs) with staypuft.
2. Try to delete the hosts from foreman. 


Results:
ERROR: update or delete on table "hosts" violates foreign key constraint "tokens_host_id_fk" on table "tokens" DETAIL: Key (id)=(2) is still referenced from table "tokens".

Comment 1 Omri Hochman 2014-06-13 14:02:40 UTC
Rubygem-Staypuft: Can't delete hosts after deployment.

Comment 3 Alexander Chuzhoy 2014-06-13 14:07:23 UTC
Created attachment 908591 [details]
production.log from the staypuft machine.

Adding the production log from the time the error was seen.

Comment 4 Alexander Chuzhoy 2014-06-13 14:08:14 UTC
Adding the Full trace:

PGError
ERROR: update or delete on table "hosts" violates foreign key constraint "tokens_host_id_fk" on table "tokens" DETAIL: Key (id)=(2) is still referenced from table "tokens".
app/controllers/hosts_controller.rb:125:in `destroy'
app/models/concerns/foreman/thread_session.rb:33:in `clear_thread'
lib/middleware/catch_json_parse_errors.rb:9:in `call'

Comment 5 Mike Burns 2014-06-13 14:14:45 UTC
Dominic/Ohad -- any ideas with this?  We have workarounds using ruby console, but that's not a very good solution for customers.

Comment 6 Dominic Cleal 2014-06-13 14:56:45 UTC
It appears that multiple build tokens are being created, while only one should ever exist at most.  This can be seen in the console:

irb(main):001:0> Token.where(:host_id => Host.find_by_name("525400702875.lab.eng.rdu2.redhat.com"))
=> [#<Token id: 16, value: "8a5444f2-57d7-4520-bcf1-f989273e8e3b", expires: "2014-06-12 21:35:08", host_id: 5>, #<Token id: 1, value: "9a7d17d4-13c8-4f55-8966-3f990fc250ab", expires: "2014-06-12 18:01:42", host_id: 5>]

I believe staypuft is deliberately creating them on one of its deployment steps without expiring the old one first.

app/lib/actions/staypuft/host/build.rb
28:          host.set_token

This should probably be preceded by host.expire_token.

Comment 7 Alexander Chuzhoy 2014-06-13 15:47:00 UTC
The workaround is below (all the steps shoud run on the staypuft machine):
1.yum install ruby193-rubygem-minitest -y
2.foreman-rake console  # this will put you in irb console
3.Host.find_by_name('HOSTNAME').token.destroy
4.Delete the host from the staypuft GUI management.

Comment 8 Mike Burns 2014-06-13 16:36:10 UTC
ok, moving back to staypuft.  Thanks

Comment 9 Jason E. Rist 2014-06-13 16:51:06 UTC
Trying this change https://github.com/theforeman/staypuft/pull/149/files

Awaiting acks

Setting to on_dev

Comment 10 Alexander Chuzhoy 2014-06-19 14:17:27 UTC
Verified: foreman-installer-staypuft-0.0.18-1.el6ost.noarch  Was able to delete all hosts from the foreman after deployment.

Comment 12 errata-xmlrpc 2014-08-04 18:34:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1003.html