Bug 1194269 - Deployment will stop on systems which boots/shutdowns quickly and foreman-proxy is terminated before buffer is flushed.
Summary: Deployment will stop on systems which boots/shutdowns quickly and foreman-pro...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: foreman-discovery-image
Version: 6.0 (Juno)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z2
: Installer
Assignee: Mike Burns
QA Contact: Alexander Chuzhoy
URL: http://projects.theforeman.org/issues...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-02-19 13:00 UTC by Leonid Natapov
Modified: 2023-02-22 23:02 UTC (History)
10 users (show)

Fixed In Version: foreman-proxy-1.6.0.30-6.el7ost foreman-discovery-image-7.0-20150227.0.el7ost
Doc Type: Known Issue
Doc Text:
In some rare cases, a host may shut down foreman-proxy before completing the reboot process. This causes hosts to fail to reboot when starting a Red Hat Enterprise Linux OpenStack Platform installer deployment. As a workaround, you can manually trigger the host to reboot and, if necessary, click 'Resume' to resume the deployment, allowing the deployment to proceed without issue.
Clone Of:
Environment:
Last Closed: 2015-04-07 15:08:41 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0791 0 normal SHIPPED_LIVE Important: Red Hat Enterprise Linux OpenStack Platform Installer update 2015-04-07 19:07:29 UTC

Description Leonid Natapov 2015-02-19 13:00:16 UTC
Description of problem:
On systems which boots/shutdowns quickly, foreman-proxy is terminated before buffer is flushed. Therefore the deployment will stop.

Scenario:
I have 3 controllers and 2 computes ready for deployment. After clicking on "deploy" button 3 controllers were rebooted but 2 computed nodes were not.
This caused deployment to stop with the error. After manually resuming deployment compute nodes went for reboot and deployment successfully finished. 

Workaround:

Resume deployment. 

production.log
---------------------------------------------------------------------------
Completed 200 OK in 585ms (Views: 580.7ms | ActiveRecord: 2.2ms)
Add the TFTP configuration for mac848f69fbc493.example.com
ForemanDiscovery: Rebooting mac848f69fbc493.example.com as its being discovered and assigned
Failed to reboot: undefined method `[]' for true:TrueClass
Add the TFTP configuration for macf04da2732fb1.example.com
ForemanDiscovery: Rebooting macf04da2732fb1.example.com as its being discovered and assigned
Failed to reboot: undefined method `[]' for true:TrueClass
---------------------------------------------------------------------------
Version-Release number of selected component (if applicable):


[root@staypuft foreman]# less production.log 
[root@staypuft foreman]# rpm -qa | grep foreman
foreman-postgresql-1.6.0.49-6.el7ost.noarch
foreman-installer-1.6.0-0.3.RC1.el7ost.noarch
rubygem-foreman_api-0.1.11-6.el7sat.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el7ost.noarch
openstack-foreman-installer-3.0.16-1.el7ost.noarch
ruby193-rubygem-foreman_discovery-1.3.0-2.3.el7ost.noarch
foreman-1.6.0.49-6.el7ost.noarch
foreman-selinux-1.6.0.14-1.el7sat.noarch
foreman-discovery-image-7.0-20140905.0.2.el7sat.noarch
ruby193-rubygem-foreman-tasks-0.6.9-1.1.1.el7ost.noarch
foreman-proxy-1.6.0.30-5.el7ost.noarch

Comment 5 Mike Burns 2015-02-19 13:08:36 UTC
Note:  bug exists in foreman-proxy, but a new foreman-discovery-image will also need to be built to include the package.

Comment 6 Mike Burns 2015-02-27 17:22:45 UTC
Backported the patch in the referenced upstream issue.  Included it in a new foreman-proxy build and rebuilt the foreman-discovery-image with this new foreman-proxy.

Comment 9 Alexander Chuzhoy 2015-03-13 22:21:04 UTC
Verified:

Environment:
rhel-osp-installer-client-0.5.7-1.el7ost.noarch
ruby193-rubygem-foreman_discovery-1.3.0-2.3.el7ost.noarch
foreman-1.6.0.49-6.el7ost.noarch
foreman-postgresql-1.6.0.49-6.el7ost.noarch
foreman-selinux-1.6.0.14-1.el7sat.noarch
openstack-foreman-installer-3.0.17-1.el7ost.noarch
rhel-osp-installer-0.5.7-1.el7ost.noarch
ruby193-rubygem-foreman-tasks-0.6.9-1.1.1.el7ost.noarch
ruby193-rubygem-staypuft-0.5.21-1.el7ost.noarch
foreman-proxy-1.6.0.30-6.el7ost.noarch
rubygem-hammer_cli_foreman-0.1.1-16.el7sat.noarch
openstack-puppet-modules-2014.2.8-2.el7ost.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el7ost.noarch
foreman-installer-1.6.0-0.3.RC1.el7ost.noarch
foreman-discovery-image-7.0-20150227.0.el7ost.noarch
rubygem-hammer_cli_foreman-doc-0.1.1-16.el7sat.no

The reported issue doesn't reproduce.

Comment 13 errata-xmlrpc 2015-04-07 15:08:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0791.html


Note You need to log in before you can comment on or make changes to this bug.