Bug 1107690

Summary: Cloud-init phone_home raise 500 error because of POST content
Product: Red Hat Satellite Reporter: Bryan Kearney <bkearney>
Component: ProvisioningAssignee: Ohad Levy <ohadlevy>
Status: CLOSED ERRATA QA Contact: jaudet
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.0.3CC: achan, bbuckingham, bkearney, cwelton, jaudet, jhutar, jmontleo, lzap, mmccune, stbenjam, sthirugn
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: http://projects.theforeman.org/issues/5754
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
When cloud_init attempts to contact Foreman to complete the phone_home action, it fails with a HTTP 500 error due to incorrect data being transmitted in the POST header. To rectify this error, post settings are now being cleared when attempting to complete the phone_home action.
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-01-16 21:07:33 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1119657    
Bug Blocks:    

Description Bryan Kearney 2014-06-10 12:49:01 UTC
POST content (hostname ?) make foreman raise a 500 error.
We should empty the content of the POST.

<pre>
Started POST "/unattended/built?token=xxxx" for 10.237.2.40 at 2014-05-16 08:28:48 +0000
Processing by UnattendedController#built as */*
  Parameters: {"instance_id"=>"i-0000198e", "pub_key_rsa"=>"xxxx", "hostname"=>"ddsfdsdfs", "pub_key_ecdsa"=>"xxxx", "pub_key_dsa"=>"xxx", "token"=>"xxx"}
WARNING: Can't verify CSRF token authenticity
Redirected to http://test.example.com:3000/users/login
Filter chain halted as :require_login rendered or redirected
Completed 302 Found in 2.5ms (ActiveRecord: 0.6ms)
Started POST "/users/login" for 10.237.2.40 at 2014-05-16 08:28:48 +0000
Processing by UsersController#login as */*
WARNING: Can't verify CSRF token authenticity
Setting current user thread-local variable to nil
Operation FAILED: undefined method `[]' for nil:NilClass
/home/rrondeau/foreman/app/controllers/users_controller.rb:72:in `login'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/abstract_controller/base.rb:167:in `process_action'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal/rendering.rb:10:in `process_action'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:461:in `block (3 levels) in _run__2098843741482245584__process_action__4538368254813143935__callbacks'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:215:in `block in _conditional_callback_around_6377'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:326:in `around'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:310:in `_callback_around_906'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_6377'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:460:in `block (2 levels) in _run__2098843741482245584__process_action__4538368254813143935__callbacks'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:215:in `block in _conditional_callback_around_6376'
/home/rrondeau/foreman/app/models/concerns/foreman/thread_session.rb:33:in `clear_thread'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_6376'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:415:in `block in _run__2098843741482245584__process_action__4538368254813143935__callbacks'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:215:in `block in _conditional_callback_around_6375'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:326:in `around'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:310:in `_callback_around_13'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_6375'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:414:in `_run__2098843741482245584__process_action__4538368254813143935__callbacks'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:405:in `__run_callback'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:81:in `run_callbacks'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/abstract_controller/callbacks.rb:17:in `process_action'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal/rescue.rb:29:in `process_action'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/notifications.rb:123:in `block in instrument'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/notifications.rb:123:in `instrument'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
/usr/lib/ruby/gems/2.0.0/gems/activerecord-3.2.18/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/abstract_controller/base.rb:121:in `process'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/abstract_controller/rendering.rb:45:in `process'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal.rb:203:in `dispatch'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_controller/metal.rb:246:in `block in action'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/routing/route_set.rb:73:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/routing/route_set.rb:36:in `call'
/usr/lib/ruby/gems/2.0.0/gems/journey-1.0.4/lib/journey/router.rb:68:in `block in call'
/usr/lib/ruby/gems/2.0.0/gems/journey-1.0.4/lib/journey/router.rb:56:in `each'
/usr/lib/ruby/gems/2.0.0/gems/journey-1.0.4/lib/journey/router.rb:56:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/routing/route_set.rb:608:in `call'
/usr/lib/ruby/gems/2.0.0/gems/apipie-rails-0.1.3/lib/apipie/static_dispatcher.rb:65:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-openid-1.4.2/lib/rack/openid.rb:98:in `call'
/usr/lib/ruby/gems/2.0.0/gems/apipie-rails-0.1.3/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/etag.rb:23:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/conditionalget.rb:35:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/head.rb:14:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
/home/rrondeau/foreman/lib/middleware/catch_json_parse_errors.rb:9:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/flash.rb:242:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:210:in `context'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:205:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/cookies.rb:341:in `call'
/usr/lib/ruby/gems/2.0.0/gems/activerecord-3.2.18/lib/active_record/query_cache.rb:64:in `call'
/usr/lib/ruby/gems/2.0.0/gems/activerecord-3.2.18/lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:405:in `_run__4061752724268364408__call__1048963326518903143__callbacks'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:405:in `__run_callback'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/callbacks.rb:81:in `run_callbacks'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/rack/logger.rb:32:in `call_app'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/rack/logger.rb:16:in `block in call'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/tagged_logging.rb:22:in `tagged'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/rack/logger.rb:16:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/request_id.rb:22:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/methodoverride.rb:21:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call'
/usr/lib/ruby/gems/2.0.0/gems/activesupport-3.2.18/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call'
/usr/lib/ruby/gems/2.0.0/gems/actionpack-3.2.18/lib/action_dispatch/middleware/static.rb:63:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
/usr/lib/ruby/gems/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb:143:in `pass'
/usr/lib/ruby/gems/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb:155:in `invalidate'
/usr/lib/ruby/gems/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb:71:in `call!'
/usr/lib/ruby/gems/2.0.0/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/engine.rb:484:in `call'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/application.rb:231:in `call'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/railtie/configurable.rb:30:in `method_missing'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/builder.rb:134:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/urlmap.rb:64:in `block in call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/urlmap.rb:49:in `each'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/urlmap.rb:49:in `call'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/content_length.rb:14:in `call'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/rack/log_tailer.rb:17:in `call'
/usr/lib/ruby/gems/2.0.0/gems/thin-1.6.2/lib/thin/connection.rb:86:in `block in pre_process'
/usr/lib/ruby/gems/2.0.0/gems/thin-1.6.2/lib/thin/connection.rb:84:in `catch'
/usr/lib/ruby/gems/2.0.0/gems/thin-1.6.2/lib/thin/connection.rb:84:in `pre_process'
/usr/lib/ruby/gems/2.0.0/gems/thin-1.6.2/lib/thin/connection.rb:53:in `process'
/usr/lib/ruby/gems/2.0.0/gems/thin-1.6.2/lib/thin/connection.rb:39:in `receive_data'
/usr/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run_machine'
/usr/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run'
/usr/lib/ruby/gems/2.0.0/gems/thin-1.6.2/lib/thin/backends/base.rb:73:in `start'
/usr/lib/ruby/gems/2.0.0/gems/thin-1.6.2/lib/thin/server.rb:162:in `start'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/handler/thin.rb:13:in `run'
/usr/lib/ruby/gems/2.0.0/gems/rack-1.4.5/lib/rack/server.rb:268:in `start'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/commands/server.rb:70:in `start'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/commands.rb:55:in `block in <top (required)>'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/commands.rb:50:in `tap'
/usr/lib/ruby/gems/2.0.0/gems/railties-3.2.18/lib/rails/commands.rb:50:in `<top (required)>'
script/rails:6:in `require'
script/rails:6:in `<main>'
  Rendered common/500.html.erb within layouts/application (3.9ms)
  Rendered layouts/base.html.erb (1.1ms)
Completed 500 Internal Server Error in 9.8ms (Views: 6.1ms | ActiveRecord: 1.5ms)
</pre>

Comment 1 Bryan Kearney 2014-06-10 12:49:03 UTC
Created from redmine issue http://projects.theforeman.org/issues/5754

Comment 2 Bryan Kearney 2014-06-10 13:04:05 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/5754 has been closed

Comment 6 Lukas Zapletal 2014-09-01 15:56:56 UTC
Unable to verify due to other bug:

Safemode doesn't allow to access 'indent' on #<Safemode::ScopeObject>

https://bugzilla.redhat.com/show_bug.cgi?id=1119657

Comment 8 Martin Korbel 2014-09-09 08:13:48 UTC
Tested on Satellite-6.0.4-RHEL-6-20140829.0. But bz1119657 is still a blocker.

Comment 9 Stephen Benjamin 2014-11-18 11:10:17 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=1119657 was merged so this can be tested again.

Comment 11 Mike McCune 2014-12-11 17:24:36 UTC
temp move to MOD for ET

Comment 13 jaudet 2015-01-07 20:39:38 UTC
Can I get more information about this issue? Based on the bug description and following discussion, all I know is that it's possible to produce an HTTP 500 by POSTing. But how? Which URLs should be POSTed to? With what parameters? Can a concrete example be given? This bug is vague.

Comment 14 jaudet 2015-01-09 19:46:42 UTC
I've spent some time watching youtube vids and reading docs, and I've finally figured out the basics of cloud-init and phone_home. It appears that cloud-init is software that initializes just-created virtual machines, and phone_home is a cloud-init module that POSTs data to a URL.

For an introduction to cloud-init: https://www.youtube.com/watch?v=-zL3BdbKyGY

For an introduction to phone_home: http://cloudinit.readthedocs.org/en/latest/topics/examples.html#call-a-url-when-finished

The procedure for verifying this bug is still being figured out. At this time, my buest guess is that the verification procedure goes like this:

1. Use a Foreman server to provision a new host. It is probably easiest to use a tool like libvirt to provision a new virtual machine, as no new hardware is necessary in that case.
2. Make sure that the host executes cloud-init at the end of the provisioning process, and that the phone_home module is configured to POST a non-empty payload to the Foreman server.

If the Foreman server can succeed in provisioning the new host, then we're good. If the Foreman server forever lists the new host as being built, then this bug is still in effect.

More info on how to test this bug would be appreciated.

Comment 15 Stephen Benjamin 2015-01-11 18:20:10 UTC
IMHO this was verified by BZ1119657, which verified the user data templates we ship are working, which include phone_home.

But, you wouldn't provision a libvirt machine, you'd use something like OpenStack and an image that comes cloud-init enabled (all the official RHEL images are).

Comment 18 errata-xmlrpc 2015-01-16 21:07:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:0054

Comment 19 Red Hat Bugzilla 2023-09-14 02:09:45 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days