Bug 1044980 - Ruby app trys to load module and connection is refused
Summary: Ruby app trys to load module and connection is refused
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: ImageStreams
Version: 2.0.0
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: ---
Assignee: Jason DeTiberus
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks: 1212583
TreeView+ depends on / blocked
 
Reported: 2013-12-19 11:21 UTC by Nan Wei
Modified: 2015-04-16 17:29 UTC (History)
7 users (show)

Fixed In Version: rubygem-passenger-3.0.21-12.el6op
Doc Type: Bug Fix
Doc Text:
A change to Passenger caused applications using Ruby 1.8 or 1.9 cartridges to report pre-starting errors, and applications would only start after a first request was made. The issue was fixed for Ruby 1.9 cartridges in an earlier release of OpenShift Enterprise; this bug fix updates Passenger to correct the issue for Ruby 1.8 cartridges as well. Pre-starting now functions properly for both Ruby cartridges, and the previous errors no longer occur.
Clone Of:
: 1212583 (view as bug list)
Environment:
Last Closed: 2014-06-23 07:37:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 984867 0 low CLOSED Errno::ECONNREFUSED shown in /var/lib/openshift/<uuid>/ruby/logs/error_log with ruby-1.9 app 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1089958 0 medium CLOSED Connection refused error on ruby-1.8 log 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHBA-2014:0781 0 normal SHIPPED_LIVE Red Hat OpenShift Enterprise 2.1.2 bug fix update 2014-06-23 11:36:38 UTC

Internal Links: 984867 1089958

Description Nan Wei 2013-12-19 11:21:55 UTC
Description of problem:
Create ruby1.8 ruby 1.9 app, and check the error_log for the apps, it shows some error message as follow: 
"[Thu Dec 19 04:42:24 2013] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c0,c1003
[Thu Dec 19 04:42:24 2013] [notice] Digest: generating secret for digest authentication ...
[Thu Dec 19 04:42:24 2013] [notice] Digest: done
[Thu Dec 19 04:42:24 2013] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations
/usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `new'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `connect'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:86:in `socket'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:90:in `head_request'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:145
/usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `new'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `connect'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:86:in `socket'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:90:in `head_request'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:145"
But it doesn't exist error message like this in the deven enironment.
OpenShift Enterprise 2.0.1 puddle: [2.0.z/2013-12-18.1]
How reproducible:
always
Steps to Reproduce:
1. Create a app 
#rhc app create $appname ruby-1.8/ruby-1.9.
2. cat haproyx/logs/scale_events.log
#cat /var/lib/openshift/$uuid/haproyx/logs/scale_events.log
Actual results:
[Thu Dec 19 04:42:24 2013] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c0,c1003
[Thu Dec 19 04:42:24 2013] [notice] Digest: generating secret for digest authentication ...
[Thu Dec 19 04:42:24 2013] [notice] Digest: done
[Thu Dec 19 04:42:24 2013] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations
/usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `new'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `connect'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:86:in `socket'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:90:in `head_request'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:145
/usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `new'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `connect'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:86:in `socket'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:90:in `head_request'
    from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:145
Expected results:
There is no error messages.

Comment 2 Nan Wei 2014-02-19 04:03:15 UTC
This bug isn't reproduced in ose puddle 2014-02-12.1

Comment 3 Nan Wei 2014-04-04 08:59:32 UTC
Openshift enterprise: 2.1/2014-04-03.2

1. Create ruby-1.8/1.9 app
2. Ssh into app and check ~/app-root/logs/ruby.log

[ru-nweidomain.ose21-manual.com.cn 533e6de8fdae85dd7e00001e]\> cat app-root/logs/ruby.log 
httpd: Could not reliably determine the server's fully qualified domain name, using node2.ose21-manual.com.cn for ServerName
[Fri Apr 04 01:31:47 2014] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c3,c339
httpd: Could not reliably determine the server's fully qualified domain name, using node2.ose21-manual.com.cn for ServerName
[Fri Apr 04 01:31:47 2014] [notice] Digest: generating secret for digest authentication ...
[Fri Apr 04 01:31:47 2014] [notice] Digest: done
[Fri Apr 04 01:31:47 2014] [notice] Apache/2.2.22 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations
/usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
	from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `new'
	from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:105:in `connect'
	from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:86:in `socket'
	from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:90:in `head_request'
	from /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:145

So move it new.

Comment 6 Brenton Leanhardt 2014-04-08 13:47:26 UTC
I figured out what is going on.  Passenger is set to prepawn.  Our apache module is actually patched so that OpenShift can correctly spawn on an IP other than 127.0.0.1.  However, the passenger rubygem is not patched.  See /usr/share/rubygems/gems/passenger-3.0.21/helper-scripts/prespawn:

class TCPPrespawnLocation < PrespawnLocation
        def request_port
                @uri.port
        end

        def connect
                TCPSocket.new('127.0.0.1', request_port)
        end
end

It turns out it's trying to respawn by issuing a request to itself.  However 127.0.0.1 is hardcoded.  In my dev environment there actually was something listening on 127.0.0.1:8080 which is why I didn't see the error.

The good news is that this error is "harmless" (technically it breaks prespawning).  We'll need to patch the passenger rubygem to know how to prespawn on something other than 127.0.0.1.  I'll have to see how Online works around this (if they do).

Comment 8 Yanping Zhang 2014-06-13 06:29:57 UTC
Verified on ose 2.1.z/2014-06-12.2
Steps to verify:
1. Create ruby-1.8/1.9 app
2. Ssh into app and check ~/app-root/logs/ruby.log

Actual results:
2.Display no errors:
[rubytest-dom13.ose21z-manual.com.cn 539a96b9db26c8b7e9000062]\> cat app-root/logs/ruby.log 
httpd: Could not reliably determine the server's fully qualified domain name, using node2.ose21z-manual.com.cn for ServerName
[Thu Jun 12 23:14:27 2014] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c3,c80
httpd: Could not reliably determine the server's fully qualified domain name, using node2.ose21z-manual.com.cn for ServerName
[Thu Jun 12 23:14:27 2014] [notice] Digest: generating secret for digest authentication ...
[Thu Jun 12 23:14:27 2014] [notice] Digest: done
[Thu Jun 12 23:14:27 2014] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations
- - - [12/Jun/2014:23:14:29 -0700] "HEAD / HTTP/1.1" 200 - "-" "-"
- - - [12/Jun/2014:23:14:30 -0700] "HEAD / HTTP/1.1" 200 - "-" "-"
- - - [12/Jun/2014:23:14:34 -0700] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
- - - [12/Jun/2014:23:14:35 -0700] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"

Comment 11 errata-xmlrpc 2014-06-23 07:37:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0781.html


Note You need to log in before you can comment on or make changes to this bug.