Hide Forgot
Description of problem: Create scalable app of any type, scale-up the app, check the haproxy-statue page, the new gear is in down status. When scale-up the app to 3 gears, the app will become unavailable since the local-gear is also down. When access the scaled-up gears' dns, it can be visited. Version-Release number of selected component (if applicable): STG (devenv-stage_549) How reproducible: always Steps to Reproduce: 1.Create scalable app 2.Set the min gear number to scale the app up 3.Check the scaled-up gear Actual results: The gears are in down status and cannot be accessed. But can be visited via the gear dns directly. Expected results: Scaled-up gears should be available. Additional info: Found the following exception in haproxy_ctld.rb.log: I, [2013-11-04T03:40:30.621166 #87713] INFO -- : *** below you find the most recent exception thrown, this will be likely (but not certainly) the exception that made the application exit abnormally *** E, [2013-11-04T03:40:30.621249 #87713] ERROR -- : Failed to get information from haproxy (Haproxy::ShouldRetry) /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld.rb:126:in `refresh' /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld.rb:67:in `initialize' /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld.rb:345:in `new' /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld.rb:345:in `<top (required)>' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/application.rb:176:in `load' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/application.rb:176:in `start_load' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/application.rb:257:in `start' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/controller.rb:69:in `run' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons.rb:139:in `block in run' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/cmdline.rb:105:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/cmdline.rb:105:in `catch_exceptions' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons.rb:138:in `run' /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld_daemon.rb:21:in `<main>' I, [2013-11-04T03:40:30.621340 #87713] INFO -- : *** below you find all exception objects found in memory, some of them may have been thrown in your application, others may just be in memory because they are standard exceptions *** E, [2013-11-04T03:40:30.622267 #87713] ERROR -- : stream closed (IOError) E, [2013-11-04T03:40:30.622370 #87713] ERROR -- : failed to allocate memory (NoMemoryError) E, [2013-11-04T03:40:30.622461 #87713] ERROR -- : stack level too deep (SystemStackError) E, [2013-11-04T03:40:30.622726 #87713] ERROR -- : exception reentered (fatal) E, [2013-11-04T03:40:30.628289 #87713] ERROR -- : exit (SystemExit) /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld.rb:355:in `exit' /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld.rb:355:in `rescue in <top (required)>' /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld.rb:341:in `<top (required)>' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/application.rb:176:in `load' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/application.rb:176:in `start_load' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/application.rb:257:in `start' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/controller.rb:69:in `run' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons.rb:139:in `block in run' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/cmdline.rb:105:in `call' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons/cmdline.rb:105:in `catch_exceptions' /opt/rh/ruby193/root/usr/share/gems/gems/daemons-1.0.10/lib/daemons.rb:138:in `run' /var/lib/openshift/52775d59dbd93ced67000002/haproxy/usr/bin/haproxy_ctld_daemon.rb:21:in `<main>'
I don't believe the errors listed in haproxy_ctld.rb.log are related - I see them in my app in prod. I believe the issue is either an issue with iptables (port proxy) or the firewall. I tested creating an app in stg and my scaled up gear (gear-5277917e2587c84142000a5a-agoldste) is supposed to be accessible by haproxy at ex-std-node1.stg.rhcloud.com:35536, but it can't connect, which is why the gear is listed as DOWN in the haproxy-status page.
Please retest - this should be working now.
Checked on current STG (devenv-stage_552), The scaled-up gears for scalable app work fine. Move bug to verified.