Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1102325

Summary: The apps can not be accessed in one minute after git push finished successfully when frontend proxies is vhost plugin
Product: OpenShift Container Platform Reporter: Brenton Leanhardt <bleanhar>
Component: ContainersAssignee: Luke Meyer <lmeyer>
Status: CLOSED ERRATA QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.1.0CC: adellape, cryan, gusun, jialiu, jokerman, libra-onpremise-devel, lmeyer, mmccomas, mpatel, yanpzhan
Target Milestone: ---Keywords: Upstream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rubygem-openshift-origin-frontend-apache-vhost-0.5.2.4-1.el6op Doc Type: Bug Fix
Doc Text:
When using the apache-vhost front-end proxy plug-in on nodes, the application was unreachable for a short period of time when changes were pushed using Git, caused by the Apache back end being temporarily disabled. This bug fix updates the plug-in to ensure that the back end is not disabled in these cases, and applications are now immediately reachable after changes are pushed. While this bug fix applies to any newly created gears, for the updated package to take effect for existing gears, the following commands must be run on affected node hosts. These steps are recommended only for systems where the rubygem-openshift-origin-frontend-apache-vhost package is installed. Please note that this will cause a node outage, and should be performed during a planned broker maintenance outage. See the OpenShift Enterprise Administration Guide for full documentation on modifying the front-end proxy server plug-in configuration. # oo-frontend-plugin-modify --save > filename # oo-frontend-plugin-modify --delete # oo-frontend-plugin-modify --restore < filename
 Story Points: ---
Clone Of: 1101836 Environment:
Last Closed: 2014-06-23 07:38:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1101836    
Bug Blocks:    

Description Brenton Leanhardt 2014-05-28 18:32:24 UTC 
+++ This bug was initially created as a clone of Bug #1101836 +++

Description of problem:
The app could not access in one minute after git push finished successfully when frontend proxies is vhost plugin, but the status is started

Version-Release number of selected component (if applicable):
devenv_4819

How reproducible:
always

Steps to Reproduce:

precondition:
change the frontend proxies to vhost plugin in node.

1. Create one app and modify something
    rhc app create php php-5.3
    cd php
    sed -i s/Welcome/Change/ index.php
2. git push
    git commit -amp; git push
3. Check the status of this app
    rhc app show php --state
4. Try to access the web page of this app
    check the result
5. Try to access this app
    curl http://php-domain1.dev.rhcloud.com/

Actual results:
This app can not be accessed in one minute after git push finished successfully.

---------------------------
[gusun@dhcp-10-92 php]$ grep Welcome *
  <title>Welcome to OpenShift</title>
            <h1>Welcome t2o 31your PHP application on OpenShift</h1>
[gusun@dhcp-10-92 php]$ sed -i s/Welcome/Change/ index.php 
[gusun@dhcp-10-92 php]$ grep Welcome *
[gusun@dhcp-10-92 php]$ grep Change *
  <title>Change to OpenShift</title>
            <h1>Change t2o 31your PHP application on OpenShift</h1>
[gusun@dhcp-10-92 php]$ echo $http_proxy
http://file.rdu.redhat.com:3128
[gusun@dhcp-10-92 php]$ git commit -amp
[master 7e356a3] p
 1 file changed, 2 insertions(+), 2 deletions(-)
[gusun@dhcp-10-92 php]$ git push
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 312 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Stopping PHP 5.3 cartridge (Apache+mod_php)
remote: Waiting for stop to finish
remote: Waiting for stop to finish
remote: Building git ref 'master', commit 7e356a3
remote: Checking .openshift/pear.txt for PEAR dependency...
remote: Preparing build for deployment
remote: Deployment id is 9f6ac432
remote: Activating deployment
remote: Starting PHP 5.3 cartridge (Apache+mod_php)
remote: Application directory "/" selected as DocumentRoot
remote: -------------------------
remote: Git Post-Receive Result: success
remote: Activation status: success
remote: Deployment completed with status: success
To ssh://537a168b066a5101370004f2.rhcloud.com/~/git/php.git/
   57951f8..7e356a3  master -> master
[gusun@dhcp-10-92 php]$ rhc app show php --state -p 4210368
Cartridge php-5.3 is started
[gusun@dhcp-10-92 php]$ curl http://php-domain1.dev.rhcloud.com/
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Temporarily Unavailable</title>
</head><body>
<h1>Service Temporarily Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
<p>Additionally, a 503 Service Temporarily Unavailable
error was encountered while trying to use an ErrorDocument to handle the request.</p>
<hr>
<address>Apache/2.2.15 (Red Hat) Server at php-domain1.dev.rhcloud.com Port 80</address>
</body></html>
[gusun@dhcp-10-92 php]$ 
---------------------------

Expected results:
This app should be accessible immediately when git push finished successfully.

Additional info:
N/A

--- Additional comment from Mrunal Patel on 2014-05-28 13:45:50 EDT ---

apache disables a backend for a minute when it can't connect to it --

[Wed May 28 17:00:35 2014] [error] (111)Connection refused: proxy: HTTP: attempt to connect to 127.1.244.129:8080 (127.1.244.129) failed
[Wed May 28 17:00:35 2014] [error] ap_proxy_connect_backend disabling worker for (127.1.244.129)
[Wed May 28 17:00:35 2014] [error] proxy: HTTP: disabled connection for (127.1.244.129)

--- Additional comment from Mrunal Patel on 2014-05-28 14:16:43 EDT ---

https://github.com/openshift/origin-server/pull/5456
Comment 1 Luke Meyer 2014-05-29 17:10:23 UTC 
Added to cherry-picks:
https://github.com/openshift/enterprise-server/pull/283

Note that the fix only applies to newly-generated frontend vhosts, i.e.
typically new gears. Would have to regenerate the frontend to fix existing gears.
Comment 5 Yanping Zhang 2014-06-11 08:05:43 UTC 
Verified on 2.1.z/2014-06-10.3
     
Steps to verify:
    Ensure vhost plugin used,and file node.conf contains:OPENSHIFT_FRONTEND_HTTP_PLUGINS=openshift-origin-frontend-apache-vhost
     
    1. Create one app and modify something
        rhc app create app2 php-5.3
        cd php
        sed -i s/Welcome/New Welcome/ index.php
    2. git push
        git commit -amp; git push
    3. Check the status of this app
        rhc app show app2 --state
    4. Try to access the web page of this app
        check the result
    5. Try to access this app
       # curl http://app2-dom2.ose21z-manual.com.cn/
     
Actual results:
    2.Pushed successfully:
    remote: -------------------------
    remote: Git Post-Receive Result: success
    remote: Activation status: success
    remote: Deployment completed with status: success
    To ssh://5397fb19db26c85e6f0000a2.com.cn/~/git/app2.git/
       92a154d..bf089db  master -> master
     
    3.Display info:
    Cartridge php-5.3 is started
    4.Could be accessed.
    5.Access normally:
    <!doctype html>
    <html lang="en">
    <head>
      <meta charset="utf-8">
      <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
      <title>New Welcome to OpenShift</title>
     
    <style>
       ------
Comment 6 Luke Meyer 2014-06-12 17:21:48 UTC 
commit 32e3e4c0153d7642b963ab3d73056376ffd7f884
Commit:     Luke Meyer <lmeyer>
CommitDate: Thu May 29 13:06:06 2014 -0400

    apache-vhost: Apache shouldn't disable backend when it can't connect to it. #cherrypick

    Note that the fix only applies to newly-generated frontend vhosts, i.e.
    typically new gears.

    origin-server:
    https://bugzilla.redhat.com/show_bug.cgi?id=1101836
    commit 6f93ae525157f8d53e7e6efbea549ba33b620138
    Author: Mrunal Patel <mrunalp>
    Date:   Wed May 28 10:49:23 2014 -0700

        Bug 1101836: Apache shouldn't disable backend when it can't connect
    to it.
Comment 8 errata-xmlrpc 2014-06-23 07:38:05 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0781.html