Bug 845332 - [ORIGIN] stickshift-proxy seems to stop when creating a scaled app
[ORIGIN] stickshift-proxy seems to stop when creating a scaled app
Status: CLOSED CURRENTRELEASE
Product: OpenShift Origin
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Linux
high Severity high
: ---
: ---
Assigned To: Rob Millner
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-02 13:03 EDT by Krishna Raman
Modified: 2015-05-14 18:57 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-08-07 16:42:18 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Krishna Raman 2012-08-02 13:03:24 EDT
Description of problem:
Scaled app fails to create since stickshift-proxy seems to stop when creating a scaled app

Version-Release number of selected component (if applicable):


How reproducible:
Always (Origin)

Steps to Reproduce:
1. Create a scaled php-5.3 app
  
Actual results:
App fails to create as the proxy port is not exposed

Expected results:
App succeeds

Additional info:
Comment 1 Rob Millner 2012-08-02 16:34:40 EDT
Systemd appears to issue a stop on stickshift-proxy some time after the proxy is properly started by the gear create.
Comment 2 Rob Millner 2012-08-02 17:07:03 EDT
It appears to do so at the same time that setroubleshootd is examining the stickshift-proxy package.

There's an sealert message at the same time that haproxy shows a safe shutdown.

Additional Information:
Source Context                system_u:system_r:mcollectived_t:s0-s0:c0.c1023
Target Context                unconfined_u:object_r:etc_t:s0
Target Objects                /etc/rc.d/init.d/stickshift-proxy [ file ]
Source                        stickshift-prox
Source Path                   /bin/bash
Port                          <Unknown>
Host                          localhost.localdomain
Source RPM Packages           
Target RPM Packages           
Policy RPM                    <Unknown>
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     broker.example.com
Platform                      Linux broker.example.com 3.4.6-1.fc16.x86_64 #1
                              SMP Fri Jul 20 12:58:04 UTC 2012 x86_64 x86_64
Alert Count                   59
First Seen                    Thu 02 Aug 2012 06:30:27 AM EDT
Last Seen                     Thu 02 Aug 2012 10:02:36 AM EDT
Local ID                      a97758c2-8be8-4584-abba-2da11d5b140b
Comment 3 Rob Millner 2012-08-02 18:16:25 EDT
Worked around the selinux issue and its still happening.  It appears as though stickshift-proxy's status call shows it as dead when the proxy is reloaded by the app framework but not by hand.

Systemd appears to do some automatic cleanup which is why we're seeing the termination.
Comment 4 Rob Millner 2012-08-02 20:02:17 EDT
This fails on Fedora 16 but works on RHEL 6:


service stickshift-proxy start; i=40000; while [ $i -lt 40100 ]; do service stickshift-proxy setproxy $i 127.0.0.1:8080; i=$(( $i + 1 )); done


This works on Fedora 16 and RHEL 6:

service stickshift-proxy start; i=40000; while [ $i -lt 40100 ]; do service stickshift-proxy reload; i=$(( $i + 1 )); done



I believe the issue is that systemd's crashed daemon detection is in a race with setproxy's reload.  But since systemd expects the PID swap on a reload it temporarily disables the detection.

I believe the simplest thing to do is move the configuration update functionality to a separate tool (ss-proxy-update) and have it call reload.
Comment 6 Peter Ruan 2012-08-07 13:31:16 EDT
verified with livecd build on Aug 3rd.

[liveuser@broker ~]$ rhc app create -a myapp -t php-5.3 -s -d
remote: Total 26 (delta 1), reused 26 (delta 1)
Receiving objects: 100% (26/26), 7.42 KiB, done.
Resolving deltas: 100% (1/1), done.
Confirming application 'myapp' is available
  Attempt # 1
Confirming application 'myapp' is available:  Success!

myapp published:  http://myapp-livecd.example.com/
git url:  ssh://218abcbc632a4f08a9dd579cc47f6a6b@myapp-livecd.example.com/~/git/myapp.git/
To make changes to 'myapp', commit to myapp/.
Successfully created application: myapp
[liveuser@broker ~]$ rhc domain show -padmin

User Info
=========
Namespace: livecd
  RHLogin: admin


Application Info
================
myapp
    Framework: php-5.3
     Creation: 2012-08-07T13:24:39-04:00
         UUID: 218abcbc632a4f08a9dd579cc47f6a6b
      Git URL: ssh://218abcbc632a4f08a9dd579cc47f6a6b@myapp-livecd.example.com/~/git/myapp.git/
   Public URL: http://myapp-livecd.example.com/

 Embedded: 
      haproxy-1.4

Note You need to log in before you can comment on or make changes to this bug.