Bug 1148418

Summary: Race condition in `oo-httpd-singular graceful` when using apache-vhost
Product: OpenShift Online Reporter: Luke Meyer <lmeyer>
Component: ContainersAssignee: Jhon Honce <jhonce>
Status: CLOSED DUPLICATE QA Contact: libra bugs <libra-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 2.xCC: jokerman, libra-bugs, libra-onpremise-devel, mmccomas, tiwillia
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1148192 Environment:
Last Closed: 2014-10-01 18:17:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Luke Meyer 2014-10-01 12:29:21 UTC
+++ This bug was initially created as a clone of Bug #1148192 +++

Description of problem:
There appears to be a race condition when using the apache-vhost frontend. If oo-httpd-singluar graceful is run while an application is creating its frontend configuration, the configuration is in a bad state and httpd will not start.

Version-Release number of selected component (if applicable):
2.1.6

How reproducible:
Very Rarely

Steps to Reproduce:
1. Create many applications while removing others
2.
3.

Actual results:
We see the following in the node's platform.log. 
-=~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~=-
September 22 13:39:32 INFO Shell command '/usr/sbin/oo-httpd-singular  graceful' ran. rc=0 out=
September 22 13:39:32 INFO Connecting frontend mapping for 54206ba04970fa5329000003/haproxy: [/haproxy-status] => [127.2.20.3:8080/] with options: {"protocols"=>["http"]}
September 22 13:39:32 WARN V2CartModel#connect_frontend: No such file or directory - /etc/httpd/conf.d/openshift/54206ba04970fa5329000003_e2e_54206ba04970fa5329000003/599984_element-_haproxy-status.conf
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.22.5.11/lib/openshift-origin-common/utils/file_needs_sync.rb:36:in `initialize'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.22.5.11/lib/openshift-origin-common/utils/file_needs_sync.rb:36:in `open'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.22.5.11/lib/openshift-origin-common/utils/file_needs_sync.rb:36:in `open'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-frontend-apache-vhost-0.5.2.4/lib/openshift/runtime/frontend/http/plugins/apache-vhost.rb:152:in `block (2 levels) in connect'
  [... backtrace cut for clarity ...]
September 22 13:39:32 ERROR Unexpected error during configure: No such file or directory - /etc/httpd/conf.d/openshift/54206ba04970fa5329000003_e2e_54206ba04970fa5329000003/599984_element-_haproxy-status.conf (Errno::ENOENT)
September 22 13:39:32 INFO openshift-agent: request end: action=cartridge_do, requestid=5373aabdbd865534bec108f8bf32d199, senderid=lae-alln-brk02, statuscode=1, data={:time=>nil, :output=>"CLIENT_ERROR: Unexpected error: No such file or directory - /etc/httpd/conf.d/openshift/54206ba04970fa5329000003_e2e_54206ba04970fa5329000003/599984_element-_haproxy-status.conf\n", :exitcode=>1, :addtl_params=>nil}
September 22 13:39:32 INFO Shell command '/usr/sbin/oo-httpd-singular  graceful' ran. rc=1 out=
September 22 13:39:32 ERROR ERROR: failure from oo-httpd-singular(1): : stdout:  stderr:httpd.worker: Syntax error on line 221 of /etc/httpd/conf/httpd.conf: Syntax error on line 45 of /etc/httpd/conf.d/000001_openshift_origin_frontend_vhost.conf: Syntax error on line 29 of /etc/httpd/conf.d/openshift/54206ba04970fa5329000003_e2e_0_54206ba04970fa5329000003.conf: Include directory '/etc/httpd/conf.d/openshift/54206ba04970fa5329000003_e2e_54206ba04970fa5329000003' not found
-=~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~=-


Expected results:
All applications created/removed successfully

Additional info:
This looks very similar to Bugzilla 1146194 but the same error messages are not observed.

Comment 1 Jhon Honce 2014-10-01 18:17:44 UTC

*** This bug has been marked as a duplicate of bug 1147054 ***