Bug 1095234

Summary: App with redmine quickstart became inaccessible with hot_deploy marker added
Product: OpenShift Online Reporter: Qixuan Wang <qixuan.wang>
Component: TemplatesAssignee: Jakub Hadvig <jhadvig>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: jhadvig
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-10 00:47:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Qixuan Wang 2014-05-07 10:32:48 UTC 
Description of problem:
Create a redmine quickstart application, add hot_deploy marker and git push, the webpage shows 500 Internal server error, meanwhile the app and cartridges are running. Restart app still have the issue. If add other markers or files without hot_deploy in local repo and then git push can't cause this issue.

Version-Release number of selected component (if applicable):
devenv_4768

How reproducible:
Always

Steps to Reproduce:
1. Create an ruby-1.8/1.9 non-scalable/scalable application with mysql-5.1, (phpmyadmin only for scalable) cartridges
# rhc app create redmine ruby-1.8 mysql-5.1 phpmyadmin --from-code git://github.com/openshift/openshift-redmine-quickstart.git

2. Add a hot_deploy marker under .openshift/markers/ and git push 

3. Access the url and check the cartridge state
[root@qwang markers]# rhc app show redmine --state
Cartridge mysql-5.1, phpmyadmin-4, ruby-1.8 is started
[root@qwang markers]# rhc cartridge status ruby-1.8 -a redmine
RESULT:
Application is running
[root@qwang qwang]# rhc cartridge status mysql-5.1 -a redmine
RESULT:
MySQL is running
[root@qwang qwang]# rhc cartridge status phpmyadmin-4 -a redmine
RESULT:
PHPMyAdmin is running


Actual results:
Webpage shows Internal error

Expected results:
Redmine webpage should show

Additional info:
app-root/runtime/repo/log/production.log
ArgumentError (A secret is required to generate an integrity hash for cookie session data. Use config.secret_token = "some secret phrase of at least 30 characters"in config/initializers/secret_token.rb):
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:319:in `ensure_secret_secure'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:284:in `initialize'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:231:in `new'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:231:in `signed'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/cookie_store.rb:50:in `block in unpacked_cookie_data'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/abstract_store.rb:57:in `stale_session_check!'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/cookie_store.rb:48:in `unpacked_cookie_data'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/cookie.rb:107:in `extract_session_id'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/abstract_store.rb:53:in `block in extract_session_id'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/abstract_store.rb:57:in `stale_session_check!'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/abstract_store.rb:53:in `extract_session_id'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:43:in `load_session_id!'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:32:in `[]'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:267:in `current_session_id'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:273:in `session_exists?'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:107:in `exists?'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:127:in `load_for_read!'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:64:in `has_key?'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/flash.rb:258:in `ensure in call'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/flash.rb:259:in `call'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:210:in `context'
  vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:205:in `call'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:341:in `call'
  vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.16/lib/active_record/query_cache.rb:64:in `call'
  vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.16/lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call'
  vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.16/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
  vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.16/lib/active_support/callbacks.rb:405:in `_run__4068673213941603306__call__2353755551128418041__callbacks'
Comment 1 Qixuan Wang 2014-05-07 10:36:29 UTC 
Sorry, correct a mistake of step 1, phpmyadmin cartridge can't be added on scalable app.
Comment 2 Jakub Hadvig 2014-06-26 14:27:47 UTC 
Found you from rails logs that the problem seems to be in generating cookie session data.

ArgumentError (A secret is required to generate an integrity hash for cookie session data. Use config.secret_token = "some secret phrase of at least 30 characters"in config/initializers/secret_token.rb)
Comment 3 Jakub Hadvig 2014-06-29 20:22:09 UTC 
Fixed by:
https://github.com/openshift/openshift-redmine-quickstart/pull/6
Comment 4 Qixuan Wang 2014-06-30 02:58:02 UTC 
Verifed on devenv_4920, redmine quickstart with hot_deploy works, thanks.
Comment 5 Qixuan Wang 2014-07-14 10:37:59 UTC 
Met the issue on STG with old app after migration. The issue can't be reproduced on INT, create a new one on devenv-stage_910 can't reproduce it either. So reopen it.

[redmine-dm.stg.rhcloud.com 53a408e8dbd93c5f25001dcd]\> cat app-root/runtime/repo/log/production.log 
ArgumentError (A secret is required to generate an integrity hash for cookie session data. Use config.secret_token = "some secret phrase of at least 30 characters"in config/initializers/secret_token.rb):
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:319:in `ensure_secret_secure'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:284:in `initialize'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:231:in `new'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:231:in `signed'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/cookie_store.rb:50:in `unpacked_cookie_data'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/abstract_store.rb:57:in `stale_session_check!'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/cookie_store.rb:48:in `unpacked_cookie_data'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/cookie.rb:107:in `extract_session_id'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/abstract_store.rb:53:in `extract_session_id'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/abstract_store.rb:57:in `stale_session_check!'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/session/abstract_store.rb:53:in `extract_session_id'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:43:in `send'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:43:in `load_session_id!'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:32:in `[]'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:267:in `current_session_id'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:273:in `session_exists?'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:107:in `send'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:107:in `exists?'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:127:in `load_for_read!'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:64:in `key?'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/flash.rb:258:in `call'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:210:in `context'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:205:in `call'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/cookies.rb:341:in `call'
  vendor/bundle/ruby/1.8/gems/activerecord-3.2.16/lib/active_record/query_cache.rb:64:in `call'
  vendor/bundle/ruby/1.8/gems/activerecord-3.2.16/lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/callbacks.rb:28:in `call'
  vendor/bundle/ruby/1.8/gems/activesupport-3.2.16/lib/active_support/callbacks.rb:405:in `_run__1268926552__call__4__callbacks'
  vendor/bundle/ruby/1.8/gems/activesupport-3.2.16/lib/active_support/callbacks.rb:405:in `send'
  vendor/bundle/ruby/1.8/gems/activesupport-3.2.16/lib/active_support/callbacks.rb:405:in `__run_callback'
  vendor/bundle/ruby/1.8/gems/activesupport-3.2.16/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
  vendor/bundle/ruby/1.8/gems/activesupport-3.2.16/lib/active_support/callbacks.rb:81:in `send'
  vendor/bundle/ruby/1.8/gems/activesupport-3.2.16/lib/active_support/callbacks.rb:81:in `run_callbacks'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
  vendor/bundle/ruby/1.8/gems/railties-3.2.16/lib/rails/rack/logger.rb:32:in `call_app'
  vendor/bundle/ruby/1.8/gems/railties-3.2.16/lib/rails/rack/logger.rb:16:in `call'
  vendor/bundle/ruby/1.8/gems/activesupport-3.2.16/lib/active_support/tagged_logging.rb:22:in `tagged'
  vendor/bundle/ruby/1.8/gems/railties-3.2.16/lib/rails/rack/logger.rb:16:in `call'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/request_id.rb:22:in `call'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/methodoverride.rb:21:in `call'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call'
  vendor/bundle/ruby/1.8/gems/activesupport-3.2.16/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
  vendor/bundle/ruby/1.8/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call'
  vendor/bundle/ruby/1.8/gems/actionpack-3.2.16/lib/action_dispatch/middleware/static.rb:63:in `call'
  vendor/bundle/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
  vendor/bundle/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:245:in `fetch'
  vendor/bundle/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:185:in `lookup'
  vendor/bundle/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!'
  vendor/bundle/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
  vendor/bundle/ruby/1.8/gems/railties-3.2.16/lib/rails/engine.rb:484:in `call'
  vendor/bundle/ruby/1.8/gems/railties-3.2.16/lib/rails/application.rb:231:in `call'
  vendor/bundle/ruby/1.8/gems/railties-3.2.16/lib/rails/railtie/configurable.rb:30:in `send'
  vendor/bundle/ruby/1.8/gems/railties-3.2.16/lib/rails/railtie/configurable.rb:30:in `method_missing'
  passenger (3.0.21) lib/phusion_passenger/rack/request_handler.rb:97:in `process_request'
  passenger (3.0.21) lib/phusion_passenger/abstract_request_handler.rb:521:in `accept_and_process_next_request'
  passenger (3.0.21) lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
  passenger (3.0.21) lib/phusion_passenger/rack/application_spawner.rb:206:in `start_request_handler'
  passenger (3.0.21) lib/phusion_passenger/rack/application_spawner.rb:79:in `spawn_application'
  passenger (3.0.21) lib/phusion_passenger/utils.rb:470:in `safe_fork'
  passenger (3.0.21) lib/phusion_passenger/rack/application_spawner.rb:64:in `spawn_application'
  passenger (3.0.21) lib/phusion_passenger/spawn_manager.rb:264:in `spawn_rack_application'
  passenger (3.0.21) lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'
  passenger (3.0.21) lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
  passenger (3.0.21) lib/phusion_passenger/abstract_server.rb:357:in `__send__'
  passenger (3.0.21) lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
  passenger (3.0.21) lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
  passenger (3.0.21) helper-scripts/passenger-spawn-server:102
[redmine-dm.stg.rhcloud.com 53a408e8dbd93c5f25001dcd]\> cat app-root/logs/ruby.log 
[Mon Jul 14 02:15:34 2014] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c2,c241
[Mon Jul 14 02:15:34 2014] [notice] Digest: generating secret for digest authentication ...
[Mon Jul 14 02:15:34 2014] [notice] Digest: done
[Mon Jul 14 02:15:34 2014] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations
- - - [14/Jul/2014:02:15:36 -0400] "HEAD / HTTP/1.1" 200 - "-" "-"
- - - [14/Jul/2014:02:15:36 -0400] "HEAD / HTTP/1.1" 200 - "-" "-"
- - - [14/Jul/2014:02:16:13 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
- - - [14/Jul/2014:02:16:14 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
*** Exception SignalException in Class (SIGABRT) (process 171918, thread #<Thread:0x7f4479541360>):
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:444:in `install_useful_signal_handlers'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:462:in `call'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:462:in `select'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:462:in `accept_and_process_next_request'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:206:in `start_request_handler'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:79:in `spawn_application'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/utils.rb:470:in `safe_fork'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:64:in `spawn_application'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:264:in `spawn_rack_application'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/abstract_server.rb:357:in `__send__'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
    from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.21/helper-scripts/passenger-spawn-server:102
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:28:41 -0400] "GET / HTTP/1.1" 200 2889 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:29:16 -0400] "GET /stylesheets/jquery/jquery-ui-1.9.2.css?1403592691 HTTP/1.1" 200 27532 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:29:17 -0400] "GET /stylesheets/application.css?1403592691 HTTP/1.1" 200 47460 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:29:17 -0400] "GET /javascripts/application.js?1403592691 HTTP/1.1" 200 16823 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:29:17 -0400] "GET /favicon.ico?1403592691 HTTP/1.1" 200 7886 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:29:17 -0400] "GET /javascripts/jquery-1.8.3-ui-1.9.2-ujs-2.0.3.js?1403592691 HTTP/1.1" 200 337799 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
127.0.0.1 - - [14/Jul/2014:02:48:05 -0400] "HEAD / HTTP/1.1" 500 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:02:48:47 -0400] "HEAD / HTTP/1.1" 500 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:02:48:47 -0400] "HEAD / HTTP/1.1" 500 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:02:48:47 -0400] "HEAD / HTTP/1.1" 500 - "-" "Ruby"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:51:34 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:52:29 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:54:06 -0400] "GET /favicon.ico HTTP/1.1" 200 7886 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
- - - [14/Jul/2014:02:54:06 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:02:56:42 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
- - - [14/Jul/2014:02:59:57 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
- - - [14/Jul/2014:02:59:57 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
[Mon Jul 14 02:59:57 2014] [notice] caught SIGWINCH, shutting down gracefully
[Mon Jul 14 03:07:53 2014] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c2,c241
[Mon Jul 14 03:07:54 2014] [notice] Digest: generating secret for digest authentication ...
[Mon Jul 14 03:07:54 2014] [notice] Digest: done
[Mon Jul 14 03:07:54 2014] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations
- - - [14/Jul/2014:03:07:55 -0400] "HEAD / HTTP/1.1" 200 - "-" "-"
- - - [14/Jul/2014:03:09:05 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
- - - [14/Jul/2014:03:07:56 -0400] "HEAD / HTTP/1.1" 200 - "-" "-"
- - - [14/Jul/2014:03:09:06 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:03:10:09 -0400] "GET / HTTP/1.1" 200 2889 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:03:10:21 -0400] "GET /stylesheets/jquery/jquery-ui-1.9.2.css?1405321167 HTTP/1.1" 200 27532 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:03:10:21 -0400] "GET /javascripts/jquery-1.8.3-ui-1.9.2-ujs-2.0.3.js?1405321167 HTTP/1.1" 200 337799 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:03:10:21 -0400] "GET /stylesheets/application.css?1405321167 HTTP/1.1" 200 47460 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:03:10:21 -0400] "GET /favicon.ico?1405321167 HTTP/1.1" 200 7886 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:03:10:21 -0400] "GET /javascripts/application.js?1405321167 HTTP/1.1" 200 16823 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.83.41.128 - - [14/Jul/2014:03:10:40 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
10.83.41.128 - - [14/Jul/2014:03:10:41 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:03:15:42 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:03:15:42 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:03:15:42 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:03:15:43 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:03:18:05 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
10.83.41.128 - - [14/Jul/2014:04:10:23 -0400] "HEAD / HTTP/1.1" 500 - "-" "Ruby"
10.83.41.128 - - [14/Jul/2014:04:10:23 -0400] "HEAD / HTTP/1.1" 500 - "-" "Ruby"
[Mon Jul 14 04:15:21 2014] [notice] SIGHUP received.  Attempting to restart
[Mon Jul 14 04:15:21 2014] [notice] Digest: generating secret for digest authentication ...
[Mon Jul 14 04:15:21 2014] [notice] Digest: done
[Mon Jul 14 04:15:21 2014] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations
- - - [14/Jul/2014:04:15:23 -0400] "HEAD / HTTP/1.1" 500 - "-" "-"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:37:42 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:37:43 -0400] "GET /favicon.ico HTTP/1.1" 200 7886 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:37:45 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
[Mon Jul 14 04:38:20 2014] [notice] caught SIGWINCH, shutting down gracefully
- - - [14/Jul/2014:04:38:20 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
- - - [14/Jul/2014:04:38:20 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
[Mon Jul 14 04:43:17 2014] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c2,c241
[Mon Jul 14 04:43:21 2014] [notice] Digest: generating secret for digest authentication ...
[Mon Jul 14 04:43:21 2014] [notice] Digest: done
[Mon Jul 14 04:43:21 2014] [notice] Apache/2.2.15 (Unix) Phusion_Passenger/3.0.21 configured -- resuming normal operations
- - - [14/Jul/2014:04:43:21 -0400] "HEAD / HTTP/1.1" 200 - "-" "-"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:43:42 -0400] "GET / HTTP/1.1" 200 2889 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
- - - [14/Jul/2014:04:43:24 -0400] "HEAD / HTTP/1.1" 200 - "-" "-"
- - - [14/Jul/2014:04:47:41 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:47:42 -0400] "GET /stylesheets/jquery/jquery-ui-1.9.2.css?1405327085 HTTP/1.1" 200 27532 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:47:42 -0400] "GET /stylesheets/application.css?1405327085 HTTP/1.1" 200 47460 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:47:42 -0400] "GET /favicon.ico?1405327085 HTTP/1.1" 200 7886 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:47:42 -0400] "GET /javascripts/jquery-1.8.3-ui-1.9.2-ujs-2.0.3.js?1405327085 HTTP/1.1" 200 337799 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:47:42 -0400] "GET /javascripts/application.js?1405327085 HTTP/1.1" 200 16823 "http://redmine-dm.stg.rhcloud.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
- - - [14/Jul/2014:04:47:42 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:04:49:06 -0400] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
127.0.0.1 - - [14/Jul/2014:04:57:01 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:04:57:01 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:04:57:01 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
127.0.0.1 - - [14/Jul/2014:04:57:02 -0400] "HEAD / HTTP/1.1" 200 - "-" "Ruby"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:05:02:20 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:05:06:06 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
- - - [14/Jul/2014:05:10:31 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
10.66.136.75, 66.187.233.202 - - [14/Jul/2014:05:08:24 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
- - - [14/Jul/2014:05:10:32 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
10.66.136.5, 66.187.233.202 - - [14/Jul/2014:05:08:12 -0400] "GET / HTTP/1.1" 500 674 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Firefox/17.0"
- - - [14/Jul/2014:05:10:43 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
Comment 6 Qixuan Wang 2014-07-15 02:10:44 UTC 
Can't reproduce on STG, so close it.