Description: a. After apps created on node3 cannot be accessed even after restart, including migrated apps and new created apps: http://wordpress54-librat29.stg.rhcloud.com/ http://spy27-libra28t.stg.rhcloud.com/ http://jbossas7-libra28t.stg.rhcloud.com/ http://drupal53-libra28t.stg.rhcloud.com/ http://ews2storage-libra28t.stg.rhcloud.com/ http://wildflys-migrate.stg.rhcloud.com/ http://perldbjks-librat30.rhcloud.com/ http://storageapp-librat29.stg.rhcloud.com/ http://py27-migdm.stg.rhcloud.com/ http://jenkins1-librat29.stg.rhcloud.com/ http://php54s-librat30.stg.rhcloud.com/ http://ruby20dbjks-librat30.stg.rhcloud.com/ http://ruby19sha-librat29.stg.rhcloud.com/ b. Some apps on node1 return 404 after being accessed: https://rubytest1-last.stg.rhcloud.com/ https://wordpress-gusun1.stg.rhcloud.com/ https://ruby-example.stg.rhcloud.com/ https://ruby-example.stg.rhcloud.com/ Version-Release number of selected component (if applicable): STG(devenv-stage_1078) How reproducible: always steps to Reproduce: 1. Create app on node3 2. Acess the app url Actual results: Some app cannot be access by returning below error: The requested URL could not be retrieved Expected results: All should be accessed. Additional info:
Fixed with pull request : https://github.com/openshift/origin-server/pull/5908 Summary - Day one bug. cartridge configure timed out on mcollective side but the thread on the node didnt die. It went on to configure the frontend apache. Meanwhile broker sent an app-destroy which deleted the vhost config directory. Configure thread wakes up to create the conf file and barfs when it finds that the app's base dir is missing (and leaves its conf file behind after throwing up). Fix : before configuring the vhost conf files, check if the dir exists or not.
To fix the node and corresponding broken apps, the solo config file that has its referenced directories deleted needs to be removed manually from the node. In this bug's case, the config files associated with gear id 5449ce66dbd93cfa4d0004c6 on node3.
Verified on devenv_5267 with below steps: 1. Create an app; 2. Delete base dir under/var/lib/openshift/.httpd.d, like 544fc7f127c8334a0300002b_d_app2; 3. Connect the fronted of the app: $oo-devel-node frontend-connect --with-container-uuid 544fc7f127c8334a0300002b 4. Below error appears: Base directory /etc/httpd/conf.d/openshift/544fc7f127c8334a0300002b_d_app2 does not exist for the app: app2-d.dev.rhcloud.com
Reopen this bug since this issue happens again on STG(devenv_stage_1083), applications created on node3 returns 404 error.
rchopra says cannot reproduce.
This is fixed on STG. Both old and new apps can be accessed now.