Bug 969098 - Cartridge control scripts should not overwrite default httpd_nolog.conf with PassEnv directives
Summary: Cartridge control scripts should not overwrite default httpd_nolog.conf with ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: ---
Assignee: Vojtech Vitek
QA Contact: libra bugs
URL:
Whiteboard:
: 1006665 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-05-30 16:06 UTC by Vojtech Vitek
Modified: 2015-05-14 23:20 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-09-19 16:46:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Vojtech Vitek 2013-05-30 16:06:48 UTC
Description of problem:
Cartridge control scripts appends PassEnv directives to the default httpd_nolog.conf file. This could cause a potential troubles when we want update the httpd_nolog.conf file in the future. I believe that file should be even locked, by the way.

Steps to Reproduce:
1. rhc app-restart <app>
or
1. rhc app-start <app>

Actual results:
Cartridge control scripts call
update_httpd_passenv $HTTPD_CFG_FILE
and overwrites the default httpd_nolog.conf config file.

Expected results:
Cartridge control script should call
update_httpd_passenv $HTTPD_PASSENV_CFG_FILE
instead and thus output the PassEnv directives into the separate config file, eg. ./configuration/etc/conf.d/PassEnv.conf

Comment 1 Vojtech Vitek 2013-06-05 15:14:42 UTC
During the update, we should recreate the httpd_nolog.conf to be sure it's not malformed.

Comment 3 Andy Grimm 2013-07-24 18:11:12 UTC
We've seen at least one instance where an OpenShift Online user's httpd_nolog.conf was truncated and populated with only PassEnv lines, and there's a thread in the forums which looks related:

https://www.openshift.com/forums/openshift/after-your-update-my-application-is-still-not-available

The most concerning thing for me is that the problem manifests in a way that completely confuses the user. They make no changes to their app, maybe add a cartridge, restart their app, and suddenly apache is broken, with errors like:

Syntax error on line 7 of /var/lib/openshift//php/configuration/etc/conf.d/openshift.conf: Invalid command 'CustomLog', perhaps misspelled or defined by a module not included in the server configuration

It takes a good bit of knowledge and investigation for a user to realize that their httpd configuration has been destroyed by the control scripts, and once they've determined that, they don't know how to fix it.  

( For those reading here who are experiencing the issue, the workaround is to get a copy of httpd_nolog.conf from the appropriate cartridge, e.g., https://github.com/openshift/origin-server/blob/master/cartridges/openshift-origin-cartridge-php/versions/shared/configuration/etc/conf/httpd_nolog.conf , and insert those contents into the top of httpd_nolog.conf in your gear's config directory for that cartridge, e.g., ~/php/configuration/etc/conf.d/ )

Comment 5 Andy Grimm 2013-07-25 01:20:27 UTC
A quick follow-up on this.  Whatever caused this problem may have been isolated to one particular node.  About 10 different gears (~15 cartridges total) had this problem on one particular node, which makes me think some thing happened during the release/upgrade process on that node.  I've fixed the affected cartridges, and will watch for this to crop up again.  Looking forward to the implementation of vvitek's suggestion, which would be much safer.

Comment 6 Andy Grimm 2013-07-26 19:00:10 UTC
I found 20 more apps with this problem on various OpenShift Online nodes in a couple of cases, I found zero-length files (httpd.conf, httpd_nolog.conf, and php.ini) in phpmyadmin/versions/shared/conf, which sounds like another issue entirely.  I'll open another bug when I have a better understanding of that, but for now, just wanted to note that here, since it results in the same end-user experience.

Comment 7 pesobility 2013-08-09 00:52:25 UTC
Help! pesobility.com is down again when I woke up, it happens everytime my node is restarted by the system (probably maintenance?). Also posted it in https://www.openshift.com/forums/openshift/my-node-js-app-suddenly-stopped-working-mongodb-error-related-i-think#comment-33110

Comment 8 Vojtech Vitek 2013-09-03 16:25:11 UTC
@Andy Grimm, I agree that the malformed files described in comment 6 are different issue, probably caused during the migration.

The only files that could be malfored because of this bug are:
$OPENSHIFT_PERL_DIR/etc/conf/httpd_nolog.conf
$OPENSHIFT_PHP_DIR/configuration/etc/conf/httpd_nolog.conf
$OPENSHIFT_PYTHON_DIR/etc/conf/httpd_nolog.conf
$OPENSHIFT_RUBY_DIR/etc/conf/httpd_nolog.conf

Comment 9 Vojtech Vitek 2013-09-05 15:41:28 UTC
https://github.com/openshift/origin-server/pull/3552

Comment 10 Yan Du 2013-09-12 08:40:26 UTC
test on devenv_3776 with php/perl/ruby/python cartridge

steps:
1. create an app
2. ssh into app to check httpd_nolog.conf and passenv.conf files
3. rhc app restart $appname
4. check httpd_nolog.conf and passenv.conf files again

after restart the app, passenv.conf will be updated, and the httpd_nolog.conf will not be changed.

move the bug to verified.

Comment 12 Yan Du 2013-09-13 09:21:39 UTC
Hi, Vojtech Vitek

 test php/perl/python app after the upgrade.


1. for old perl/php app, both httpd_nolog.conf and passenv.conf are existed in app, after restart, httpd_nolog.conf will not update, just passenv.conf updated

2. for python-2.6/python-2.7 old app, only httpd_nolog.conf existed in app (no passenv.conf file existed), after restart, httpd_nolog.conf will not update, no new passenv.conf file created.

3. for python-3.3 old and new app, httpd_nolog.conf and passenv.conf are not existed in app, after restart, no change.

4. for ruby app, it is blocked by the #bug 1007730, will test after #bug 1007730 fixed.

Comment 13 Vojtech Vitek 2013-09-19 13:22:00 UTC
*** Bug 1006665 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.