Bug 1001400

Hi,

As far as I investigated this I think the problem is with php.ini file where we set the memory limit too low:

See $OPENSHIFT_PHP_DIR/configuration/etc/php.ini:

; Maximum amount of memory a script may consume (128MB)
; http://www.php.net/manual/en/ini.core.php#ini.memory-limit
memory_limit = 128M


I'm not 100% sure if users can modify the 'php.ini' file and set different limits. Vojta?

Seems like the APC in your case reaches it's memory limit:
apc.shm_size=64M

I will investigate both apc.shm_size and apc.ttl settings and report back if we can prevent this. Increasing APC shared memory size doesn't look like the only thing to change.

Assuming you're using Drupal 7 with APC module enabled, as a workaround, I suggest you changing your CACHE backends in your settings.php config file appropriately to reduce the APC memory usage:

$conf['cache_default_class'] = 'DrupalDatabaseCache';
$conf['cache_class_cache'] = 'DrupalAPCCache';
$conf['cache_class_cache_bootstrap'] = 'DrupalAPCCache';

Reproducer - index.php:
<?php
$data = array();
for ($i = 0; $i < 100; $i++) {
  $data=array_merge($data,$GLOBALS);
}
for ($i = 0; ; $i++) {
  if (apc_store("foo$i", $data))
    echo "$i+\n";
  else
    echo "$i-\n";
  $tmp = apc_fetch("foo$i");
}

Created attachment 799485 [details]
apc.php - statistics

https://github.com/openshift/origin-server/pull/3661 - a related PR to enable custom APC settings.

Workaround (git):
cd <app-git>
echo "php_flag apc.cache_by_default Off" > php/.htaccess
git add php/.htaccess
git commit -am 'Turn off APC cache by default'
git push

Quick workaround on gear (next git push will overwrite this):
rhc ssh <app> 'echo "php_flag apc.cache_by_default Off" > $OPENSHIFT_HOMEDIR/app-root/repo/php/.htaccess'

#8 didn't work for me.
I guess in order to disable we should use another variable apc.enabled, that is:

php_flag apc.enabled off

Nevertheless I went to find a working solution, which doesn't disable APC.

**Solution**
Here is the couple of settings I've added to my .htaccess:

# Give more memory.
php_value memory_limit 512M
# APC settings.
php_flag apc.enabled on
php_flag apc.cache_by_default on
php_value apc.shm_segments 1
#32M per OpenShift install.
php_value apc.shm_size 128M
#This MUST be 0, OpenShift can have errors otherwise!
php_flag apc.include_once_override off
#Leave at 2M or lower. Drupal seems not having any file sizes close to 2M.
php_value apc.max_file_size 2M

As you could guess, the key is apc.include_once_override, which is described as EXPERIMENTAL at http://www.php.net/manual/en/apc.configuration.php#ini.apc.include-once-override - so we definitely should avoid using it at production.

These settings completely get rid of annoying warnings - yay!
Please review the solution - and correct the default php.ini settings in order to fix the bug for others.
@assignee Please let us know when it is going to be released.

**Related links**
- https://drupal.org/node/807680 - apc.include_once_override caused another problem in another module.
- https://bugs.php.net/bug.php?id=58469 - PHP bug discussion, concluding that apc.include_once_override should be disabled.

Forgot to mention that my solution is based on (almost stolen from) this link: http://gregrickaby.com/the-perfect-apc-configuration . Thanks Gregory for nice post, describing how to configure APC at AWS, which is very much related to our use case.

I was wrong. According to https://bugs.php.net/bug.php?id=57064 and http://www.php.net/manual/ru/ini.list.php it becomes clear that the only APC setting which could be customized via .htaccess is "apc.cache_by_default".
Setting it to Off did the trick for me - though it means disabling, because "apc.filters" is nil.
My vote is for increasing "apc.shm_size" globally. I am going to dig the opportunity mentioned by @Vojtech in #7.

Moving this bug back to Vojtech, as the APC was not part of the cartridge tuning. APC is specific PHP extension that might/might be not enabled or used in application. Also there are known workarounds currently.

Confirm that you can't set the above in .htaccess http://www.php.net/manual/en/apc.configuration.php so it is just a case of switching it off in .htaccess 
<code>
  # APC settings (which can be used depends on version)
  php_flag apc.enabled off
  php_flag apc.cache_by_default off
</code>
Does seem that the cache size just wants to be increased. But it also doesn't seem that you can change the system values - due to permissions. Guess it can just be changed in the cartridge? Or may to be a variable read from somewhere? 

And can confirm from experience that this is an issue with APC as it is now configured and Drupal as soon as you've got a decent number of modules enabled.

Trello cards for APC:
1. Disable/Edit APC settings manually: https://trello.com/c/1VcpoZTf
2. Disable APC automatically via DEVELOPMENT MODE: https://trello.com/c/lbjDOA6S

Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/b149d7e27c485ab91ba71ed509b6b28535e14c43
Copy PHP extensions INI files to the cartridge code for maintainability

- Copy RHEL PHP 5.3 RPM /etc/php.d/*.ini files
- Copy RHEL PHP 5.4 SCL /opt/rh/php54/root/etc/php.d/*.ini files
- Lock all the INI files by default (not editable in userspace)

Motivation:
- We want to be able to enable/disable PHP extensions in gears
- We want to be able to edit PHP extensions settings in gears
- We want consistency for all the INI files across all nodes.
  Currently, the INI files are marked %config(noreplace) in RPMs
  and may theoretically differ from node to node.

PHP_INI_SCAN_DIR (PHP/PECL extensions):
- Set PHP_INI_SCAN_DIR env var to scan customized INI files
  from ${OPENSHIFT_PHP_DIR}etc/php.d/ directory
  instead of the default system's scan dir (eg. /etc/php.d/)

https://trello.com/c/lbjDOA6S - origin_cartridge_20
https://trello.com/c/1VcpoZTf - origin_cartridge_140
https://trello.com/c/iDrhIUof - origin_cartridge_136
https://bugzilla.redhat.com/show_bug.cgi?id=1001400 - bug 1001400

This will get addressed by https://trello.com/c/1VcpoZTf functionality during this sprint:

> - APC is enabled by default
> - Users can enable/disable APC by running
>   `rhc env-set OPENSHIFT_PHP_APC_ENABLED=false --app <app>`
>   and `rhc app-restart <app>` commands
> 
> APC shared memory size:
> - Formula: GEAR_MEMORY/8
>   - 64M on Small gears by default
>   - 128M on Medium gears by default
>   - 256M on Large gears by default
> - Users can change the APC shared memory size by running
>   `rhc env-set OPENSHIFT_PHP_APC_SHM_SIZE=128M --app <app>`
>   and `rhc app-restart <app>` commands
> 
> New environment variables:
> - $OPENSHIFT_PHP_APC_ENABLED=true/false
> - $OPENSHIFT_PHP_APC_SHM_SIZE={64,128,256}M