Bug 1004122

Hi @xianrenb, could you please provide the content of your deplist.txt file? Also, could you please describe the full steps to reproduce this issue? I couldn't reproduce it with the provided steps yet.

Hi @Vojtech Vitek. I am sorry that I did not give enough information previously.

The "deplist.txt" file is only a single line "pear.apigen.org/apigen".

Related lines in the "build" action hook script are as follows:
#start
pushd ${OPENSHIFT_HOMEDIR}php/phplib/pear/pear/php
../apigen --source ${OPENSHIFT_REPO_DIR}php --destination ${OPENSHIFT_DATA_DIR}docs
ln -sf ${OPENSHIFT_DATA_DIR}docs ${OPENSHIFT_REPO_DIR}php
popd
#end

The action hook sript worked before the recent OpenShift Online upgrade. I believe the include path settings for using PEAR (within action hook script) changed after the upgrade.

Thanks.

My steps to reproduce:

rhc app create p php-5.3 && cd p
echo "pear.apigen.org/apigen" > deplist.txt
git add deplist.txt 
git commit -amdeplist
git push
> remote: Stopping PHP cartridge
> remote: Building PHP cartridge
> remote: Checking pear: pear.apigen.org/apigen
> remote: 
> remote: Attempting to discover channel "pear.apigen.org"...
> remote: downloading channel.xml ...
> remote: Starting to download channel.xml (802 bytes)
> remote: ....done: 802 bytes
> remote: Auto-discovered channel "pear.apigen.org", alias "apigen", adding to registry
> remote: Attempting to discover channel "pear.nette.org"...
> remote: downloading channel.xml ...
> remote: Starting to download channel.xml (812 bytes)
> remote: ...done: 812 bytes
> remote: Auto-discovered channel "pear.nette.org", alias "nette", adding to registry
> remote: Attempting to discover channel "pear.texy.info"...
> remote: downloading channel.xml ...
> remote: Starting to download channel.xml (810 bytes)
> remote: ...done: 810 bytes
> remote: Auto-discovered channel "pear.texy.info", alias "texy", adding to registry
> remote: Attempting to discover channel "pear.kukulich.cz"...
> remote: downloading channel.xml ...
> remote: Starting to download channel.xml (811 bytes)
> remote: ...done: 811 bytes
> remote: Auto-discovered channel "pear.kukulich.cz", alias "kukulich", adding to registry
> remote: Attempting to discover channel "pear.andrewsville.cz"...
> remote: downloading channel.xml ...
> remote: Starting to download channel.xml (839 bytes)
> remote: ...done: 839 bytes
> remote: Auto-discovered channel "pear.andrewsville.cz", alias "andrewsville", adding to registry
> remote: nette/Nette can optionally use PHP extension "memcache"
> remote: downloading ApiGen-2.8.0.tgz ...
> remote: Starting to download ApiGen-2.8.0.tgz (188,843 bytes)
> remote: ...done: 188,843 bytes
> remote: downloading Nette-2.0.12.tgz ...
> remote: Starting to download Nette-2.0.12.tgz (229,712 bytes)
> remote: ...done: 229,712 bytes
> remote: downloading Texy-2.3.0.tgz ...
> remote: Starting to download Texy-2.3.0.tgz (46,499 bytes)
> remote: ...done: 46,499 bytes
> remote: downloading FSHL-2.1.0.tgz ...
> remote: Starting to download FSHL-2.1.0.tgz (65,593 bytes)
> remote: ...done: 65,593 bytes
> remote: downloading TokenReflection-1.3.1.tgz ...
> remote: Starting to download TokenReflection-1.3.1.tgz (62,720 bytes)
> remote: ...done: 62,720 bytes
> remote: install ok: channel://pear.nette.org/Nette-2.0.12
> remote: install ok: channel://pear.texy.info/Texy-2.3.0
> remote: install ok: channel://pear.kukulich.cz/FSHL-2.1.0
> remote: install ok: channel://pear.andrewsville.cz/TokenReflection-1.3.1
> remote: install ok: channel://pear.apigen.org/ApiGen-2.8.0
> remote: Starting application p
> remote: Starting PHP cartridge

I couldn't reproduce the issue yet.

Can you please manually verify your installed PEARs? Seems like you could have troubles with installing one of the dependencies (Texy) during the install stage, for example because of PEAR mirror servers that could have been down at that time or something..

You can also try clearing your PEAR directory (./php/phplib/) and start building from scratch to be able to provide the full PEAR install log.

Check this issue with steps in Comment 3 on devenv_3776, it's not reproduced, so verify it.

I have just tried to clear the ~/php/phplib/ directory and tested:

On gear:
[myquicknet-xianrenb.rhcloud.com phplib]\> pwd
/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib
[myquicknet-xianrenb.rhcloud.com phplib]\> rm -rf *
[myquicknet-xianrenb.rhcloud.com phplib]\> ls



On my local git repository/machine:
$ touch dummy
$ ls
deplist.txt  dummy  libs  misc  php  README

$ git add dummy
$ git commit -m 'test PEAR'
[master 6a5f15f] test PEAR
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 dummy

$ git push
Counting objects: 3, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 258 bytes, done.
Total 2 (delta 1), reused 0 (delta 0)
remote: Stopping PHP cartridge
remote: Waiting for stop to finish
remote: Waiting for stop to finish
remote: Waiting for stop to finish
remote: Stopping MySQL cartridge
remote: Stopping PHPMyAdmin cartridge
remote: [Thu Sep 12 20:18:06 2013] [warn] module php5_module is already loaded, skipping
remote: Waiting for stop to finish
remote: /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/app-root/data /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1
remote: Cloning into 'jsdoc'...
remote: /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1
remote: Building PHP cartridge
remote: Checking pear: pear.apigen.org/apigen
remote:
remote: Attempting to discover channel "pear.apigen.org"...
remote: downloading channel.xml ...
remote: Starting to download channel.xml (802 bytes)
remote: ....done: 802 bytes
remote: Auto-discovered channel "pear.apigen.org", alias "apigen", adding to registry
remote: Attempting to discover channel "pear.nette.org"...
remote: downloading channel.xml ...
remote: Starting to download channel.xml (812 bytes)
remote: ...done: 812 bytes
remote: Auto-discovered channel "pear.nette.org", alias "nette", adding to registry
remote: Attempting to discover channel "pear.texy.info"...
remote: downloading channel.xml ...
remote: Starting to download channel.xml (810 bytes)
remote: ...done: 810 bytes
remote: Auto-discovered channel "pear.texy.info", alias "texy", adding to registry
remote: Attempting to discover channel "pear.kukulich.cz"...
remote: downloading channel.xml ...
remote: Starting to download channel.xml (811 bytes)
remote: ...done: 811 bytes
remote: Auto-discovered channel "pear.kukulich.cz", alias "kukulich", adding to registry
remote: Attempting to discover channel "pear.andrewsville.cz"...
remote: downloading channel.xml ...
remote: Starting to download channel.xml (839 bytes)
remote: ...done: 839 bytes
remote: Auto-discovered channel "pear.andrewsville.cz", alias "andrewsville", adding to registry
remote: nette/Nette can optionally use PHP extension "memcache"
remote: downloading ApiGen-2.8.0.tgz ...
remote: Starting to download ApiGen-2.8.0.tgz (188,843 bytes)
remote: ...done: 188,843 bytes
remote: downloading Nette-2.0.12.tgz ...
remote: Starting to download Nette-2.0.12.tgz (229,712 bytes)
remote: ...done: 229,712 bytes
remote: downloading Texy-2.3.0.tgz ...
remote: Starting to download Texy-2.3.0.tgz (46,499 bytes)
remote: ...done: 46,499 bytes
remote: downloading FSHL-2.1.0.tgz ...
remote: Starting to download FSHL-2.1.0.tgz (65,593 bytes)
remote: ...done: 65,593 bytes
remote: downloading TokenReflection-1.3.1.tgz ...
remote: Starting to download TokenReflection-1.3.1.tgz (62,720 bytes)
remote: ...done: 62,720 bytes
remote: install ok: channel://pear.nette.org/Nette-2.0.12
remote: install ok: channel://pear.texy.info/Texy-2.3.0
remote: install ok: channel://pear.kukulich.cz/FSHL-2.1.0
remote: install ok: channel://pear.andrewsville.cz/TokenReflection-1.3.1
remote: install ok: channel://pear.apigen.org/ApiGen-2.8.0
remote: /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1
remote: PHP Warning:  require(Texy.php): failed to open stream: No such file or directory in /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen on line 48
remote: PHP Stack trace:
remote: PHP   1. {main}() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:0
remote: PHP   2. class_exists() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:64
remote: PHP   3. ApiGen\{closure}() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:0
remote: PHP Fatal error:  require(): Failed opening required 'Texy.php' (include_path='.:/usr/share/pear:/usr/share/php') in /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen on line 48
remote: PHP Stack trace:
remote: PHP   1. {main}() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:0
remote: PHP   2. class_exists() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:64
remote: PHP   3. ApiGen\{closure}() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:0
remote: /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1
remote: /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/app-root/data /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1
remote: /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1
remote: Starting application myquicknet
remote: Starting MySQL cartridge
remote: Starting PHPMyAdmin cartridge
remote: [Thu Sep 12 20:19:14 2013] [warn] module php5_module is already loaded, skipping
remote: Starting PHP cartridge
To ssh://1786cb06c4314cd4b346e9f0ac47b8d1.com/~/git/myquicknet.git/
   e472dbe..6a5f15f  master -> master



Also on gear:
[myquicknet-xianrenb.rhcloud.com 1786cb06c4314cd4b346e9f0ac47b8d1]\> cd ~/php/phplib/pear/pear/php/
[myquicknet-xianrenb.rhcloud.com php]\> ../apigen --help
PHP Warning:  require(Texy.php): failed to open stream: No such file or directory in /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen on line 48
PHP Stack trace:
PHP   1. {main}() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:0
PHP   2. class_exists() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:64
PHP   3. ApiGen\{closure}() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:0
PHP Fatal error:  require(): Failed opening required 'Texy.php' (include_path='.:/usr/share/pear:/usr/share/php') in /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen on line 48
PHP Stack trace:
PHP   1. {main}() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:0
PHP   2. class_exists() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:64
PHP   3. ApiGen\{closure}() /var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/apigen:0


The last check confirmed that ApiGen could not be used on the gear. Something must have been changed since the last upgrade of OpenShift Online.

P.S.
I have just found out the followings on my gear:
[myquicknet-xianrenb.rhcloud.com php]\> pwd
/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php
[myquicknet-xianrenb.rhcloud.com php]\> ls
ApiGen  FSHL  Nette  readme.txt  texy  TokenReflection
[myquicknet-xianrenb.rhcloud.com php]\> ls texy
src
[myquicknet-xianrenb.rhcloud.com php]\> ls texy/src/
Texy  texy.php



"Texy" should be loaded using the following line in the "../apigen" script:
@include '~/php/phplib/pear/pear/php/Texy/texy.php';


I don't know why the PEAR system make the path of downloaded files changing from "Texy" to "texy/src"!

Hope the above information could lead to fixing the problem.
Thank you.

I have just checked "https://github.com/apigen/apigen/issues/253". The reason that my action hook script failed is that Texy upgraded from version 2.1 to 2.3 and the directory structure changed, while ApiGen (2.8.0) did not upgrade to fix the change. It should NOT be the problem of OpenShift Online. I am sorry for incorrectly reporting this as a bug.

Although the main reason for ApiGen/Texy not working properly is unrelated to the recent upgrade of OpenShift Online, the current settings of PEAR seems incorrect.

I checked the content of the mentioned "../apigen" action hook script, and found out the following lines:
        @include '~/php/phplib/pear/pear/php/Nette/loader.php';
        @include '~/php/phplib/pear/pear/php/Texy/texy.php';



However, the correct settings of PEAR should give:
        @include '/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php/Nette/loader.php';
        @include '/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php/Texy/texy.php';



I believe PHP does not know how to expand "~/":
[myquicknet-xianrenb.rhcloud.com php]\> php -r 'include "/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php/texy/src/texy.php";'
[myquicknet-xianrenb.rhcloud.com php]\> php -r 'include "~/php/phplib/pear/pear/php/texy/src/texy.php";'
PHP Warning:  include(~/php/phplib/pear/pear/php/texy/src/texy.php): failed to open stream: No such file or directory in Command line code on line 1
PHP Stack trace:
PHP   1. {main}() Command line code:0
PHP Warning:  include(): Failed opening '~/php/phplib/pear/pear/php/texy/src/texy.php' for inclusion (include_path='.:/usr/share/pear:/usr/share/php') in Command line code on line 1
PHP Stack trace:
PHP   1. {main}() Command line code:0



Hope the above information could help improving OpenShift.

As showed in my last comment, PHP does not know how to expand '~/'. However, OpenShift is a cloud environment, where the code need to copy/deploy to multiple machine, meaning the path (in code) should be the same for multiple gears. I would suggest redesigning the (PEAR) environment, possibly using path names matching the application's URL, e.g. '/var/lib/openshift/app_data/myapp-mynamespace/php/pear/'. This would allow direct copying/deploying code to different gears.

@xianrenb thanks for your feedback.

Closing as NOTABUG.

I have checked the "package.xml" file for the ApiGen 2.8.0 PEAR package. The configuration would replace from "@php_dir@" to PEAR configuration variable "php_dir" for the file "apigen.php", which would be installed as the "../apigen" script mentioned earlier. Which means "php_dir" for "deplist.txt" to work currently is "~/php/phplib/pear/pear/php", but changed to, and it needs to be, "/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php" for the action hook script "build" (in my gear) to work. In other words, the PEAR configuration for "deplist.txt" to work is WRONG, and it needs to change to the same settings as what the action hook script would have.

The current configuration should affect all PHP PEAR package which use the PEAR configuration variable "php_dir" to modify PHP scripts when installing PEAR package.

Previously, I thought the UUID in the path is different for different gears. But if the UUID in the path is the same for all gears sharing the same application (which should be the fact), it should be ok.

The bottom line is, the PEAR configuration should change to be the same for both "deplist.txt" and action hook scripts to work.

Perhaps my explanation before was not clear.

I know ApiGen 2.8 does not work with Texy 2.3, but there is something more to look at.

The "deplist.txt" was set to install ApiGen 2.8.0 package.
The original include lines in the "../apigen" script before modification made by PEAR installation are:
        @include '@php_dir@/Nette/loader.php';
        @include '@php_dir@/Texy/texy.php';



But it was modified to:
        @include '~/php/phplib/pear/pear/php/Nette/loader.php';
        @include '~/php/phplib/pear/pear/php/Texy/texy.php';



However, it needs to be:
        @include '/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php/Nette/loader.php';
        @include '/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php/Texy/texy.php';



I would guess the PEAR configuration during the time that "deplist.txt" effects is wrong, as the modification would depend on PEAR configuration variable "php_dir", where "php_dir" appeared to be "~/php/phplib/pear/pear/php", but the correct settings should be "/var/lib/openshift/1786cb06c4314cd4b346e9f0ac47b8d1/php/phplib/pear/pear/php" (for my gear). However, the PEAR configuration for action hook scripts seems ok.

I am trying to change status to ASSIGNED to reopen this bug.

@xianrenb thank you for your detailed explanation.

So, it seems like php-cli does not handle tilde expansion, as you demonstrated in the comment 11. We're going to address this issue.

Issue:

When building PEAR, the absolute path to the pear root directory is built into [1] the resulting library (set of PHP files). This means that every PEAR is tight to the builder gear. There was no real fix for this in v2 (unless we got rid of gear-specific /var/lib/openshift/UUID/ directory), so we had to apply the following workaround: https://github.com/openshift/origin-server/blob/master/cartridges/openshift-origin-cartridge-php/bin/control#L123
The above workaround actually substitutes the gear $OPENSHIFT_HOME absolute path with the tilde '~/' path in all of the PEAR files. We found out that this works for most of the cases, but fails for some scenarios (PHP itself can't handle the tilde characters in require_once() and include_once() functions).

We can't really solve this issue (without any further nasty workarounds) unless we have full access to the /root directory. So this will get fixed in v3 naturally.

@Ben, are you ok with WONTFIX-ing this bug or should we still try to fix this in v2?

Vojtech:  I think I need a little better understanding of the issue, can you ping me on IRC next time you think of it?

Know limitation of OpenShift 2.x:

When the 'pear install' is called in the context of '/var/lib/openshift/<UUID>/php/phplib/pear' directory, the PEAR's build is triggered, and the previously mentioned path is "baked" into some of the resulting PEAR .php files. Unfortunately, PEAR lacks of configuration option to change the absolute path to a relative path or to something else.
When scaling such PHP gears or when using Jenkins builder, the above mentioned .php files are copied over to the other gears, where the "baked" absolute path doesn't work. This causes "File not found" errors. OpenShift 2.x works-arounds this with https://github.com/openshift/origin-server/blob/08000dcb8fd777b5cb7e433504370fbe8e17e7dc/cartridges/openshift-origin-cartridge-php/bin/control#L130 (this changes the absolute paths to relative path starting with tilde character, eg. '~/php/phplib/pear/..'), which works for some cases, but introduces the reported tilde error when using php-cli.

This can't be fixed unless PEAR supports building the resulting .php files using relative path or using the injected variable with the content of $php_dir variable.
> $ pear config-get php_dir

Workaround using Composer dependency manager:

1. Go to your application git repository.

2. Copy/paste the below commands:

cat <<EOF > composer.json
{
  "require": {
    "apigen/apigen": "*"
  }
}
EOF
mkdir -p .openshift/markers
touch .openshift/markers/use_composer
git add composer.json .openshift/markers/use_composer
git commit -am'Add apigen Composer dependency'
git push

3. Now, you can invoke the apigen executable in your gear (both while ssh'd or from the action hooks) using:

$OPENSHIFT_REPO_DIR/vendor/bin/apigen