Bug 977796

Summary: Build hook terminated (gear postreceive)
Product: OpenShift Online Reporter: Warnar Boekkooi <warnar>
Component: TemplatesAssignee: Mrunal Patel <mpatel>
Status: CLOSED NOTABUG QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: agrimm, dmcphers
Target Milestone: ---Keywords: SupportQuestion
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-08-16 19:37:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Warnar Boekkooi 2013-06-25 10:56:39 UTC
When creating a DIY app using https://github.com/boekkooi/openshift-diy-nginx-php the build proccess get's terminated with a 'gear postreceive' while not error occured.

Version-Release number of selected component (if applicable):
openshift.com - diy-0.1

How reproducible:

Steps to Reproduce:
1. rhc app create diy diy --from-code=git://github.com/boekkooi/openshift-diy-nginx-php.git
2. Wait.....

Actual results:
remote: /bin/sh /tmp/build/php-5.5.0/libtool --silent --preserve-dup-deps --mode=compile cc -I/tmp/build/php-5.5.0/ext/gd/libgd -DHAVE_LIBPNG -Iext/gd/ -I/tmp/build/php-5.5.0/ext/gd/ -DPHP_ATOM_INC -I/tmp/build/php-5.5.0/include -I/tmp/build/php-5.5.0/main -I/tmp/build/php-5.5.0 -I/tmp/build/php-5.5.0/ext/date/lib -I/tmp/build/php-5.5.0/ext/ereg/regex -I/usr/include/libxml2 -I/var/lib/openshift/51c7e919e0b8cdbad7000028/diy//runtime//libs/include -I/tmp/build/php-5.5.0/ext/mbstring/oniguruma -I/tmp/build/php-5.5.0/ext/mbstring/libmbfl -I/tmp/build/php-5.5.0/ext/mbstring/libmbfl/mbfl -I/tmp/build/php-5.5.0/ext/sqlite3/libsqlite -I/tmp/build/php-5.5.0/TSRM -I/tmp/build/php-5.5.0/Zend    -I/usr/include -g -O2 -fvisibility=hidden  -c /tmp/build/php-5.5.0/ext/gd/libgd/gd_transform.c -o ext/gd/libgd/gd_transform.lo 
remote: An error occurred executing 'gear postreceive'
remote: 
remote: For more details about the problem, try running the command again with the '--trace' option.

Expected results:
No error


Additional info:

Comment 1 Abhishek Gupta 2013-06-26 17:11:53 UTC
The post receive hook seems to be timing out. Need to take a look at the node logs to figure out the exact issue.

Comment 2 Warnar Boekkooi 2013-06-27 07:38:06 UTC
There is a example app on http://php55-boltconcepts.rhcloud.com/ and a running app on http://boltform-boltconcepts.rhcloud.com/ both had this problem.
If you can access those logs please do or tell me how i can get them to you.

Comment 3 Mrunal Patel 2013-07-01 17:01:15 UTC
Hi Warnar,
How long is the build expected to take?
If it takes too long (>3.5mins), the better approach would be to add upstream
and git push instead.

Thanks,
Mrunal

Comment 4 Warnar Boekkooi 2013-07-01 17:18:34 UTC
Hi Mrunal,

Well it's making php 5.5, nginx ( and nodejs if needed ) so it needs a while.
Sorry but i'm not getting the upstream remark can you explain what you mean?

Regards,
Warnar

(In reply to Mrunal Patel from comment #3)
> Hi Warnar,
> How long is the build expected to take?
> If it takes too long (>3.5mins), the better approach would be to add upstream
> and git push instead.
> 
> Thanks,
> Mrunal

Comment 5 Mrunal Patel 2013-07-01 17:27:03 UTC
Warnar,
I mean using these steps instead of --from-code:

cd diy (dir containing the git repo locally -- should be app name)
git remote add upstream -m master git:boekkooi/openshift-diy-nginx-php.git
git pull -s recursive -X theirs upstream master
git push

Thanks,
Mrunal

Comment 6 Warnar Boekkooi 2013-07-01 20:41:49 UTC
Same problem here is what I did:
```
rhc app create test diy
git clone ssh://51d1d73d5973caae6300006d.com/~/git/test.git/ diy
cd diy
git remote add upstream -m master git:boekkooi/openshift-diy-nginx-php.git
git pull -s recursive -X theirs upstream master
git push
```

Result:
```
remote: /bin/sh /tmp/build/php-5.5.0/libtool --silent --preserve-dup-deps --mode=compile cc -I/tmp/build/php-5.5.0/ext/sqlite3/libsqlite -DSQLITE_ENABLE_FTS3=1 -DSQLITE_CORE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_THREADSAFE=0  -Iext/sqlite3/ -I/tmp/build/php-5.5.0/ext/sqlite3/ -DPHP_ATOM_INC -I/tmp/build/php-5.5.0/include -I/tmp/build/php-5.5.0/main -I/tmp/build/php-5.5.0 -I/tmp/build/php-5.5.0/ext/date/lib -I/tmp/build/php-5.5.0/ext/ereg/regex -I/usr/include/libxml2 -I/var/lib/openshift/51d1d73d5973caae6300006d/diy//runtime//libs/include -I/tmp/build/php-5.5.0/ext/mbstring/oniguruma -I/tmp/build/php-5.5.0/ext/mbstring/libmbfl -I/tmp/build/php-5.5.0/ext/mbstring/libmbfl/mbfl -I/tmp/build/php-5.5.0/ext/sqlite3/libsqlite -I/tmp/build/php-5.5.0/TSRM -I/tmp/build/php-5.5.0/Zend    -I/usr/include -g -O2 -fvisibility=hidden  -c /tmp/build/php-5.5.0/ext/sqlite3/libsqlite/sqlite3.c -o ext/sqlite3/libsqlite/sqlite3.lo
remote: An error occurred executing 'gear postreceive'
remote:
remote: For more details about the problem, try running the command again with the '--trace' option.
To ssh://51d1d73d5973caae6300006d.com/~/git/test.git/
   7e70621..998c29f  master -> master
```

Comment 7 Mrunal Patel 2013-07-01 21:48:47 UTC
Most likely the build is failing. From reading https://github.com/boekkooi/openshift-diy-nginx-php, it looks like the build is expected to take an hour and there are few things that need to be changed before the build works. I would suggest ssh'ing into the gear the trying the build by hand so you can debug any build related issues.

Comment 8 Warnar Boekkooi 2013-07-02 08:05:44 UTC
Well done you found the bug! (if I do another git push after the first one (with one change add a newline some where) it continues to build and finishes correctly.)

The build scripts work correct, the bug/problem is that openshift kills/terminates/timesout while still building (executing `make install` file .openshift/action_hooks/build_php:71).

Comment 9 Warnar Boekkooi 2013-07-02 13:21:55 UTC
Okee so i did some more research and did another build the error changed to:
```
remote: /bin/sh /tmp/build/php-5.5.0/libtool --silent --preserve-dup-deps --mode=compile cc  -Iext/hash/ -I/tmp/build/php-5.5.0/ext/hash/ -DPHP_ATOM_INC -I/tmp/build/php-5.5.0/include -I/tmp/build/php-5.5.0/main -I/tmp/build/php-5.5.0 -I/tmp/build/php-5.5.0/ext/date/lib -I/tmp/build/php-5.5.0/ext/ereg/regex -I/usr/include/libxml2 -I/var/lib/openshift/51d2c3e84382ec2817000152/diy//runtime//libs/include -I/tmp/build/php-5.5.0/ext/mbstring/oniguruma -I/tmp/build/php-5.5.0/ext/mbstring/libmbfl -I/tmp/build/php-5.5.0/ext/mbstring/libmbfl/mbfl -I/tmp/build/php-5.5.0/ext/sqlite3/libsqlite -I/tmp/build/php-5.5.0/TSRM -I/tmp/build/php-5.5.0/Zend    -I/usr/include -g -O2 -fvisibility=hidden  -c /tmp/build/php-5.5.0/ext/hash/hash_tiger.c -o ext/hash/hash_tiger.lo 
remote: Putting child 0x023f24f0 (ext/hash/hash_tiger.lo) PID 10177 on the chain.
remote: Live child 0x023f24f0 (ext/hash/hash_tiger.lo) PID 10177 
remote: An error occurred executing 'gear postreceive' (exit code: -1)
remote: Error message: Shell command '/var/lib/openshift/51d2c3e84382ec2817000152/app-root/runtime/repo/.openshift/action_hooks/build' exceeded timeout of 3598
remote: 
remote: For more details about the problem, try running the command again with the '--trace' option.
To ssh://51d2c3e84382ec2817000152.com/~/git/test.git/
 + a022ae5...12f1b0d master -> master (forced update)
```

It looks like the error "Shell command '/var/lib/openshift/51d2c3e84382ec2817000152/app-root/runtime/repo/.openshift/action_hooks/build' exceeded timeout of 3598" is the problem and that this maybe related to https://github.com/openshift/origin-server/pull/2789

Comment 10 Mrunal Patel 2013-07-02 15:41:14 UTC
How long does the build take?

Comment 11 Warnar Boekkooi 2013-07-02 17:08:44 UTC
It depends on the configuration but on a clean diy with both php and nodejs needing to be build/compiled it takes about 2 and a half hours.

Comment 12 Mrunal Patel 2013-07-02 17:42:02 UTC
A build taking so long isn't really the right approach here. The quickstart should bundle the built binaries and pull them down once to the gear.

Comment 13 Warnar Boekkooi 2013-07-03 11:12:03 UTC
I'm not disagreeing there. So then my question is how can I best package this?

The problem will remain and in my opinion it's still a bit strange that a running script is terminated but with the new error message it's understandable.

Comment 14 Andy Grimm 2013-08-16 19:37:28 UTC
Closing this, since it's behaving as expected.