Hide Forgot
Description of problem: Keep trying to create apps. When the application is assigned to node2 (ex-std-node2.int.rhcloud.com), it will fail with the following error: rhc app create zd1 zend-6.1 mysql-5.5 Application 'zd1' not found. Application Options ------------------- Domain: qiuzhang Cartridges: zend-6.1, mysql-5.5 Gear Size: default Scaling: no Creating application 'zd1' ... Unable to complete the requested operation due to: An invalid exit code (157) was returned from the server ex-std-node2.int.rhcloud.com. This indicates an unexpected problem during the execution of your request.. Reference ID: 7f19093674011383808831ab65db71af Version-Release number of selected component (if applicable): INT(devenv_4158) How reproducible: always Steps to Reproduce: 1. Keep trying to create apps 2. 3. Actual results: When the app is assigned to node2, it will fail to be created. rhc app create zd1 zend-6.1 mysql-5.5 Application 'zd1' not found. Application Options ------------------- Domain: qiuzhang Cartridges: zend-6.1, mysql-5.5 Gear Size: default Scaling: no Creating application 'zd1' ... Unable to complete the requested operation due to: An invalid exit code (157) was returned from the server ex-std-node2.int.rhcloud.com. This indicates an unexpected problem during the execution of your request.. Reference ID: 7f19093674011383808831ab65db71af Expected results: App is created successfully. Additional info:
Also met this on ex-std-node1.int.rhcloud.com. # rhc app create app2 jbossas-7 mysql-5.5 mongodb-2.2 postgresql-8.4 --no-git Application Options ------------------- Domain: lxiamigrate Cartridges: jbossas-7, mysql-5.5, mongodb-2.2, postgresql-8.4 Gear Size: default Scaling: no Creating application 'app2' ... Starting MySQL cartridge Details from MySQL error log: 131220 02:59:07 mysqld_safe Starting mysqld daemon with databases from /var/lib/openshift/52b3f86203ef64fd0700000f/mysql//data/ 131220 2:59:07 [Note] Plugin 'FEDERATED' is disabled. 131220 2:59:07 InnoDB: The InnoDB memory heap is disabled 131220 2:59:07 InnoDB: Mutexes and rw_locks use GCC atomic builtins 131220 2:59:07 InnoDB: Compressed tables use zlib 1.2.3 131220 2:59:07 InnoDB: Using Linux native AIO 131220 2:59:07 InnoDB: Initializing buffer pool, size = 32.0M 131220 2:59:07 InnoDB: Completed initialization of buffer pool InnoDB: The first specified data file ./ibdata1 did not exist: InnoDB: a new database to be created! 131220 2:59:07 InnoDB: Setting file ./ibdata1 size to 10 MB InnoDB: Database physically writes the file full: wait... 131220 2:59:10 InnoDB: Log file ./ib_logfile0 did not exist: new to be created InnoDB: Setting log file ./ib_logfile0 size to 16 MB InnoDB: Database physically writes the file full: wait... 131220 2:59:13 InnoDB: Log file ./ib_logfile1 did not exist: new to be created InnoDB: Setting log file ./ib_logfile1 size to 16 MB InnoDB: Database physically writes the file full: wait... InnoDB: Doublewrite buffer not found: creating new InnoDB: Doublewrite buffer created InnoDB: 127 rollback segment(s) active. InnoDB: Creating foreign key constraint system tables InnoDB: Foreign key constraint system tables created 131220 2:59:35 InnoDB: Waiting for the background threads to start Unable to complete the requested operation due to: An invalid exit code (157) was returned from the server ex-std-node1.int.rhcloud.com. This indicates an unexpected problem during the execution of your request.. Reference ID: 4d0a81e207ed2bd2a120ad3fe590ebe4
I'm not able to reproduce this on devenv_4178. The '157' error is if I do remember correctly something about the database files created with different version of mysql that is trying to use them....
+ I just tried this on current INT: ~ → rhc app create app2 jbossas-7 mysql-5.5 mongodb-2.2 postgresql-8.4 --no-git Application Options ------------------- Domain: mfojtik Cartridges: jbossas-7, mysql-5.5, mongodb-2.2, postgresql-8.4 Gear Size: default Scaling: no ..... Waiting for your DNS name to be available ... done Your application 'app2' is now available. Can you please confirm, that this is not longer a bug?
It's fixed, Check on latest INT, please refer to the following results: 1. on ex-std-node1 $ rhc app create app5 jbossas-7 mysql-5.5 mongodb-2.2 postgresql-8.4 --no-git Application Options ------------------- Domain: cdm Cartridges: jbossas-7, mysql-5.5, mongodb-2.2, postgresql-8.4 Gear Size: default Scaling: no Creating application 'app5' ... Artifact: ./ROOT.war is still deploying done PostgreSQL 8.4 database added. Please make note of these credentials: Root User: adminbyt5t9z Root Password: gZ3l5D-F4xlV Database Name: app5 Connection URL: postgresql://$OPENSHIFT_POSTGRESQL_DB_HOST:$OPENSHIFT_POSTGRESQL_DB_PORT MongoDB 2.2 database added. Please make note of these credentials: Root User: admin Root Password: EYlA9ZvMXj7r Database Name: app5 Connection URL: mongodb://$OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/ MySQL 5.5 database added. Please make note of these credentials: Root User: admintl3QkFh Root Password: 61vyyqtYGFZ4 Database Name: app5 Connection URL: mysql://$OPENSHIFT_MYSQL_DB_HOST:$OPENSHIFT_MYSQL_DB_PORT/ You can manage your new MySQL database by also embedding phpmyadmin. The phpmyadmin username and password will be the same as the MySQL credentials above. Waiting for your DNS name to be available ... done Your application 'app5' is now available. URL: http://app5-cdm.int.rhcloud.com/ SSH to: 52ca1b0003ef648d340000d7.rhcloud.com Git remote: ssh://52ca1b0003ef648d340000d7.rhcloud.com/~/git/app5.git/ Run 'rhc show-app app5' for more details about your app. $ rhc ssh app5 'hostname' Warning: Permanently added 'app5-cdm.int.rhcloud.com' (RSA) to the list of known hosts. ex-std-node1.int.rhcloud.com 2. on ex-std-node2 $ rhc app create app2 jbossas-7 mysql-5.5 mongodb-2.2 postgresql-8.4 --no-git Application Options ------------------- Domain: cdm Cartridges: jbossas-7, mysql-5.5, mongodb-2.2, postgresql-8.4 Gear Size: default Scaling: no Creating application 'app2' ... done PostgreSQL 8.4 database added. Please make note of these credentials: Root User: adminexpxr8z Root Password: rJ-l2RuKZgcx Database Name: app2 Connection URL: postgresql://$OPENSHIFT_POSTGRESQL_DB_HOST:$OPENSHIFT_POSTGRESQL_DB_PORT MongoDB 2.2 database added. Please make note of these credentials: Root User: admin Root Password: gICHA22C6xgl Database Name: app2 Connection URL: mongodb://$OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/ MySQL 5.5 database added. Please make note of these credentials: Root User: adminSBdysiU Root Password: zYzg9DHiLueD Database Name: app2 Connection URL: mysql://$OPENSHIFT_MYSQL_DB_HOST:$OPENSHIFT_MYSQL_DB_PORT/ You can manage your new MySQL database by also embedding phpmyadmin. The phpmyadmin username and password will be the same as the MySQL credentials above. Waiting for your DNS name to be available ... done Your application 'app2' is now available. URL: http://app2-cdm.int.rhcloud.com/ SSH to: 52ca0e8c03ef646442000179.rhcloud.com Git remote: ssh://52ca0e8c03ef646442000179.rhcloud.com/~/git/app2.git/ Run 'rhc show-app app2' for more details about your app. $ rhc ssh app2 'hostname' Warning: Permanently added 'app2-cdm.int.rhcloud.com' (RSA) to the list of known hosts. ex-std-node2.int.rhcloud.com
Tried on current INT to create 10 apps. # for i in `seq 1 10` ; do rhc app create app$i jbossas-7 mysql-5.5 mongodb-2.2 postgresql-8.4 --no-git ; done 1 of them still met the error: Creating application 'app5' ... Starting MySQL cartridge Details from MySQL error log: 140105 21:46:08 mysqld_safe Starting mysqld daemon with databases from /var/lib/openshift/52ca188203ef648d34000072/mysql//data/ 140105 21:46:09 [Note] Plugin 'FEDERATED' is disabled. 140105 21:46:09 InnoDB: The InnoDB memory heap is disabled 140105 21:46:09 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140105 21:46:09 InnoDB: Compressed tables use zlib 1.2.3 140105 21:46:09 InnoDB: Using Linux native AIO 140105 21:46:09 InnoDB: Initializing buffer pool, size = 32.0M 140105 21:46:09 InnoDB: Completed initialization of buffer pool InnoDB: The first specified data file ./ibdata1 did not exist: InnoDB: a new database to be created! 140105 21:46:09 InnoDB: Setting file ./ibdata1 size to 10 MB InnoDB: Database physically writes the file full: wait... 140105 21:46:10 InnoDB: Log file ./ib_logfile0 did not exist: new to be created InnoDB: Setting log file ./ib_logfile0 size to 16 MB InnoDB: Database physically writes the file full: wait... 140105 21:46:10 InnoDB: Log file ./ib_logfile1 did not exist: new to be created InnoDB: Setting log file ./ib_logfile1 size to 16 MB InnoDB: Database physically writes the file full: wait... InnoDB: Doublewrite buffer not found: creating new InnoDB: Doublewrite buffer created InnoDB: 127 rollback segment(s) active. InnoDB: Creating foreign key constraint system tables InnoDB: Foreign key constraint system tables created 140105 21:46:36 InnoDB: Waiting for the background threads to start 140105 21:46:37 InnoDB: 5.5.32 started; log sequence number 0 140105 21:46:37 [Note] Server hostname (bind-address): '127.7.43.132'; port: 3306 140105 21:46:37 [Note] - '127.7.43.132' resolves to '127.7.43.132'; 140105 21:46:37 [Note] Server socket created on IP: '127.7.43.132'. 140105 21:46:37 [Note] Event Scheduler: Loaded 0 events 140105 21:46:37 [Note] /opt/rh/mysql55/root/usr/libexec/mysqld: ready for connections. Version: '5.5.32' socket: '/var/lib/openshift/52ca188203ef648d34000072/mysql//socket/mysql.sock' port: 3306 MySQL Community Server (GPL) Unable to complete the requested operation due to: An invalid exit code (157) was returned from the server ex-std-node2.int.rhcloud.com. This indicates an unexpected problem during the execution of your request.. Reference ID: 14047b370aaf50ee71a818bb63984ddc
Move to verified based on comment #4 and following checkings. 1. Create 20 applications. 2. Verify apps are created successfully on node1/node2/node3. # for i in `seq 1 20` ; do rhc ssh app$i hostname ; done | sort | uniq ex-std-node1.int.rhcloud.com ex-std-node2.int.rhcloud.com ex-std-node3.int.rhcloud.com Notes: Issue in comment #5 will be tracked in a separate bug.