Bug 1024698

Summary: Met undefined method error when creating a scalable app with Min value set to 2 in manifest file
Product: OpenShift Online Reporter: Johnny Liu <jialiu>
Component: ContainersAssignee: Andy Goldstein <agoldste>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: agoldste, bmeng, xtian, yadu
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-24 03:27:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Johnny Liu 2013-10-30 09:37:38 UTC
Description of problem:
After set Min to 2 in cartridge's manifest.yml file, try to create a scalable app, failure is seen.

And the following error is seen in mcollective.log:
<--snip-->
I, [2013-10-30T05:20:11.850003 #26451]  INFO -- : openshift.rb:92:in `cartridge_do_action' cartridge_do_action call / action: cartridge_do, agent=openshift, data={:cartridge=>"openshift-origin-node",
 :action=>"update-cluster",
 :args=>
  {"--with-app-uuid"=>"5270cf25714bd27e8a000001",
   "--with-app-name"=>"scaphp53app",
   "--with-container-uuid"=>"5270cf25714bd27e8a000001",
   "--with-container-name"=>"scaphp53app",
   "--with-namespace"=>"jialiu",
   "--with-request-id"=>"f5ac40004c584f4b69b21a74c3e2dab9",
   "--sync-new-gears"=>true,
   "--proxy-gears"=>
    "5270cf25714bd27e8a000001,scaphp53app,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com",
   "--web-gears"=>
    "5270cf25714bd27e8a000001,scaphp53app,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com,38031 6403e2e6414411e3a3a922000aef05be,6403e2e6414411e3a3a922000aef05be,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com,38036",
   "--cart-name"=>"openshift-origin-node"},
 :process_results=>true}

I, [2013-10-30T05:20:11.850267 #26451]  INFO -- : openshift.rb:93:in `cartridge_do_action' cartridge_do_action validation = openshift-origin-node update-cluster {"--with-app-uuid"=>"5270cf25714bd27e8a000001", "--with-app-name"=>"scaphp53app", "--with-container-uuid"=>"5270cf25714bd27e8a000001", "--with-container-name"=>"scaphp53app", "--with-namespace"=>"jialiu", "--with-request-id"=>"f5ac40004c584f4b69b21a74c3e2dab9", "--sync-new-gears"=>true, "--proxy-gears"=>"5270cf25714bd27e8a000001,scaphp53app,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com", "--web-gears"=>"5270cf25714bd27e8a000001,scaphp53app,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com,38031 6403e2e6414411e3a3a922000aef05be,6403e2e6414411e3a3a922000aef05be,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com,38036", "--cart-name"=>"openshift-origin-node"}
I, [2013-10-30T05:20:11.850674 #26451]  INFO -- : openshift.rb:134:in `execute_action' Executing action [update-cluster] using method oo_update_cluster with args [{"--with-app-uuid"=>"5270cf25714bd27e8a000001", "--with-app-name"=>"scaphp53app", "--with-container-uuid"=>"5270cf25714bd27e8a000001", "--with-container-name"=>"scaphp53app", "--with-namespace"=>"jialiu", "--with-request-id"=>"f5ac40004c584f4b69b21a74c3e2dab9", "--sync-new-gears"=>true, "--proxy-gears"=>"5270cf25714bd27e8a000001,scaphp53app,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com", "--web-gears"=>"5270cf25714bd27e8a000001,scaphp53app,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com,38031 6403e2e6414411e3a3a922000aef05be,6403e2e6414411e3a3a922000aef05be,jialiu,ec2-54-224-237-89.compute-1.amazonaws.com,38036", "--cart-name"=>"openshift-origin-node"}]
E, [2013-10-30T05:20:12.061603 #26451] ERROR -- : openshift.rb:312:in `rescue in with_container_from_args' undefined method `keys' for nil:NilClass
E, [2013-10-30T05:20:12.061867 #26451] ERROR -- : openshift.rb:313:in `rescue in with_container_from_args' /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.16.8/lib/openshift-origin-node/model/application_container_ext/cartridge_actions.rb:1237:in `block in update_cluster'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.16.8/lib/openshift-origin-node/model/application_container_ext/cartridge_actions.rb:1236:in `select'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.16.8/lib/openshift-origin-node/model/application_container_ext/cartridge_actions.rb:1236:in `update_cluster'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:1000:in `block in oo_update_cluster'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:301:in `with_container_from_args'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:999:in `oo_update_cluster'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:139:in `execute_action'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:104:in `cartridge_do_action'
/opt/rh/ruby193/root/usr/share/ruby/mcollective/rpc/agent.rb:86:in `handlemsg'
/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:126:in `block (2 levels) in dispatch'
/opt/rh/ruby193/root/usr/share/ruby/timeout.rb:69:in `timeout'
/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:125:in `block in dispatch'
I, [2013-10-30T05:20:12.062301 #26451]  INFO -- : openshift.rb:150:in `execute_action' Finished executing action [update-cluster] (1)
I, [2013-10-30T05:20:12.096891 #26451]  INFO -- : openshift.rb:114:in `cartridge_do_action' cartridge_do_action failed (1)
------
undefined method `keys' for nil:NilClass
------)
<--snip-->
I, [2013-10-30T05:20:12.320608 #26451]  INFO -- : openshift.rb:92:in `cartridge_do_action' cartridge_do_action call / action: cartridge_do, agent=openshift, data={:cartridge=>"openshift-origin-node",
 :action=>"update-cluster",
 :args=>
  {"--with-app-uuid"=>"5270cf25714bd27e8a000001",
   "--with-app-name"=>"scaphp53app",
   "--with-container-uuid"=>"5270cf25714bd27e8a000001",
   "--with-container-name"=>"scaphp53app",
   "--with-namespace"=>"jialiu",
   "--with-request-id"=>"f5ac40004c584f4b69b21a74c3e2dab9",
   "--rollback"=>true,
   "--cart-name"=>"openshift-origin-node"},
 :process_results=>true}

I, [2013-10-30T05:20:12.320842 #26451]  INFO -- : openshift.rb:93:in `cartridge_do_action' cartridge_do_action validation = openshift-origin-node update-cluster {"--with-app-uuid"=>"5270cf25714bd27e8a000001", "--with-app-name"=>"scaphp53app", "--with-container-uuid"=>"5270cf25714bd27e8a000001", "--with-container-name"=>"scaphp53app", "--with-namespace"=>"jialiu", "--with-request-id"=>"f5ac40004c584f4b69b21a74c3e2dab9", "--rollback"=>true, "--cart-name"=>"openshift-origin-node"}
I, [2013-10-30T05:20:12.321348 #26451]  INFO -- : openshift.rb:134:in `execute_action' Executing action [update-cluster] using method oo_update_cluster with args [{"--with-app-uuid"=>"5270cf25714bd27e8a000001", "--with-app-name"=>"scaphp53app", "--with-container-uuid"=>"5270cf25714bd27e8a000001", "--with-container-name"=>"scaphp53app", "--with-namespace"=>"jialiu", "--with-request-id"=>"f5ac40004c584f4b69b21a74c3e2dab9", "--rollback"=>true, "--cart-name"=>"openshift-origin-node"}]
E, [2013-10-30T05:20:12.478930 #26451] ERROR -- : openshift.rb:312:in `rescue in with_container_from_args' undefined method `each_value' for nil:NilClass
E, [2013-10-30T05:20:12.479213 #26451] ERROR -- : openshift.rb:313:in `rescue in with_container_from_args' /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.16.8/lib/openshift-origin-node/model/application_container_ext/cartridge_actions.rb:1155:in `generate_update_cluster_control_args'
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.16.8/lib/openshift-origin-node/model/application_container_ext/cartridge_actions.rb:1289:in `update_cluster'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:1000:in `block in oo_update_cluster'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:301:in `with_container_from_args'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:999:in `oo_update_cluster'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:139:in `execute_action'
/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:104:in `cartridge_do_action'
/opt/rh/ruby193/root/usr/share/ruby/mcollective/rpc/agent.rb:86:in `handlemsg'
/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:126:in `block (2 levels) in dispatch'
/opt/rh/ruby193/root/usr/share/ruby/timeout.rb:69:in `timeout'
/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:125:in `block in dispatch'
I, [2013-10-30T05:20:12.479461 #26451]  INFO -- : openshift.rb:150:in `execute_action' Finished executing action [update-cluster] (1)
I, [2013-10-30T05:20:12.512965 #26451]  INFO -- : openshift.rb:114:in `cartridge_do_action' cartridge_do_action failed (1)
------
undefined method `each_value' for nil:NilClass
------)
<--snip-->

Version-Release number of selected component (if applicable):
devenv_3966

How reproducible:
Always

Steps to Reproduce:
1.Update /usr/libexec/openshift/cartridges/php/metadata/manifest.yml, and set Min to 2.
2. Do the following operation to make your change take effect.
# /etc/init.d/ruby193-mcollective restart
# oo-admin-broker-cache -c --console
# /etc/init.d/rhc-broker restart
3. Create a php scalable app

Actual results:
Fail to create it.
# rhc app create scaphp53app php-5.3 -p redhat -l jialiu -s
Application Options
-------------------
  Namespace:  jialiu
  Cartridges: php-5.3
  Gear Size:  default
  Scaling:    yes
Creating application 'scaphp53app' ... 
Unable to complete the requested operation due to: An invalid exit code (1) was returned from the server ip-10-239-5-190.  This indicates an unexpected problem during the execution of
your request..
Reference ID: f5ac40004c584f4b69b21a74c3e2dab9


Expected results:
Create app successfully with 2 active gears.

Additional info:

Comment 1 Andy Goldstein 2013-10-30 14:03:18 UTC
https://github.com/openshift/origin-server/pull/4047

Comment 2 openshift-github-bot 2013-10-30 19:42:11 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/241bff8db0f318cf256b731a5df474733d2e3595
Bug 1024698

Fix update_cluster to handle the case where a cartridge's minimum
scaling count is >= 2 - treat all web gears except self as new in this
situation.

Don't rotate out when stopping a gear if it's called during the initial
build from post-configure.

Don't check for the force_clean_build marker in prereceive if it's
called during the initial build from post-configure; default to false
instead.

Don't distribute or attempt to do remote activations during an initial
build (this will be taken care of when update-cluster is invoked
afterward).

Comment 3 Johnny Liu 2013-10-31 05:39:29 UTC
Verified this bug with devenv_devenv_3973, and PASS.

Set Min to 2, Max to 10 in maifest file, then create scalable app, app could be created successfully.