Description of problem: The OpenShift Console / RHC can see duplicate cartridges when running oo-admin-ctl-cartridge -c import-profile --activate. Version-Release number of selected component (if applicable): 2.1 and 2.2 How reproducible: 50% (guess) Steps to Reproduce: * 1. oo-admin-ctl-cartridge -c import-profile --activate - Run this multiple times. Verification can be done with: 2. oo-admin-console-cache --clear oo-admin-broker-cache --clear 3. oo-admin-ctl-cartridge -c list - This shows duplicate cartridges some with the same TIME (Priority) rhc cartridges - This shows duplicate cartridges Actual results: What ends up happening after running this command a few times is something like the following: This is seen when importing the cartridges. Updating 2 cartridges ... Failed to update ruby-1.9 Another cartridge version 54da53802f6c6ec8be000001 was activated at 2015-02-10T18:52:48+00:00. However when you run (the following) you see a list of cartridges with different ID's # oo-admin-ctl-cartridge --raw -c list | grep -e Display-Name -e Id -e Cartridge-Version "Display-Name": "Ruby 1.9", "Id": {"$oid": "54da53802f6c6ec8be000001"}, "Cartridge-Version": "0.0.18.1", "Display-Name": "Ruby 1.9", "Id": {"$oid": "54da53802f6c6ec8be000002"}, Expected results: You should not import cartridges of the same NAME / Version. Additional info: https://github.com/openshift/enterprise-server/blob/enterprise-2.2/controller/app/models/cartridge_type.rb#L111-L127 The issue seems to be with the TIME (Priority) and how this is determined. Currently the following is the only effective way to correct the duplicates: # for cartridge in $(oo-admin-ctl-cartridge -c list | awk '{print $2}'); do oo-admin-ctl-cartridge -c delete -n $cartridge; done # oo-admin-ctl-cartridge -c import-profile --activate # oo-admin-console-cache --clear # oo-admin-broker-cache --clear
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/b1e1d1d1d909328fad7f8803808bad93bc61d4d5 Resolve race condition where cartridges activated within the same second conflicted Bug 1191283 Bugzilla link https://bugzilla.redhat.com/show_bug.cgi?id=1191283 Using the ":priority.lt => self.priority" would not find types with a priority within the same second of this types priority. Because of this, older types did not have their priority properly set to 'nil'. This caused us to believe that a cartridge type with the same name was activated at the same time (or after) this type.
Check on rubygem-openshift-origin-controller-1.36.2.3-1.el6op.noarch QE can not reproduce this issue. move it to VERIFIED.
Check on puddle [2015-09-17] Update or install a cartridge on all nodes unless the cartridge list has not yet been imported on the broker. [root@broker conf.d]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. Updating 1 cartridges ... 55fba00b82611d04fa000001 # A perl-5.10 (active) Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node3.ose22-auto.com.cn'. No changes [root@broker conf.d]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node3.ose22-auto.com.cn'. No changes Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. No changes [root@broker conf.d]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node3.ose22-auto.com.cn'. No changes Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. No changes [root@broker conf.d]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node3.ose22-auto.com.cn'. No changes Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. No changes There is no duplicate cartridge.
Check on puddle [2.2.7/2015-09-22.1] [root@broker ~]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. Updating 1 cartridges ... 5602054282611d93b7000001 # A perl-5.10 (active) Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node1.ose22-auto.com.cn'. No changes [root@broker ~]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. No changes Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node1.ose22-auto.com.cn'. No changes [root@broker ~]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node2.ose22-auto.com.cn'. No changes Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. No changes [root@broker ~]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. No changes Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node1.ose22-auto.com.cn'. No changes [root@broker ~]# oo-admin-ctl-cartridge -c import-profile --activate Importing cartridges for profile 'medium' and platform 'linux'. Importing cartridges from node 'broker.ose22-auto.com.cn'. No changes Importing cartridges for profile 'small' and platform 'linux'. Importing cartridges from node 'node2.ose22-auto.com.cn'. No changes There is no duplicate cartridge.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-1844.html