Bug 1229300 - oo-admin-move across node profiles should update quota limits appropriately
Summary: oo-admin-move across node profiles should update quota limits appropriately
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 2.2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Timothy Williams
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-08 12:18 UTC by Miheer Salunke
Modified: 2019-08-15 04:40 UTC (History)
8 users (show)

Fixed In Version: rubygem-openshift-origin-msg-broker-mcollective-1.35.3.1-1.el6op
Doc Type: Bug Fix
Doc Text:
Previously, when moving a non-scaled application across node profiles, the proper quota for the new profile was not applied to the gear. The gear still used the quota from its previous gear size. Additionally, any additional gear storage was not added to the quota of the new gear. This bug fix ensures the new node profile's quota limits are used, taking into account additional storage the gear may have. As a result, gears moved across node profiles have the proper quota applied, and additional gear storage still exists after the move.
Clone Of:
Environment:
Last Closed: 2015-09-30 16:37:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:1844 0 normal SHIPPED_LIVE Important: Red Hat OpenShift Enterprise 2.2.7 security, bug fix and enhancement update 2015-09-30 20:35:28 UTC

Description Miheer Salunke 2015-06-08 12:18:16 UTC
Description of problem:

An application has been created with a medium gear size. Then, we had to move it on a specific node for test purpose. The destination node was a small one. The oo-admin-move went without problem and no error has been raised. The application gear size and quotas have not been changed by this operation. So it is now on a small node with a medium gear size. 

Version-Release number of selected component (if applicable):
2.2

How reproducible:
Always

Steps to Reproduce:
1.Create an application with medium size gear.
2.Move it on a node having small gear profile.

Actual results:
oo-admin-move command shall raise an error
The application gear size, quotas and limits don't change

Expected results:

oo-admin-move command shall raise an error


Additional info:

Comment 2 Timothy Williams 2015-06-08 13:32:54 UTC
Could you supply the oo-admin-move command you used to move the gear?

I believe the quotas are actually changed when the gear is moved to a small node. Run the below in your gear to see if the current limits match the "small" node profile's resource limits or the "large" node profile's:

  # oo-cgroup-read memory.limit_in_bytes
  # oo-cgroup-read cpu.cfs_quota_us

You are correct in that the system still reports the gear as the initial, larger size (which is certainly an issue).

Comment 4 Miheer Salunke 2015-06-16 14:02:25 UTC
Use this command to reproduce the bug
  oo-admin-move --gear_uuid 54ff0733c3215e8027002de2 -p small --change_district

Please find the output of the commands from gears :
]\> oo-cgroup-read memory.limit_in_bytes
536870912
]\> oo-cgroup-read cpu.cfs_quota_us
30000

So they have the memory.limit_in_bytes and the cpu.cfs_quota_us from their dev-small profile. But when they do a quota -s :

]\> quota -s

Disk quotas for user 54ff0733c3215e8027002de2 (uid 6917):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
/dev/mapper/rootvg-openshift_lv
                  22796       0   3072M             901       0    160k

They have the quota inodes and blocks of the medium (previous) profile.

As a workaround they need to st quotas to off, to turn them on again and to reapply indoes and blocks quotas.

Comment 5 Timothy Williams 2015-06-16 14:29:36 UTC
Thanks for the info. Looks like the cgroup configuration is changed as it should be, but the quotas are indeed still set to the quotas defined by the original node profile.

The fix for this shouldn't be too difficult.

Comment 6 Timothy Williams 2015-06-17 17:52:38 UTC
https://github.com/openshift/origin-server/pull/6172

Comment 7 openshift-github-bot 2015-08-14 19:39:41 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/4908a102bbb3c7a7f742f16841a250782701039e
Ensure proper quota is used when moving gears across node profiles

Bug 1229300
https://bugzilla.redhat.com/show_bug.cgi?id=1229300
When moving gears across node profiles, ensure that the quota for the new node profile is imposed on the gear during the move.

Comment 9 Anping Li 2015-08-20 08:30:58 UTC
QE need a new puddle/package to verify this bug.

Comment 13 Yanping Zhang 2015-09-18 07:36:08 UTC
Check in the puddle-2-2-2015-09-17:
# rpm -qa|grep broker-mcollective
rubygem-openshift-origin-msg-broker-mcollective-1.34.1.1-1.el6op.noarch

It doesn't contain the fix pkg: rubygem-openshift-origin-msg-broker-mcollective-1.35.3.1-1.el6op

The bug still can be reproduced.

Comment 15 Yanping Zhang 2015-09-21 08:10:31 UTC
Found a small issue on puddle 2015-09-18.2, after moved a medium gear to small profile node, the "rhc app show <appname>" still showed "Gears: 1 medium".(Same when moved small gear to medium profile node).
Detailed info as below:
[root@dhcp-129-219 ~]# rhc app show pltest
pltest @ http://pltest-domzyp.ose22-auto.com.cn/ (uuid: 55ff783782611d559a000027)
---------------------------------------------------------------------------------
  Domain:     domzyp
  Created:    11:23 AM
  Gears:      1 (defaults to medium)
  Git URL:    ssh://domzyp-pltest-1.com.cn/~/git/pltest.git/
  SSH:        domzyp-pltest-1.com.cn
  Deployment: auto (on git push)

  perl-5.10 (Perl 5.10)
  ---------------------
    Gears: 1 medium
[root@dhcp-129-219 ~]# rhc app show pltest -g
ID              State   Cartridges Size  SSH URL
--------------- ------- ---------- ----- -----------------------------------------------
domzyp-pltest-1 started perl-5.10  small domzyp-pltest-1.com.cn

Comment 17 Yanping Zhang 2015-09-23 07:38:58 UTC
Opened a bug to track issue in Comment15: https://bugzilla.redhat.com/show_bug.cgi?id=1265528

Since the original bug has been fixed in current puddle ose 2.2/2015-09-22.1, so move the bug to Verified.

Comment 19 errata-xmlrpc 2015-09-30 16:37:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1844.html


Note You need to log in before you can comment on or make changes to this bug.