Bug 1064631
| Summary: | Cannot change the starting UID for gears to a high number | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Luke Meyer <lmeyer> | ||||
| Component: | Containers | Assignee: | Brenton Leanhardt <bleanhar> | ||||
| Status: | CLOSED ERRATA | QA Contact: | libra bugs <libra-bugs> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 2.0.0 | CC: | anli, jialiu, libra-onpremise-devel, nicholas_schuetz, pep, yanpzhan | ||||
| Target Milestone: | --- | Keywords: | UpcomingRelease | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | 1064570 | Environment: | |||||
| Last Closed: | 2014-05-15 14:40:42 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | 1064570 | ||||||
| Bug Blocks: | 1075735 | ||||||
| Attachments: |
|
||||||
|
Description
Luke Meyer
2014-02-13 01:54:56 UTC
Just to further confuse things, I noticed that there's a parameter /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf:DISTRICTS_FIRST_UID that apparently needs to be set to match the beginning of the node range, prior to creating a district. However, with this done gear creation still failed. I note it here to enable testing once this is fixed. Current problems needing to be fixed: * IP addresses do not properly wrap around and the upper limit is not documented * TC does not wrap around. This should be very easy. The fix for this BZ will require tests to ensure backwards compatibility. To test this make sure the following configs match: node.conf GEAR_MIN_UID GEAR_MAX_UID openshift-origin-msg-broker-mcollective.conf DISTRICTS_MAX_CAPACITY DISTRICTS_FIRST_UID eg, GEAR_MIN_UID=501000 GEAR_MAX_UID=506999 DISTRICTS_MAX_CAPACITY=6000 DISTRICTS_FIRST_UID=501000 Be careful to ensure the settings are correct at the time of district creation. In theory UIDs up to 2^32 should now be allowed. I have only tested in the 500k range so there is a good chance there is a point in which the selinux categories do not properly wrap around. This change allows ip address ranges and TC queuing disciplines to wrap forever. *** Bug 1075735 has been marked as a duplicate of this bug. *** *** Bug 1051251 has been marked as a duplicate of this bug. *** The practical UID limit for our MCS labeling in OpenShift is currently 523,776. I think we have options to increase this if it's ever needed but given that a free range of 6000 UIDs is all that is needed this should be high enough for now. You can see the problem here: https://github.com/openshift/origin-server/blob/fa31633e6eab70783950bfc5139f74cc5c1beb4d/node/lib/openshift-origin-node/utils/selinux_context.rb#L220 The default group_size is 2 which yields 523,776 possibilities. The code is optimized for the default. Modifying the group size is possible however that triggers the logic at line #237 which a simplistic brute force way to find the correct mls label. Setting a group size of 3 will likely OOM. For now I would recommend customers keep the UID ranges below 500,000. Verified and pass on puddle-2-1-2014-05-02
Set values as below:
In node.conf
GEAR_MIN_UID=523774
GEAR_MAX_UID=523777
In openshift-origin-msg-broker-mcollective.conf
DISTRICTS_MAX_CAPACITY=4
DISTRICTS_FIRST_UID=523774
1. Don't enable district.
Create app, scaled up app or add cartridge. the result is as expected.
1)All actions can create userid=523775.
2)All actions will fail with "Unable to complete the requested operation due to: An invalid exit code (1) was returned from the server" when try to create user >523775.
3) The gear uid=523775 can be moved to district nodes
2. Enable district
1) Create district and set max_uid=>523775.(Valid userid are 523774, and 523775 now)
Create app, scaled up app or add cartridge. catridge can be created and userid can be 523775.
2) Create one scaled app with two gears, and then add capacity one. (current avaliable UID is 523776)
Scale up app,add cartridge to app, create app. all action failed with 'Unable to complete the requested operation due to: An invalid exit code (1) was returned from the server'. That is as expected.
3) The gear uid=523775 can be moved in district.
>Additional info:
>By the way, once this is working, if I already have nodes with gears in the >usual range, I expect to be able to move those gears to new nodes with the >higher range so that I can get rid of the old nodes.
currently, Openshift don't allow to change the uid during moving.
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/b11ca1afcc9663c16e4725c2eb178c430dda7641 Bug 1064631 - Wrap UID-based ip addresses and netclasses calculations |