Bug 1416506 - [3.3] Race condition is seen when updating a batch of nodes in cluster using "oadm manage-node"
Summary: [3.3] Race condition is seen when updating a batch of nodes in cluster using ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 3.3.1
Assignee: Fabiano Franz
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On: 1279303
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-25 16:24 UTC by Fabiano Franz
Modified: 2017-02-22 18:11 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: race condition when updating a batch of nodes (we verified the issue on 8+ nodes) to (un)schedulable with "oadm manage-node --schedulable=<true|false>". Consequence: several nodes couldn't be updated with the "object has been modified" error. Fix: use a patch on the "unschedulable" field of the node object instead of a full update. Result: all nodes could be properly updated schedulable and/or unschedulable (tested on a 60 nodes cluster).
Clone Of:
Environment:
Last Closed: 2017-02-22 18:11:26 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0289 0 normal SHIPPED_LIVE OpenShift Container Platform 3.4.1.7, 3.3.1.14, and 3.2.1.26 bug fix update 2017-02-22 23:10:04 UTC

Description Fabiano Franz 2017-01-25 16:24:32 UTC 
Bug opened to track the backport of the bug below to 3.3.x:

https://bugzilla.redhat.com/show_bug.cgi?id=1279303
Comment 3 Johnny Liu 2017-02-03 06:13:52 UTC 
Verified this bug with atomic-openshift-3.3.1.12-1.git.0.3d14c02.el7.x86_64, and PASS.

# oc get nodes --selector="registry=enabled"|wc -l
59

# oadm manage-node --selector="registry=enabled" --schedulable=false
NAME                               STATUS                     AGE
qe-jialiu-node-registry-router-1   Ready,SchedulingDisabled   1h
qe-jialiu-node-registry-router-10   Ready,SchedulingDisabled   1h
qe-jialiu-node-registry-router-11   Ready,SchedulingDisabled   1h
<--snip-->
qe-jialiu-node-registry-router-7   Ready,SchedulingDisabled   1h
qe-jialiu-node-registry-router-8   Ready,SchedulingDisabled   1h
qe-jialiu-node-registry-router-9   Ready,SchedulingDisabled   1h


# oadm manage-node --selector="registry=enabled" --schedulable=true
NAME                               STATUS    AGE
qe-jialiu-node-registry-router-1   Ready     1h
qe-jialiu-node-registry-router-10   Ready     1h
qe-jialiu-node-registry-router-11   Ready     1h
<--snip-->
qe-jialiu-node-registry-router-7   Ready     1h
qe-jialiu-node-registry-router-8   Ready     1h
qe-jialiu-node-registry-router-9   Ready     1h
Comment 5 errata-xmlrpc 2017-02-22 18:11:26 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0289
Note You need to log in before you can comment on or make changes to this bug.