Description of problem: When there are a lot of nodes in a cluster, use "oadm manage-node" command to update all nodes' attribute, several nodes can not be updated, and error message is saying "the object has been modified; please apply your changes to the latest version and try again", so seem like race condition is happening to prevent node's attribute to be updated. Version-Release number of selected component (if applicable): atomic-openshift-3.1.0.0-0.git.0.0e71938.el7aos.x86_64 How reproducible: Always Steps to Reproduce: 1.Set up a cluster including 60 nodes. 2.Run "oadm manage-node" command to disable scheduling for all the nodes. # oadm manage-node --selector="region=primary" --schedulable=false NAME LABELS STATUS AGE test1.cluster.local kubernetes.io/hostname=10.66.81.15,region=primary,zone=default Ready,SchedulingDisabled 2d test10.cluster.local kubernetes.io/hostname=10.66.80.12,region=primary,zone=default Ready,SchedulingDisabled 2d test11.cluster.local <--snip--> Error from server: node "test15.cluster.local" cannot be updated: the object has been modified; please apply your changes to the latest version and try again Error from server: node "test21.cluster.local" cannot be updated: the object has been modified; please apply your changes to the latest version and try again Error from server: node "test22.cluster.local" cannot be updated: the object has been modified; please apply your changes to the latest version and try again <--snip--> Actual results: several nodes failed to be updated. Expected results: All the nodes should be updated successfully. Additional info:
I could not reproduce locally, sending to QE to check if this is already fixed.
Fixed in https://github.com/openshift/origin/pull/12486.
This has been merged into ocp and is in OCP v3.5.0.7 or newer.
Verified this bug with atomic-openshift-3.5.0.7-1.git.0.390ef18.el7, and PASS.
This fix was backported to 3.3 and 3.4, so sending to QA again to be tested in those versions.
Opened separate bugs to track 3.3 and 3.4: https://bugzilla.redhat.com/show_bug.cgi?id=1416506 https://bugzilla.redhat.com/show_bug.cgi?id=1416509
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0884