Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1559271 - AWS nodes does not keep persistent labels after shut down
AWS nodes does not keep persistent labels after shut down
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Pod (Show other bugs)
3.6.1
Unspecified Unspecified
high Severity high
: ---
: 3.11.0
Assigned To: Seth Jennings
weiwei jiang
: OpsBlocker, Reopened
: 1569043 1608912 1608971 1619599 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2018-03-22 03:15 EDT by Suhaas Bhat
Modified: 2018-10-11 03:20 EDT (History)
18 users (show)

See Also:
Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of:
Environment:
Last Closed: 2018-10-11 03:19:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3570631 None None None 2018-09-05 23:06 EDT
Red Hat Product Errata RHBA-2018:2652 None None None 2018-10-11 03:20 EDT

  None (edit)
Description Suhaas Bhat 2018-03-22 03:15:01 EDT
Description of problem:
OCP nodes on AWS do not keep persistent labels when given from "oc label" command after shutting down the node.

Version-Release number of selected component (if applicable):
oc v3.6.173.0.21
kubernetes v1.6.1+5115d708d7


Actual results: Labels are not persistent after node shutdown using "oc command"


Expected results: Should be persistent

Additional Info :
Correct entries in /etc/origin/cloudprovider/aws.conf
Same Kubernetes Cluster Tags across all cluster
Comment 2 Seth Jennings 2018-03-22 10:09:32 EDT
Recently had a similiar issue with Openstack cloudprovider in that the node controller was getting a filtered list of instances, only getting the active ones.
https://bugzilla.redhat.com/show_bug.cgi?id=1558422
Comment 3 Seth Jennings 2018-04-10 23:29:47 EDT
More relevant upstream discussion/work:
https://github.com/kubernetes/kubernetes/issues/58635
https://github.com/kubernetes/kubernetes/pull/60009
https://github.com/kubernetes/kubernetes/issues/46442
https://github.com/kubernetes/kubernetes/issues/45118

There is also an old open PR to remove the instance filtering from the AWS cloudprovider, which would address is bug:
https://github.com/kubernetes/kubernetes/pull/45986
Comment 4 Seth Jennings 2018-04-18 10:08:06 EDT
The effort has stalled upstream.  I am pushing for acceptance of a targeted fix for just this case (one line change in the AWS cloudprovider) as a stopgap while a more general solution is developed.
Comment 5 Seth Jennings 2018-05-02 12:39:04 EDT
*** Bug 1569043 has been marked as a duplicate of this bug. ***
Comment 10 Seth Jennings 2018-07-30 16:16:00 EDT
*** Bug 1608971 has been marked as a duplicate of this bug. ***
Comment 11 Seth Jennings 2018-07-30 16:54:51 EDT
*** Bug 1608912 has been marked as a duplicate of this bug. ***
Comment 13 Seth Jennings 2018-07-31 14:55:14 EDT
I posted a more concise PR upstream to avoid friction:
https://github.com/kubernetes/kubernetes/pull/66835
Comment 15 Seth Jennings 2018-08-15 16:52:44 EDT
Origin PR:
https://github.com/openshift/origin/pull/20663
Comment 16 Seth Jennings 2018-08-22 10:05:39 EDT
*** Bug 1619599 has been marked as a duplicate of this bug. ***
Comment 21 weiwei jiang 2018-09-06 03:20:26 EDT
Checked on 
# oc version 
oc v3.11.0-0.28.0
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://ip-172-18-7-85.ec2.internal:8443
openshift v3.11.0-0.28.0
kubernetes v1.11.0+d4cacc0

And this issue has been fixed.
1. labels from `oc label` is persistent now.
2. labels from node-config.yaml(from 3.10, should be configmap in openshift-node namespace) is also applied after reboot system
3. labels already applied to node from node-config.yaml will still keep after reboot the system
Comment 23 errata-xmlrpc 2018-10-11 03:19:09 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2652

Note You need to log in before you can comment on or make changes to this bug.