Bug 1894977

Summary:	Feature Gate enablement - add CLI section and add an example to enable multiple feature gates
Product:	OpenShift Container Platform	Reporter:	Andreas Karis <akaris>
Component:	Documentation	Assignee:	Servesha <sdudhgao>
Status:	CLOSED NOTABUG	QA Contact:	Sunil Choudhary <schoudha>
Severity:	medium	Docs Contact:	Latha S <lmurthy>
Priority:	medium
Version:	4.6	CC:	aos-bugs, jokerman, lmurthy, oarribas, rphillips
Target Milestone:	---
Target Release:	4.6.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-12-09 10:28:04 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Andreas Karis 2020-11-05 14:06:50 UTC

Document URL: 
https://docs.openshift.com/container-platform/4.6/nodes/clusters/nodes-cluster-enabling-features.html

Section Number and Name: 

Describe the issue: 
Feature Gate enablement - add CLI section and add an example to enable multiple feature gates

The current exampe lists only:
apiVersion: config.openshift.io/v1
kind: FeatureGate
metadata:
  name: cluster
spec:
  featureSet: TechPreviewNoUpgrade 

But how would I enable both:

RotateKubeletServerCertificate and TechPreviewNoUpgrade 

Also, how can admins list all available feature gate featureSets?

Suggestions for improvement: 

Additional information:

Comment 1 Andreas Karis 2020-11-05 14:08:27 UTC

The explain does not help, either:

[root@openshift-jumpserver-0 ~]# oc explain featuregate.spec.featureSet
KIND:     FeatureGate
VERSION:  config.openshift.io/v1

FIELD:    featureSet <string>

DESCRIPTION:
     featureSet changes the list of features in the cluster. The default is
     empty. Be very careful adjusting this setting. Turning on or off features
     may cause irreversible changes in your cluster which cannot be undone.

Comment 2 Andreas Karis 2020-11-05 14:18:13 UTC

https://docs.openshift.com/container-platform/4.6/nodes/clusters/nodes-cluster-enabling-features.html

i) Alternatively, is there just a mismatch between the table, saying:
The following features are affected by FeatureGates: RotateKubeletServerCertificate SupportPodPidsLimit MachineHealthCheck LocalStorageCapacityIsolation

And between the next section:

The following Technology Preview features are enabled by feature gates:
RotateKubeletServerCertificate
SupportPodPidsLimit

One section lists 4 features, the other lists 2 which are enabled by:
featureSet: TechPreviewNoUpgrade

ii) Looking at:
https://github.com/openshift/api/blob/master/config/v1/types_feature.go#L25

We only have 4 featuresets:

var (
// Default feature set that allows upgrades.
Default FeatureSet = ""

// TechPreviewNoUpgrade turns on tech preview features that are not part of the normal supported platform. Turning
// this feature set on CANNOT BE UNDONE and PREVENTS UPGRADES.
TechPreviewNoUpgrade FeatureSet = "TechPreviewNoUpgrade"

// CustomNoUpgrade allows the enabling or disabling of any feature. Turning this feature set on IS NOT SUPPORTED, CANNOT BE UNDONE, and PREVENTS UPGRADES.
// Because of its nature, this setting cannot be validated. If you have any typos or accidentally apply invalid combinations
// your cluster may fail in an unrecoverable way.
CustomNoUpgrade FeatureSet = "CustomNoUpgrade"

// TopologyManager enables ToplogyManager support. Upgrades are enabled with this feature.
LatencySensitive FeatureSet = "LatencySensitive"

// IPv6DualStackNoUpgrade enables dual-stack. Turning this feature set on IS NOT SUPPORTED, CANNOT BE UNDONE, and PREVENTS UPGRADES.
IPv6DualStackNoUpgrade FeatureSet = "IPv6DualStackNoUpgrade"
)

So let me rephrase the above question: Should we document how to enable TechPreviewNoUpgrade and LatencySensitive at the same time? If so, how can this be done? The other 2 feature gates seem to be unsupported.

iii) The above source code is very specific about the negative impact on upgrades when enabling TechPreviewNoUpgrade:
~~~
// TechPreviewNoUpgrade turns on tech preview features that are not part of the normal supported platform. Turning
// this feature set on CANNOT BE UNDONE and PREVENTS UPGRADES.
~~~

But our doc does not contain this warning. Can we make this more visible in the doc, by adding the same warning?

Comment 6 Sunil Choudhary 2021-12-09 10:27:09 UTC

Reviewing doc section [1], I see it is already mentioned that enabling TechPreviewNoUpgrade also enables RotateKubeletServerCertificate

[1] https://docs.openshift.com/container-platform/4.6/nodes/clusters/nodes-cluster-enabling-features.html#nodes-cluster-enabling-features-about_nodes-cluster-enabling