Bug 1947248

Summary: Compliance Operator fails to deploy if a defaultNodeSelector is set in Scheduler config - cluster
Product: OpenShift Container Platform Reporter: Sunil Thaha <sthaha>
Component: DocumentationAssignee: Ashley Hardin <ahardin>
Status: CLOSED CURRENTRELEASE QA Contact: Xiaoli Tian <xtian>
Severity: medium Docs Contact: Vikram Goyal <vigoyal>
Priority: medium    
Version: 4.6CC: ahardin, aos-bugs, jhrozek, jokerman, josorior, mrogers, xiyuan
Target Milestone: ---   
Target Release: 4.8.0   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
Cause: Setting a default nodeSelector may add an unexpected value to the default nodeSelectors the compliance scans use Consequence: This causes the operator to not be able to schedule all the scans it needs in order to do a compliance check Workaround (if any): The openshift-compliance namespace (or the namespace where the compliance operator was installed) may need to be annotated with: openshift.io/node-selector: "" To remove the default node selector. Result: Once this is done, the operator should work as usual.
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-05 20:16:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sunil Thaha 2021-04-08 05:08:49 UTC
Description of problem:
Compliance Operator fails to deploy (pods in pending status for ever) if the user has defaultNodeSelector  set in scheduler - cluster. 

Version-Release number of selected component (if applicable): 0.1.29 


How reproducible: Always

Steps to Reproduce:
1. oc edit scheduler cluster

2. set `defaultNodeSelector` to worker as below 
   defaultNodeSelector: node-role.kubernetes.io/worker=

3. Use OLM to install the Compliance Operator 

Actual results:

The Compliance Operator pod gets created but is in `Pending` state of ever. 
Inspecting the Pod shows that its nodeSelector now has 2 values 

nodeSelector:
    node-role.kubernetes.io/master: ""
    node-role.kubernetes.io/worker: ""
 
Thus making none of the nodes match the selector 



Expected results: The operator should be in `Running` state 


Additional info:

There exists a workaround for this which is to create apply the following annotation to openshift-compliance (default project that OLM chooses) project

 openshift.io/node-selector: “”

Comment 1 Juan Antonio Osorio 2021-04-08 05:51:44 UTC
Note that this is an issue when installing the operator through the UI. When installing the operator through the CLI, one needs to create the namespace manually, and thus is able to set the needed annotations. We'll investigate if we can indicate to the UI what annotations we need.

Either way, we need to document this scenario. Thanks for reporting this!

Comment 2 Ashley Hardin 2021-05-05 19:24:15 UTC
The docs update is in https://github.com/openshift/openshift-docs/pull/31746 and QE verified in the PR