Bug 1696726

Summary: [OCP4 Beta] [marketplace] NodeSelector missing for openshift-marketplace
Product: OpenShift Container Platform Reporter: Derek Carr <decarr>
Component: OLMAssignee: Aravindh Puthiyaparambil <aravindh>
OLM sub component: OperatorHub QA Contact: Fan Jia <jfan>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: high CC: aravindh, florin-alexandru.peter, jforrest, sauchter, scuppett, shurley
Version: 4.1.0   
Target Milestone: ---   
Target Release: 4.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: marketplace operator always overwrites the pod deployment spec of registry deployments associated with OperatorSources Consequence: Users cannot NodeSelectors Fix: Only replace the required fields in the deployment spec on OperatorSource updates, allowing users to add NodeSelectors to the operator registry pod associated with OperatorSources. By default no NodeSelector will be present by design. Result: Users can NodeSelectors to the pod spec in the registry pod deployments. Example: For the "community-operators" OperatorSource, the user would edit the "community-operators" deployment in the "openshift-marketplace" namespace.
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-16 06:28:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Derek Carr 2019-04-05 13:58:21 UTC
Description of problem:

The operators that are responsible for the pods in the namespaces 
- openshift-marketplace

have no option to configure the workload via an NodeSelector.

Version-Release number of selected component (if applicable):

OCP4 HTB 3

How reproducible:

100%

Steps to Reproduce:
1. Deploy cluster
2. There is no NodeSelector available to configure the workload as desired

see: https://bugzilla.redhat.com/show_bug.cgi?id=1696444

Comment 1 Jian Zhang 2019-04-08 05:47:12 UTC
Derek,

The marketplace-operator runs on the master nodes, and its "replicas" is 1. As below:
mac:~ jianzhang$ oc get deployment marketplace-operator -o yaml|grep -i nodeSelector -A 2
      nodeSelector:
        node-role.kubernetes.io/master: ""
      priorityClassName: system-cluster-critical

mac:~ jianzhang$ oc get deployment marketplace-operator -o yaml|grep replicas
  replicas: 1
  replicas: 1

mac:~ jianzhang$ oc get pods -o wide
NAME                                    READY     STATUS    RESTARTS   AGE       IP            NODE                                           NOMINATED NODE   READINESS GATES
certified-operators-7bd7768fcf-lbxb2    1/1       Running   0          3h49m     10.128.2.6    ip-172-31-130-114.us-east-2.compute.internal   <none>           <none>
community-operators-74d75b747c-d7rvr    1/1       Running   0          3h49m     10.131.0.7    ip-172-31-151-102.us-east-2.compute.internal   <none>           <none>
marketplace-operator-5d8bf7668f-r28pn   1/1       Running   0          3h49m     10.128.0.32   ip-172-31-158-104.us-east-2.compute.internal   <none>           <none>
redhat-operators-6865f9b7b5-g9xm4       1/1       Running   0          3h49m     10.128.2.7    ip-172-31-130-114.us-east-2.compute.internal   <none>           <none>

Comment 2 Florin Peter 2019-04-08 06:32:59 UTC
Hi Jian,

only the marketplace-operator is running on the masters, all others pods in the namespace openshift-marketplace are not bound via nodeSelector to the masters.

oc get po -n openshift-marketplace -o wide
NAME                                    READY   STATUS    RESTARTS   AGE     IP              NODE                                      NOMINATED NODE
certified-operators-7dc989fc9d-fmzl7    1/1     Running   0          3d18h   192.168.20.11   ocp4-nmz4m-infra-s2large4-eude03-nxbfd    <none>
community-operators-5995cd994b-szhrq    1/1     Running   0          3d18h   192.168.20.12   ocp4-nmz4m-infra-s2large4-eude03-nxbfd    <none>
marketplace-operator-6ffff946cc-24vm7   1/1     Running   0          3d18h   192.168.2.58    ocp4-nmz4m-master-2                       <none>
redhat-operators-6c5595fd66-mk224       1/1     Running   0          3d18h   192.168.14.9    ocp4-nmz4m-worker-s2large4-eude03-rwmgn   <none>

From my point of view it will be good to be able to bind the pods to the infra nodes.

Comment 3 Aravindh Puthiyaparambil 2019-04-08 21:36:44 UTC
certified-operators, redhat-operators and community-operator pods are ones associated with CatalogSources that are created as a result of adding those OperatorSources (which is a CR that the marketplace operator watches) to the cluster. They are not part of the marketplace operator. Given that, they shouldn't be bound to the master node via a nodeSelector.

Comment 4 Aravindh Puthiyaparambil 2019-04-10 16:53:01 UTC
After discussing with Jessica, we concluded that the grpc CatalogSource pods should *not* be running on master as they are secondary resources. Moreover they should be launched with multiple replicas in such a way that the OperatorHub experience does not degrade. Currently we shouldn't see any degradation given the way CatalogSource are synced, but we should not depend on that. This is something we can fix in the 4.2 time frame rather introducing this change at this juncture.

Comment 10 errata-xmlrpc 2019-10-16 06:28:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922