Bug 1832652
| Summary: | The kibana pod doesn't use the node selector configurations. | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Qiaoling Tang <qitang> |
| Component: | Logging | Assignee: | Periklis Tsirakidis <periklis> |
| Status: | CLOSED ERRATA | QA Contact: | Anping Li <anli> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 4.5 | CC: | aos-bugs, periklis |
| Target Milestone: | --- | ||
| Target Release: | 4.5.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause:
Kibana CRD is a managed resource by elasticsearch-operator in 4.5. The cluster-logging-operator only creates a custom resource for kibana and not the kibana pods anymore.
Consequence:
Missed to passed node selectors from ClusterLogging CR to Kibana CR.
Fix:
Pass node selectors to Kibana CR
Result:
A Kibana CR with custom node selectors is reconciled by the elasticsearch-operator to a pod spec with customized resources for the kibana container and for the kibana proxy container.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-07-13 17:35:44 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1832662 | ||
| Bug Blocks: | |||
Move manually to MODIFIED because same fix as https://bugzilla.redhat.com/show_bug.cgi?id=1832662 Verified with images from 4.5.0-0.ci-2020-05-14-224329.
spec:
managementState: Managed
nodeSelector:
logging: test
proxy:
resources:
limits:
memory: 256Mi
requests:
cpu: 100m
memory: 256Mi
replicas: 1
resources:
limits:
memory: 736Mi
requests:
cpu: 100m
memory: 736Mi
status:
- clusterCondition:
kibana-7cbdff4bfd-mz8kb:
- lastTransitionTime: "2020-05-15T07:13:06Z"
message: '0/6 nodes are available: 6 node(s) didn''t match node selector.'
reason: Unschedulable
status: "True"
type: Unschedulable
$ oc get deploy kibana -oyaml |grep -A 3 nodeSelector
f:nodeSelector:
.: {}
f:kubernetes.io/os: {}
f:logging: {}
--
nodeSelector:
kubernetes.io/os: linux
logging: test
restartPolicy: Always
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409 |
Description of problem: Deploy clusterlogging, set nodeSelector for kibana in the clusterlogging instance, then check the logging pods, there doesn't have nodeSelector in the kibana instance, deployment and pods. $ oc get clusterlogging instance -oyaml apiVersion: logging.openshift.io/v1 kind: ClusterLogging metadata: creationTimestamp: "2020-05-07T02:01:30Z" generation: 1 name: instance namespace: openshift-logging resourceVersion: "65950" selfLink: /apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterloggings/instance uid: de323da1-5cb4-468e-8526-1f26ef8a7ac4 spec: collection: logs: fluentd: nodeSelector: logging: test type: fluentd logStore: elasticsearch: nodeCount: 3 nodeSelector: logging: test redundancyPolicy: SingleRedundancy resources: requests: memory: 2Gi storage: size: 20Gi storageClassName: standard retentionPolicy: application: maxAge: 1d audit: maxAge: 1w infra: maxAge: 7d type: elasticsearch managementState: Managed visualization: kibana: nodeSelector: logging: test replicas: 1 type: kibana No nodes have the label `logging=test`, but the kibana pod could be scheduled to the worker nodes. $ oc get pod NAME READY STATUS RESTARTS AGE cluster-logging-operator-75774d56b6-jpr22 1/1 Running 0 76m elasticsearch-cdm-3d1zpsdr-1-5c57b569b8-v777x 0/2 Pending 0 4m26s elasticsearch-cdm-3d1zpsdr-2-66fd465db9-jr5rv 0/2 Pending 0 3m25s elasticsearch-cdm-3d1zpsdr-3-69b56bbf7d-64cmk 0/2 Pending 0 2m24s kibana-779ff5c9d5-9tgcj 2/2 Running 0 4m26s $ oc get kibana instance -oyaml apiVersion: logging.openshift.io/v1 kind: Kibana metadata: creationTimestamp: "2020-05-07T02:01:30Z" generation: 1 name: instance namespace: openshift-logging ownerReferences: - apiVersion: logging.openshift.io/v1 controller: true kind: ClusterLogging name: instance uid: de323da1-5cb4-468e-8526-1f26ef8a7ac4 resourceVersion: "65585" selfLink: /apis/logging.openshift.io/v1/namespaces/openshift-logging/kibanas/instance uid: 91a54360-4f08-41d0-ab80-56bf74e7cc3d spec: image: "" managementState: Managed proxy: image: "" resources: null replicas: 1 resources: limits: memory: 736Mi requests: cpu: 100m memory: 736Mi status: - deployment: kibana pods: failed: [] notReady: [] ready: - kibana-779ff5c9d5-9tgcj replicaSets: - kibana-779ff5c9d5 replicas: 1 $ oc get deploy kibana -oyaml |grep -A 3 nodeSelector f:nodeSelector: .: {} f:kubernetes.io/os: {} f:restartPolicy: {} -- nodeSelector: kubernetes.io/os: linux restartPolicy: Always schedulerName: default-scheduler Version-Release number of selected component (if applicable): Logging images are from 4.5.0-0.ci-2020-05-06-225918 Manifests are copied from the master branch Cluster version: 4.5.0-0.nightly-2020-05-06-003431 How reproducible: Always Steps to Reproduce: 1.deploy clusterlogging with: apiVersion: "logging.openshift.io/v1" kind: "ClusterLogging" metadata: name: "instance" namespace: "openshift-logging" spec: managementState: "Managed" logStore: type: "elasticsearch" retentionPolicy: application: maxAge: 1d infra: maxAge: 7d audit: maxAge: 1w elasticsearch: nodeCount: 3 nodeSelector: logging: test redundancyPolicy: "SingleRedundancy" resources: requests: memory: "2Gi" storage: storageClassName: "standard" size: "20Gi" visualization: type: "kibana" kibana: nodeSelector: logging: test replicas: 1 collection: logs: type: "fluentd" fluentd: nodeSelector: logging: test note: no nodes have label `logging=test` in the cluster 2. check pods status 3. Actual results: Kibana doesn't use the node selector in the clusterlogging instance. Expected results: The Kibana should use the node selector configurations in the clusterlogging instance. Additional info: