Bug 1707072 - no termination message provided by failing openshift-service-catalog-controller-manager-operator pod
Summary: no termination message provided by failing openshift-service-catalog-controll...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Service Catalog
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.1.0
Assignee: Dan Geoffroy
QA Contact: Jian Zhang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-06 18:04 UTC by Luis Sanchez
Modified: 2019-06-04 10:48 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:48:31 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 0 None None None 2019-06-04 10:48:42 UTC

Description Luis Sanchez 2019-05-06 18:04:09 UTC
The openshift-service-catalog-apiserver-operator pod does not provide a termination message, hindering debugging efforts when the pods are crash looping.

At minimum, the pod's terminationMessagePolicy should be "FallbackToLogsOnError".

See https://kubernetes.io/docs/tasks/debug-application-cluster/determine-reason-pod-failure/#customizing-the-termination-message

Expected Results:
The termination message should appear in a pod container's  .status.lastState.terminated.message field.

Comment 1 Jay Boyd 2019-05-06 19:22:27 UTC
This is actually set properly on the openshift-service-catalog-apiserver-operator pod:

This was committed April 16 via https://github.com/openshift/cluster-svcat-apiserver-operator/pull/46

Verifying with nightly 05-06-163240:

[jaboyd@jboyd openshift]$ oc get pods -n openshift-service-catalog-apiserver-operator -o yaml | grep Fall -A5 -B30
      uid: 43cb3e57-7032-11e9-a7ea-0a0d33364010
    resourceVersion: "7882"
    selfLink: /api/v1/namespaces/openshift-service-catalog-apiserver-operator/pods/openshift-service-catalog-apiserver-operator-5cfd6f4f5-5k479
    uid: c47e4333-7032-11e9-938c-0ebc19e0b298
  spec:
    containers:
    - args:
      - --config=/var/run/configmaps/config/config.yaml
      - -v=5
      command:
      - cluster-svcat-apiserver-operator
      - operator
      env:
      - name: IMAGE
        value: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:203e3a95d47f5e283cfc0822a5f4ffc7f9c038587c093c0d83de611e938671a0
      - name: OPERATOR_IMAGE_VERSION
        value: 4.1.0-0.nightly-2019-05-06-163240
      - name: OPERAND_IMAGE_VERSION
        value: 4.1.0-0.nightly-2019-05-06-163240
      image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1dd3b4443c13b156e686c9eabd5523cfd9a70416d12951815cf094b0f2a8b974
      imagePullPolicy: IfNotPresent
      name: operator
      ports:
      - containerPort: 8443
        name: metrics
        protocol: TCP
      resources:
        requests:
          memory: 50Mi
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: FallbackToLogsOnError





However there is not one on the cluster-svcat-controller-manager-operator.  

@sanchezl are you ok if I change this bug to track setting this on the controller-manager?

Comment 2 Luis Sanchez 2019-05-07 03:21:24 UTC
@jaboyd you are correct. I incorrectly flagged the openshift-service-catalog-apiserver-operator pod. Feel free to re-purpose this bug for the cluster-svcat-controller-manager-operator pod.

Comment 4 Jay Boyd 2019-05-07 14:56:59 UTC
I just submitted the pr (its a simple change).  Should be merged shortly.

Comment 8 Jian Zhang 2019-05-08 09:08:00 UTC
LGTM, verify it.
cluster-svcat-controller-manager-operator version info:
io.openshift.build.commit.id=9ac2a14537c4b880f0ac12f8acacb3c29c9823c7
Cluster version is 4.1.0-0.nightly-2019-05-08-012425

mac:~ jianzhang$ oc get pods -o yaml -n openshift-service-catalog-controller-manager-operator|grep terminationMessage -A 3
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: FallbackToLogsOnError
      volumeMounts:
      - mountPath: /var/run/configmaps/config
        name: config

mac:~ jianzhang$ oc get pods -o yaml -n openshift-service-catalog-apiserver-operator |grep terminationMessage -A 3
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: FallbackToLogsOnError
      volumeMounts:
      - mountPath: /var/run/configmaps/config
        name: config

mac:~ jianzhang$ oc get pods -o yaml -n openshift-service-catalog-apiserver|grep terminationMessage -A 3
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: FallbackToLogsOnError
      volumeMounts:
      - mountPath: /var/run/kubernetes-service-catalog
        name: apiserver-ssl
...

mac:~ jianzhang$ oc get pods -o yaml -n openshift-service-catalog-controller-manager|grep terminationMessage -A 3
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: FallbackToLogsOnError
      volumeMounts:
      - mountPath: /var/run/kubernetes-service-catalog
        name: apiserver-ssl
...

Comment 11 errata-xmlrpc 2019-06-04 10:48:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.