Bug 1991770 - The logLevel and operatorLogLevel values do not work with Cloud Credential Operator
Summary: The logLevel and operatorLogLevel values do not work with Cloud Credential Op...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cloud Credential Operator
Version: 4.6
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.10.0
Assignee: Nobody
QA Contact: Jianping SHu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-08-10 00:11 UTC by Michael Washer
Modified: 2022-03-12 04:37 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Setting the log level in the cloudcredentials.operator.openshift.io custom resource would be ignored. Consequence: Could not set logging level on the cloud-credential-operator by editing the cloudcredentials custom resource. Fix: Respect the values in the cloudcredentials.operator.openshift.io custom resource to affect the logging output of the cloud-credential-operator. Result: Logging verbosity can be controlled via the cloudcredentials.operator.openshift.io custom resource.
Clone Of:
Environment:
Last Closed: 2022-03-12 04:37:24 UTC
Target Upstream Version:
Embargoed:
arane: needinfo+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cloud-credential-operator pull 379 0 None open Bug 1991770: The logLevel and operatorLogLevel values do not work with Cloud Credential Operator 2021-09-15 12:11:21 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-12 04:37:46 UTC

Description Michael Washer 2021-08-10 00:11:30 UTC
Description of problem:
The logLevel and operatorLogLevel values are not registered by the Cloud Credential Operator (CCO) and there is no way to change the log-level using this CR.

The options are described here:
https://docs.openshift.com/container-platform/4.8/rest_api/operator_apis/cloudcredential-operator-openshift-io-v1.html

But as seen in the code, the log-level is for the CCO is only hardcoded or available for configuration with the `--log-level` flag.
https://github.com/openshift/cloud-credential-operator/blob/master/pkg/cmd/operator/cmd.go#L72-L79

Version-Release number of selected component (if applicable):
OpenShift 4.x
Cloud Credential Operator

How reproducible:
Every time

Steps to Reproduce:
1. Change the `cloudcredentials.spec.logLevel` and `cloudcredentials.spec.operatorLogLevel` options to Debug, Trace or TraceAll
2. Restart the CloudCredential Pods
3. Loglevel is not increased

Actual results:
Loglevel is not increased


Expected results:
LogLevel is increased immediately or upon restart

Additional info:

Comment 4 Jianping SHu 2021-09-16 06:10:28 UTC
Verified with 4.10.0-0.ci-2021-09-15-210454
https://amd64.ocp.releases.ci.openshift.org/releasestream/4.10.0-0.ci/release/4.10.0-0.ci-2021-09-15-210454

1. Check default logLevel and operatorLogLevel are Normal
$oc get pods -n openshift-cloud-credential-operator
NAME                                         READY   STATUS    RESTARTS   AGE
cloud-credential-operator-6957c874f9-sr5j7   2/2     Running   0          31m
pod-identity-webhook-569d7844b-8n8l4         1/1     Running   0          22m

$oc edit cloudcredential.operator.openshift.io
...
spec:
  credentialsMode: ""
  logLevel: Normal
  operatorLogLevel: Normal

2. check logs, only msg of level=info
$oc logs cloud-credential-operator-6957c874f9-sr5j7 -c cloud-credential-operator -n openshift-cloud-credential-operator
Copying system trust bundle
time="2021-09-16T02:10:34Z" level=info msg="setting up client for manager"
time="2021-09-16T02:10:34Z" level=info msg="running file observer" file=/var/run/configmaps/trusted-ca-bundle/tls-ca-bundle.pem
I0916 02:10:34.121599       1 observer_polling.go:159] Starting file observer
time="2021-09-16T02:10:34Z" level=info msg="generated leader election ID" id=1a1d2895-84ec-4168-86ca-db99e18fe9ae

3. Change logLevel and operatorLogLevel to Debug, TraceAll, Trace in order
$oc edit cloudcredential.operator.openshift.io
...
spec:
  credentialsMode: ""
  logLevel: Debug
  operatorLogLevel: Debug

4. Check log, there will be msg of both level=info and level=debug
time="2021-09-16T03:05:23Z" level=info msg="klog level changed to Debug"
time="2021-09-16T03:05:23Z" level=info msg="logrus level changed to debug"
time="2021-09-16T03:05:23Z" level=info msg="requeueing all CredentialsRequests"
time="2021-09-16T03:05:23Z" level=debug msg="received conditions from handler" controller=status handlerconditions="[]" statushandler=secretannotator
time="2021-09-16T03:05:23Z" level=debug msg="received conditions from handler" controller=status handlerconditions="[]" statushandler=awspodidentity
time="2021-09-16T03:05:23Z" level=debug msg="received conditions from handler" controller=status handlerconditions="[]" statushandler=loglevel
time="2021-09-16T03:05:23Z" level=debug msg="received conditions from handler" controller=status handlerconditions="[]" statushandler=cleanup
time="2021-09-16T03:05:23Z" level=info msg="validating cloud cred secret" controller=secretannotator
time="2021-09-16T03:05:23Z" level=info msg="requeueing all CredentialsRequests"
time="2021-09-16T03:05:23Z" level=debug msg="calculated upgradeable condition" condition="<nil>"
time="2021-09-16T03:05:23Z" level=info msg="syncing credentials request" controller=credreq cr=openshift-cloud-credential-operator/openshift-azure-cloud-controller-manager
time="2021-09-16T03:05:23Z" level=debug msg="ignoring cr as it is for a different cloud" controller=credreq cr=openshift-cloud-credential-operator/openshift-azure-cloud-controller-manager secret=openshift-cloud-controller-manager/azure-cloud-credentials

5. Change logLevel and operatorLogLevel back to Normal
$oc edit cloudcredential.operator.openshift.io
...
spec:
  credentialsMode: ""
  logLevel: Normal
  operatorLogLevel: Normal

6. check logs, only msg of level=info

Comment 8 errata-xmlrpc 2022-03-12 04:37:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.