Bug 1970102 - Update 'mig-log-reader' filters to reduce noise caused by Velero 1.6 log strings
Summary: Update 'mig-log-reader' filters to reduce noise caused by Velero 1.6 log strings
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Migration Toolkit for Containers
Classification: Red Hat
Component: Controller
Version: 1.5.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 1.5.0
Assignee: Derek Whatley
QA Contact: Xin jiang
Avital Pinnick
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-06-09 19:54 UTC by Derek Whatley
Modified: 2021-07-28 04:08 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-28 04:08:13 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github konveyor mig-log-reader pull 14 0 None open Bug 1970102: Add new log exclusion for no default backup location 2021-06-09 20:12:58 UTC
Github konveyor mig-log-reader pull 15 0 None open Bug 1970102: [Cherry-pick] Add new log exclusion for no default backup location 2021-06-10 18:27:06 UTC
Red Hat Product Errata RHEA-2021:2929 0 None None None 2021-07-28 04:08:22 UTC

Description Derek Whatley 2021-06-09 19:54:17 UTC
Description of problem:
See title.

Examples of new noisy log messages that dominate logs. This is not exahustive. Would be good to run through a migration and decide on a new list of log messages to filter.

openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=info msg="Backup storage location valid, marking as available" backup-storage-location=aws-s3-mb5d8 controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:121"
openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=info msg="Validating backup storage location" backup-storage-location=aws-s3-dlzjc controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:114"
openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=info msg="Backup storage location valid, marking as available" backup-storage-location=aws-s3-dlzjc controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:121"
openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=warning msg="There is no existing backup storage location set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:173"
openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=warning msg="There is no existing backup storage location set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:173"
openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=warning msg="There is no existing backup storage location set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:173"
openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=warning msg="There is no existing backup storage location set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:173"
openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=warning msg="There is no existing backup storage location set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:173"
openshift-migration velero-77cd56899c-qkdnc velero time="2021-06-09T19:52:51Z" level=warning msg="There is no existing backup storage location set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:173"


Exclusions need to be updated in https://github.com/konveyor/mig-log-reader/blob/master/stern.sh

Comment 5 Sergio 2021-06-25 13:52:02 UTC
Verified using MTC 1.5.0

    openshift-migration-rhel7-operator@sha256:00e77706ca22bcb557d13c16822180fc877e6ea1639a72fda8eb9f5488b039a2
    - name: MIG_CONTROLLER_REPO
      value: openshift-migration-controller-rhel8@sha256
    - name: MIG_CONTROLLER_TAG
      value: 7f657df15e9514df4ef42da3431f558a19b8d3233a2ef1222cd8e27793c93816
    - name: MIG_UI_REPO
      value: openshift-migration-ui-rhel8@sha256
    - name: MIG_UI_TAG
      value: bda1959e1f44170905499b304ce28f9f95ee1841b0dc29f9a9e2790d0dd0bd71
    - name: MIG_LOG_READER_REPO
      value: openshift-migration-log-reader-rhel8@sha256
    - name: MIG_LOG_READER_TAG
      value: 5b89aaf00f8a660905c69ea41cc9ed2c7822f20f02a3b85ff3c4b008b8b5ced4


We can find the excluded texts in the stern.sh file

$ oc exec migration-log-reader-58dc94fbf8-47hhd cat /usr/bin/stern.sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Defaulting container name to color.
Use 'oc describe pod/migration-log-reader-58dc94fbf8-47hhd -n openshift-migration' to see all of the containers in this pod.
#!/bin/bash
CONFIGPATH=/var/cache/sa2kubeconfig/kubeconfig-${KSUFFIX}

stern -l app.kubernetes.io/part-of=openshift-migration \
--color ${STERN_COLOR} \
--exclude-container discovery \
--exclude "watch is too old"  \
--exclude "Found new dockercfg secret" \
--exclude "specified default backup location" \
--exclude "needs to be at least 1 backup location" \
--exclude "No backup locations were ready to be verified" \
--exclude "Backup cannot be garbage-collected" \
--exclude "Error checking repository for stale locks" \
--exclude "Backup has expired" \
--exclude "error getting backup storage location" \
--exclude "There is no existing backup storage location set as default" \
--since 5s \
--all-namespaces \
--kubeconfig ${CONFIGPATH} \



We can find the log message in the velero pod, but we cannot find it in the migration-logs pod. So it is effectively excluded

$ oc logs velero-5bb4644844-wkm2x  | grep "location set as default" | wc -l
382
$ oc logs migration-log-reader-58dc94fbf8-47hhd -c plain |grep "location set as default" | wc -l
0

Moved to VERIFIED status.

Comment 11 errata-xmlrpc 2021-07-28 04:08:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Migration Toolkit for Containers (MTC) image release advisory 1.5.0), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:2929


Note You need to log in before you can comment on or make changes to this bug.