Bug 1413492 - Event messages say "(events with common reason combined)" and doesn't tell us the reason
Summary: Event messages say "(events with common reason combined)" and doesn't tell us...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
low
medium
Target Milestone: ---
: ---
Assignee: Derek Carr
QA Contact: DeShuai Ma
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-16 07:27 UTC by Kenjiro Nakayama
Modified: 2020-02-14 18:28 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-31 22:54:23 UTC
Target Upstream Version:

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2858401 0 None None None 2017-01-16 07:35:13 UTC

Description Kenjiro Nakayama 2017-01-16 07:27:27 UTC 
Description of problem:

- The system keeps producing DeletingLoadBalancerFailed error, but the Message is "(events with common reason combined)" and cannot find the root cause.

  $ oc get events
  LASTSEEN   FIRSTSEEN   COUNT     NAME                       KIND      SUBOBJECT   TYPE      REASON                       SOURCE                  MESSAGE
  1m         40d         3589450   xxx-sample-api-dev        Service               Normal    DeletingLoadBalancer         {service-controller }   Deleting load balancer
  1m         40d         3589441   xxx-sample-api-dev        Service               Warning   DeletingLoadBalancerFailed   {service-controller }   (events with common reason combined)
  1m         37d         3027567   yyy-sample-api-sit        Service               Normal    DeletingLoadBalancer         {service-controller }   Deleting load balancer
  1m         37d         3027558   yyy-sample-api-sit        Service               Warning   DeletingLoadBalancerFailed   {service-controller }   (events with common reason combined)
  1m         42d         4057605   zzz-request-api-v1         Service               Normal    DeletingLoadBalancer         {service-controller }   Deleting load balancer
  1m         42d         4057596   zzz-request-api-v1         Service               Warning   DeletingLoadBalancerFailed   {service-controller }   (events with common reason combined)

Version-Release number of selected component (if applicable):
  OCP 3.3

How reproducible:
Steps to Reproduce:
  1. We couldn't determine the steps, but it suddenly started and all application produced the DeletingLoadBalancerFailed error.

Actual results:
  The message only says "(events with common reason combined)".

Expected results:
  The error should say some message which allow us to debug this issue.

Additional info:
- After restarting the master-controllers services, the Warning message has stopped.
Comment 4 Ben Bennett 2017-01-27 15:47:53 UTC 
Reassigning because the bug seems to be about the event aggregator... and the code in vendor/k8s.io/kubernetes/pkg/client/record/events_cache.go at line 94 talks about prefixing the message... but just returns a constant string without the original message.
Comment 5 Derek Carr 2017-01-31 22:54:23 UTC 
The event aggregator intentionall collapses all events with the same reason to the fixed message in order to prevent event spam.  It is possible to lose the original source of the event message if the original event prior to aggregation has been purged from the data store.

The "Could not determine public IP message" is spam that is being removed here:
https://github.com/kubernetes/kubernetes/pull/40763

If the load balancer itself is not being deleted, please open a separate bug, but I am closing this bug as the event compaction is per the intended design.
Note You need to log in before you can comment on or make changes to this bug.