Bug 1785445

Summary:	mgr/k8sevents does not account for incomplete events passed in from kubernetes
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Paul Cuzner <pcuzner>
Component:	Ceph-Mgr Plugins	Assignee:	Paul Cuzner <pcuzner>
Status:	CLOSED ERRATA	QA Contact:	Madhavi Kasturi <mkasturi>
Severity:	low	Docs Contact:
Priority:	medium
Version:	4.0	CC:	ceph-eng-bugs, ceph-qe-bugs, flucifre, gmeno, hyelloji, jefbrown, tchandra, tserlin
Target Milestone:	z1
Target Release:	4.1
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	ceph-14.2.8-64.el8cp, ceph-14.2.8-68.el7cp	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-07-20 14:20:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Paul Cuzner 2019-12-19 21:44:41 UTC

Description of problem:
The namespacewatcher in k8sevents pulls in events from kubernetes within the namespace the mgr is running under. If these events have an improper/non-existent  timestamp, the 'ls' command fails when it tries to sort the events.

The impact is low, since k8sevents carries on - but it means that from a ceph console you no longer have visibility of pod events within the namespace

Version-Release number of selected component (if applicable):
OCS 4.2.4

How reproducible:
Whenever the k8sevents module is loaded

Steps to Reproduce:
1. deploy OCS
2. launch the toolbox and enable the module
3. issue ceph k8sevents ls command

Actual results:
command fails referring to NoneType and datetime.datetime comparisoon

Expected results:
ls should show all events in the namespace, sorted - most recent first

Additional info:

Comment 3 Yaniv Kaul 2020-02-26 16:17:21 UTC

Jeff?

Comment 7 Yaniv Kaul 2020-04-22 15:54:40 UTC

This one has the blocker flag set for 4.1. Can we ensure this is worked on, moved or closed?

Comment 8 Jeff Brown 2020-04-22 16:33:46 UTC

I will review with Paul when he's online.

Comment 9 Jeff Brown 2020-04-22 16:34:06 UTC

I will review with Paul when he's online.

Comment 10 Federico Lucifredi 2020-04-22 21:12:49 UTC

This is not an issue for RHCS.  This is not a regression for Ceph (OCS). 

The bug does not meet blocker criteria. Moving to 4.1z1.

Comment 11 Paul Cuzner 2020-04-22 22:22:21 UTC

+1 This is not a blocker for 4.1

Some background;
- k8sevents is NOT currently enabled by default in either RHCS or OCS4.x
- k8sevents performs two tasks
  - it sends ceph events (pool created, osd added/removed, pool protection level changed, host added/removed) to the k8s events API(this is the main feature of the module and it's not affected by the bug)
  - it pulls events from the namespace that the mgr is running in, so at the ceph command line the admin can see k8sevents happening in the namespace. This was a secondary feature, and IS where the bug lies

To hit the bug in OCS the user would need to start the toolbox pod, rsh to it, and run the ceph k8sevents ls command. However, this workflow is not exactly normal.

Comment 15 Paul Cuzner 2020-06-11 02:09:42 UTC

upstream is merged (master) - upstream backport to do

downstream packport complete, commit pushed to ceph-4.1-rhel-patches

Comment 22 errata-xmlrpc 2020-07-20 14:20:55 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3003