Bug 1765253

Summary:	Crash if IO in progress during VDO suspend/stop
Product:	Red Hat Enterprise Linux 8	Reporter:	Sweet Tea Dorminy <sweettea>
Component:	kmod-kvdo	Assignee:	Sweet Tea Dorminy <sweettea>
Status:	CLOSED ERRATA	QA Contact:	Filip Suba <fsuba>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	8.1	CC:	awalsh, cmarthal, corwin, fsuba, rhandlin
Target Milestone:	rc	Flags:	pm-rhel: mirror+
Target Release:	8.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	6.2.2.83	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1765255 (view as bug list)		Environment:
Last Closed:	2020-04-28 16:43:10 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1765255

Description Sweet Tea Dorminy 2019-10-24 16:01:04 UTC

Description of problem:
If IO is in progress when VDO receives a suspend request, for instance to grow logical, grow physical, or shut down, it is possible for some IO to run while or after VDO internally suspends. This can cause crashes, infinite loops, or assorted other chaos.

Version-Release number of selected component (if applicable):
6.2.1

How reproducible:
Somewhat difficult. Most likely if the VDO has a lot of in-memory dirty metadata to write out.

Steps to Reproduce:
1. Write 15G unique data in 4k randwrites, to a VDO with 15T logical space and a 15G block map cache size, in sync mode.
2. dmsetup suspend vdo0 & dd if=/dev/urandom of=/dev/mapper/vdo0 oflag=direct bs=4k


Actual results:
Chaos, potentially a crashdump. One manifestation could be:

[1388725.540680] uds: kvdo0:journalQ: assertion "count to be initialized not in use" ((*journalValue == atomicLoad32(decrementCount))) failed at /builddir/build/BUILD/kvdo-a50744b1ca2aa461a761076d051a21612fd45aba/obj/./vdo/base/lockCounter.c:259
[1388725.540681] uds: kvdo0:journalQ: [backtrace]
[1388725.540683] CPU: 4 PID: 768 Comm: kvdo0:journalQ Kdump: loaded Tainted: P           OE    --------- -  - 4.18.0-145.el8.x86_64 #1
[1388725.540684] Hardware name: Supermicro X9SRE/X9SRE-3F/X9SRi/X9SRi-3F/X9SRE/X9SRE-3F/X9SRi/X9SRi-3F, BIOS 3.2 01/16/2015
[1388725.540685] Call Trace:
[1388725.540692]  dump_stack+0x5c/0x80
[1388725.540703]  assertionFailedLogOnly+0x49/0x70 [uds]
[1388725.540717]  ? enqueueWorkQueue+0x3e/0x80 [kvdo]
[1388725.540725]  ? noDefaultAction+0x10/0x10 [kvdo]
[1388725.540734]  ? scheduleOperationWithContext+0xee/0x130 [kvdo]
[1388725.540743]  ? kvdoGetCurrentThreadID+0xa/0x20 [kvdo]
[1388725.540752]  initializeLockCount+0x6f/0x80 [kvdo]
[1388725.540759]  ? prepareToAssignEntry+0x1d0/0x1d0 [kvdo]
[1388725.540765]  prepareToAssignEntry+0x17a/0x1d0 [kvdo]
[1388725.540772]  assignEntries.part.6+0x47/0xb0 [kvdo]
[1388725.540782]  workQueueRunner+0x1b9/0x660 [kvdo]
[1388725.540784]  ? finish_wait+0x80/0x80
[1388725.540793]  ? kvdoCompareDataVIOs+0x90/0x90 [kvdo]
[1388725.540795]  kthread+0x112/0x130
[1388725.540796]  ? kthread_flush_work_fn+0x10/0x10
[1388725.540798]  ret_from_fork+0x35/0x40

Expected results:
No crashdumps, no hangs.

Additional info:

Comment 2 Sweet Tea Dorminy 2020-01-29 17:04:26 UTC

*** Bug 1796132 has been marked as a duplicate of this bug. ***

Comment 4 Filip Suba 2020-03-20 08:12:01 UTC

Verified SanityOnly with kmod-kvod-6.2.2.117-65.el8. Regression testing passed.

Comment 6 errata-xmlrpc 2020-04-28 16:43:10 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1782