Description of problem:

This is a problem we have been seeing in our nightly testing.  We do not have a way of reproducing the failure, but once or twice a month we see this kind of report:

Apr  2 09:33:00 parkst-31 kernel: [25652.104178] uds: zub:indexW: assertion "Delta list number (173) is out of range (171)" ((listNumber < deltaIndex->numLists)) failed at /var/lib/dkms/zubenelgenubi/1/build/uds/deltaIndex.c:1062
...
Apr  2 09:33:00 parkst-31 kernel: [25652.104326] uds: zub:indexW: index_0: searchLockedIndex() failed: UDS Error: Index data in memory is corrupt (1036)

Our RHEL8 work has required us to examine our multithreading of our code.  Historically we have coded to work on x86 hardware, but for RHEL8 we need to pay attention to proper use of atomic values and memory barriers.  As part of this work, we have found data race conditions in the UDS code that can lead to the failure asserted above in kern.log.


Version-Release number of selected component (if applicable):

All released versions of kmod-kvdo have these problems.


How reproducible:

Very difficult.  We see failures once or twice a month in our nightly testing.  The actual races were found by inspecting the source code.


Steps to Reproduce:
1. ??
2. ??

Actual results:
Expected results:

Obviouely no report of corruption.  The best suggestion is to run sanity tests and any basic operational testing that QA has developed.


Additional info:

We want to merge the RHEL8 work related to these races back to RHEL7.6.  It is a large change, but we have been testing it nightly in our RHEL8 code stream.