Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1516081

Summary: Do not destroy an old dedupe index unless it does not exist or is unrecoverable
Product: Red Hat Enterprise Linux 7 Reporter: Thomas Jaskiewicz <tjaskiew>
Component: kmod-kvdoAssignee: Thomas Jaskiewicz <tjaskiew>
Status: CLOSED ERRATA QA Contact: Jakub Krysl <jkrysl>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.5CC: awalsh, jkrysl, limershe, tjaskiew
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 6.1.0.63 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-10 16:26:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Thomas Jaskiewicz 2017-11-22 01:06:09 UTC
Description of problem:

The logs attached to bugzilla 1510567 showed a case of an attempt to load a dedupe index failing due to a kernel memory allocation failure, and the driver proceeded to throw away the dedupe index and create a new index.  While this is the correct behavior for a non-existant or corrupted index, it is not acceptable for a more normal system error.

Version-Release number of selected component (if applicable):
vdo-6.1.0.34-8
kmod-kvdo-6.1.0.34-7

How reproducible:
100%

Steps to Reproduce:
1. Create a VDO (with enough memory)
2. Shut down the VDO device.
3. Tie up enough system memory so that you can ...
4. Load the VDO device without sufficient memory
5. Observe the kern.log messages

Actual results:
A memory allocation failure is followed up by attempting to create a new index.

Expected results:
A memory allocation failure just fails, and preserves the existing index.

Additional info:

Comment 2 Jakub Krysl 2017-11-22 14:43:07 UTC
Tried to reproduce to be able to qe_ack, found this in log:
[15329.239460] uds: kvdo1:dedupeQ: Failed loading or rebuilding index: Cannot allocate memory (12) 
[15329.239466] uds: kvdo1:dedupeQ: creating index: dev=/dev/md127 offset=4096 size=2781704192 

acking...

Comment 3 Thomas Jaskiewicz 2017-11-22 23:23:22 UTC
We have fixed the primary problem (too aggressive about creating a new index).
We have implemented "dmsetup message index-create".

We still need vdoManager to invoke index-create.

Comment 4 Andy Walsh 2017-11-23 14:57:47 UTC
Are you saying that we will never re-create the index automatically now?  Is it now VDO manager's responsibility to re-create the index whenever there is any issue at all?  Or another way to say it is, when is VDO manager supposed to invoke 'index-create'?

Comment 7 Thomas Jaskiewicz 2017-11-27 22:01:02 UTC
We automatically create the index when we see that there is no index there.  This happens the first time we "dmsetup create" the VDO device.

We automatically re-create the index when we see a corrupted index.  We know that there is no way to recover such an index.  Any other error will not re-create an index.  In such a case, it will be necessary for VDO manager to do a "dmsetup message index-create" to create the index.

Comment 10 Jakub Krysl 2017-12-04 12:41:13 UTC
kmod-kvdo-6.1.0.72-10:

[  264.771730] uds: kvdo1:dedupeQ: Could not allocate 196608 bytes for sparse index page data in 1002 msecs 
[  264.771755] uds: kvdo1:dedupeQ: index_0: could not allocate index: Cannot allocate memory (12) 
[  264.777199] uds: kvdo1:dedupeQ: failed to create index: Cannot allocate memory (12) 
[  264.777202] uds: kvdo1:dedupeQ: Failed to make router: Cannot allocate memory (12) 
[  264.777209] uds: kvdo1:dedupeQ: Failed loading or rebuilding index: Cannot allocate memory (12) 
[  264.777214] kvdo1:dedupeQ: Error opening index dev=/dev/mapper/mpatha offset=4096 size=2781704192: System error 12 (12) 
[  264.777216] kvdo1:dedupeQ: Setting UDS index target state to error 

THe index is no longer rebuilt but its state is set to error. BZ Verified.

Comment 13 errata-xmlrpc 2018-04-10 16:26:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0900