Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1570156

Summary: Creating a VDO volume with --sparseIndex enabled results in an index on-disk size matching that of a dense index
Product: Red Hat Enterprise Linux 7 Reporter: Bryan Gurney <bgurney>
Component: vdoAssignee: Sweet Tea Dorminy <sweettea>
Status: CLOSED ERRATA QA Contact: Jakub Krysl <jkrysl>
Severity: unspecified Docs Contact:
Priority: high    
Version: 7.5CC: awalsh, jkrysl, limershe, rhandlin
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 6.1.1.79 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1572496 (view as bug list) Environment:
Last Closed: 2018-10-30 09:39:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1572496    

Description Bryan Gurney 2018-04-20 18:27:31 UTC 
Description of problem:

When creating a VDO volume with default options, a UDS index using 0.25 GB of memory is created, which uses about 2.6 GB of disk space:

# vdo create --name=vdo1 --device=/dev/nvme0n1p1 --verbose
Creating VDO vdo1
    grep MemAvailable /proc/meminfo
    pvcreate -qq --test /dev/nvme0n1p1
    modprobe kvdo
    vdoformat --uds-checkpoint-frequency=0 --uds-memory-size=0.25 /dev/nvme0n1p1
    vdodumpconfig /dev/nvme0n1p1
Starting VDO vdo1
    dmsetup status vdo1
    grep MemAvailable /proc/meminfo
    modprobe kvdo
    vdodumpconfig /dev/nvme0n1p1
    dmsetup create vdo1 --uuid VDO-bee917e8-cc33-4c59-8943-623eebac2b1b --table '0 200940480 vdo /dev/nvme0n1p1 4096 disabled 0 32768 16380 on auto vdo1 ack=1,bio=4,bioRotationInterval=64,cpu=2,hash=1,logical=1,physical=1'
    dmsetup status vdo1
Starting compression on VDO vdo1
    dmsetup message vdo1 0 compression on
    dmsetup status vdo1
VDO instance 11 volume is ready at /dev/mapper/vdo1
# vdo status | grep Index
    Index checkpoint frequency: 0
    Index memory setting: 0.25
    Index parallel factor: 0
    Index sparse: disabled
    Index status: online
# cat /sys/uds/index/12/name
dev=/dev/nvme0n1p1 offset=4096 size=2781704192
# hexdump -C -n 4096 /dev/nvme0n1p1
00000000  64 6d 76 64 6f 30 30 31  05 00 00 00 04 00 00 00  |dmvdo001........|
00000010  00 00 00 00 5d 00 00 00  00 00 00 00 09 01 02 00  |....]...........|
00000020  bb 7d 8a 70 4b 6a 05 00  be e9 17 e8 cc 33 4c 59  |.}.pKj.......3LY|
00000030  89 43 62 3e eb ac 2b 1b  00 00 00 00 01 00 00 00  |.Cb>..+.........|
00000040  00 00 00 00 01 00 00 00  d8 5c 0a 00 00 00 00 00  |.........\......|
00000050  00 ff ff ff 00 00 00 00  00 68 2e d4 e7 00 00 00  |.........h......|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000


If the user attempts to create a VDO volume with default options, except adding "--sparseIndex=enabled" to create a sparse index.  This should result in about 25 GB of disk space being used for the index, but it doesn't; instead it seems to show the same "size=2781704192" string in the /sys/uds "name", which suggests the on-disk index is about 2.6 GB in size. 

# vdo create --name=vdo1 --device=/dev/nvme0n1p1 --sparseIndex=enabled --verbose
Creating VDO vdo1
    grep MemAvailable /proc/meminfo
    pvcreate -qq --test /dev/nvme0n1p1
    modprobe kvdo
    vdoformat --uds-checkpoint-frequency=0 --uds-memory-size=0.25 --uds-sparse /dev/nvme0n1p1
    vdodumpconfig /dev/nvme0n1p1
Starting VDO vdo1
    dmsetup status vdo1
    grep MemAvailable /proc/meminfo
    modprobe kvdo
    vdodumpconfig /dev/nvme0n1p1
    dmsetup create vdo1 --uuid VDO-8b6dc126-f9dc-4898-9e68-cfe58ae8cf34 --table '0 200940480 vdo /dev/nvme0n1p1 4096 disabled 0 32768 16380 on auto vdo1 ack=1,bio=4,bioRotationInterval=64,cpu=2,hash=1,logical=1,physical=1'
    dmsetup status vdo1
Starting compression on VDO vdo1
    dmsetup message vdo1 0 compression on
    dmsetup status vdo1
VDO instance 12 volume is ready at /dev/mapper/vdo1
# vdo status | grep Index
    Index checkpoint frequency: 0
    Index memory setting: 0.25
    Index parallel factor: 0
    Index sparse: enabled
    Index status: online
# cat /sys/uds/index/13/name 
dev=/dev/nvme0n1p1 offset=4096 size=2781704192
# hexdump -C -n 4096 /dev/nvme0n1p1
00000000  64 6d 76 64 6f 30 30 31  05 00 00 00 04 00 00 00  |dmvdo001........|
00000010  00 00 00 00 5d 00 00 00  00 00 00 00 09 01 02 00  |....]...........|
00000020  84 ad 82 73 4b 6a 05 00  8b 6d c1 26 f9 dc 48 98  |...sKj...m.&..H.|
00000030  9e 68 cf e5 8a e8 cf 34  00 00 00 00 01 00 00 00  |.h.....4........|
00000040  00 00 00 00 01 00 00 00  d8 5c 0a 00 00 00 00 00  |.........\......|
00000050  00 ff ff ff 00 00 00 00  00 3d 47 39 ab 00 00 00  |.........=G9....|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000


Version-Release number of selected component (if applicable):
vdo-6.1.0.149-16.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create a VDO volume with the command "vdo create --name=vdo1 --device=<blockdevice> --sparseIndex=enabled" to try to create a VDO volume with a dense index.  Run "hexdump -C -n 4096 <blockdevice>" to dump the contents of the geometry block.  Run "vdo status", and save the output.  Remove this VDO volume.

2. Create a VDO volume with the command "vdo create --name=vdo1 --device=<blockdevice>" to try to create a VDO volume with a dense index.  Run "hexdump -C -n 4096 <blockdevice>" to dump the contents of the geometry block.  Run "vdo status", and save the output.  Remove this VDO volume.

3.  Compare the values in 0x48 to 0x4f of the geometry block; they should not be equal.

Actual results:
The values in 0x48 to 0x4f of the geometry block are equal.

Expected results:
The values in 0x48 to 0x4f of the geometry block are not equal (and the integer there is about 10 times larger).

Additional info:
Comment 3 Jakub Krysl 2018-07-03 13:29:38 UTC 
Tested on:
RHEL-7.6-20180626.0
kernel-3.10.0-915.el7
kmod-vdo-6.1.1.99-1.el7
vdo-6.1.1.99-2.el7

# vdo create --name vdo --device /dev/sdb
Creating VDO vdo
Starting VDO vdo
Starting compression on VDO vdo
VDO instance 109 volume is ready at /dev/mapper/vdo
# vdo status --name vdo | grep 'Index sparse'
    Index sparse: disabled
# hexdump -C /dev/sdb -n 4K
00000000  64 6d 76 64 6f 30 30 31  05 00 00 00 04 00 00 00  |dmvdo001........|
00000010  00 00 00 00 5d 00 00 00  00 00 00 00 09 01 02 00  |....]...........|
00000020  c5 ac 79 38 18 70 05 00  a3 c1 39 51 1e 12 44 0d  |..y8.p....9Q..D.|
00000030  a7 4a 8c a4 93 44 bd 9a  00 00 00 00 01 00 00 00  |.J...D..........|
00000040  00 00 00 00 01 00 00 00  d8 5c 0a 00 00 00 00 00  |.........\......|
00000050  00 ff ff ff 00 00 00 00  00 dc 64 0f 61 00 00 00  |..........d.a...|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000

0x0a5cd8 * 4096 = 2781708288

# vdo create --name vdo --device /dev/sdb --sparseIndex enabled
Creating VDO vdo
Starting VDO vdo
Starting compression on VDO vdo
VDO instance 110 volume is ready at /dev/mapper/vdo
# vdo status --name vdo | grep 'Index sparse'
    Index sparse: enabled
# hexdump -C /dev/sdb -n 4K
00000000  64 6d 76 64 6f 30 30 31  05 00 00 00 04 00 00 00  |dmvdo001........|
00000010  00 00 00 00 5d 00 00 00  00 00 00 00 09 01 02 00  |....]...........|
00000020  19 0c ce 40 18 70 05 00  7d 53 cd c9 6f c6 42 6f  |...@.p..}S..o.Bo|
00000030  96 d5 62 34 d0 5d b3 47  00 00 00 00 01 00 00 00  |..b4.].G........|
00000040  00 00 00 00 01 00 00 00  2c d7 58 00 00 00 00 00  |........,.X.....|
00000050  00 ff ff ff 00 00 00 00  01 fb 19 c1 7f 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000

0x58d72c * 4096 = 23847944192

23847944192 / 2781708288 = 8.57
So the sparse index on disk is roughly 8.57x bigger than dense index.
Regression testing passed.
Comment 5 errata-xmlrpc 2018-10-30 09:39:31 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3094