Bug 1570156 - Creating a VDO volume with --sparseIndex enabled results in an index on-disk size matching that of a dense index
Summary: Creating a VDO volume with --sparseIndex enabled results in an index on-disk ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: vdo
Version: 7.5
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: rc
: ---
Assignee: Sweet Tea Dorminy
QA Contact: Jakub Krysl
URL:
Whiteboard:
Depends On:
Blocks: 1572496
TreeView+ depends on / blocked
 
Reported: 2018-04-20 18:27 UTC by Bryan Gurney
Modified: 2019-03-06 01:11 UTC (History)
4 users (show)

Fixed In Version: 6.1.1.79
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1572496 (view as bug list)
Environment:
Last Closed: 2018-10-30 09:39:31 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:3094 None None None 2018-10-30 09:40:09 UTC

Description Bryan Gurney 2018-04-20 18:27:31 UTC
Description of problem:

When creating a VDO volume with default options, a UDS index using 0.25 GB of memory is created, which uses about 2.6 GB of disk space:

# vdo create --name=vdo1 --device=/dev/nvme0n1p1 --verbose
Creating VDO vdo1
    grep MemAvailable /proc/meminfo
    pvcreate -qq --test /dev/nvme0n1p1
    modprobe kvdo
    vdoformat --uds-checkpoint-frequency=0 --uds-memory-size=0.25 /dev/nvme0n1p1
    vdodumpconfig /dev/nvme0n1p1
Starting VDO vdo1
    dmsetup status vdo1
    grep MemAvailable /proc/meminfo
    modprobe kvdo
    vdodumpconfig /dev/nvme0n1p1
    dmsetup create vdo1 --uuid VDO-bee917e8-cc33-4c59-8943-623eebac2b1b --table '0 200940480 vdo /dev/nvme0n1p1 4096 disabled 0 32768 16380 on auto vdo1 ack=1,bio=4,bioRotationInterval=64,cpu=2,hash=1,logical=1,physical=1'
    dmsetup status vdo1
Starting compression on VDO vdo1
    dmsetup message vdo1 0 compression on
    dmsetup status vdo1
VDO instance 11 volume is ready at /dev/mapper/vdo1
# vdo status | grep Index
    Index checkpoint frequency: 0
    Index memory setting: 0.25
    Index parallel factor: 0
    Index sparse: disabled
    Index status: online
# cat /sys/uds/index/12/name
dev=/dev/nvme0n1p1 offset=4096 size=2781704192
# hexdump -C -n 4096 /dev/nvme0n1p1
00000000  64 6d 76 64 6f 30 30 31  05 00 00 00 04 00 00 00  |dmvdo001........|
00000010  00 00 00 00 5d 00 00 00  00 00 00 00 09 01 02 00  |....]...........|
00000020  bb 7d 8a 70 4b 6a 05 00  be e9 17 e8 cc 33 4c 59  |.}.pKj.......3LY|
00000030  89 43 62 3e eb ac 2b 1b  00 00 00 00 01 00 00 00  |.Cb>..+.........|
00000040  00 00 00 00 01 00 00 00  d8 5c 0a 00 00 00 00 00  |.........\......|
00000050  00 ff ff ff 00 00 00 00  00 68 2e d4 e7 00 00 00  |.........h......|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000


If the user attempts to create a VDO volume with default options, except adding "--sparseIndex=enabled" to create a sparse index.  This should result in about 25 GB of disk space being used for the index, but it doesn't; instead it seems to show the same "size=2781704192" string in the /sys/uds "name", which suggests the on-disk index is about 2.6 GB in size. 

# vdo create --name=vdo1 --device=/dev/nvme0n1p1 --sparseIndex=enabled --verbose
Creating VDO vdo1
    grep MemAvailable /proc/meminfo
    pvcreate -qq --test /dev/nvme0n1p1
    modprobe kvdo
    vdoformat --uds-checkpoint-frequency=0 --uds-memory-size=0.25 --uds-sparse /dev/nvme0n1p1
    vdodumpconfig /dev/nvme0n1p1
Starting VDO vdo1
    dmsetup status vdo1
    grep MemAvailable /proc/meminfo
    modprobe kvdo
    vdodumpconfig /dev/nvme0n1p1
    dmsetup create vdo1 --uuid VDO-8b6dc126-f9dc-4898-9e68-cfe58ae8cf34 --table '0 200940480 vdo /dev/nvme0n1p1 4096 disabled 0 32768 16380 on auto vdo1 ack=1,bio=4,bioRotationInterval=64,cpu=2,hash=1,logical=1,physical=1'
    dmsetup status vdo1
Starting compression on VDO vdo1
    dmsetup message vdo1 0 compression on
    dmsetup status vdo1
VDO instance 12 volume is ready at /dev/mapper/vdo1
# vdo status | grep Index
    Index checkpoint frequency: 0
    Index memory setting: 0.25
    Index parallel factor: 0
    Index sparse: enabled
    Index status: online
# cat /sys/uds/index/13/name 
dev=/dev/nvme0n1p1 offset=4096 size=2781704192
# hexdump -C -n 4096 /dev/nvme0n1p1
00000000  64 6d 76 64 6f 30 30 31  05 00 00 00 04 00 00 00  |dmvdo001........|
00000010  00 00 00 00 5d 00 00 00  00 00 00 00 09 01 02 00  |....]...........|
00000020  84 ad 82 73 4b 6a 05 00  8b 6d c1 26 f9 dc 48 98  |...sKj...m.&..H.|
00000030  9e 68 cf e5 8a e8 cf 34  00 00 00 00 01 00 00 00  |.h.....4........|
00000040  00 00 00 00 01 00 00 00  d8 5c 0a 00 00 00 00 00  |.........\......|
00000050  00 ff ff ff 00 00 00 00  00 3d 47 39 ab 00 00 00  |.........=G9....|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000


Version-Release number of selected component (if applicable):
vdo-6.1.0.149-16.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create a VDO volume with the command "vdo create --name=vdo1 --device=<blockdevice> --sparseIndex=enabled" to try to create a VDO volume with a dense index.  Run "hexdump -C -n 4096 <blockdevice>" to dump the contents of the geometry block.  Run "vdo status", and save the output.  Remove this VDO volume.

2. Create a VDO volume with the command "vdo create --name=vdo1 --device=<blockdevice>" to try to create a VDO volume with a dense index.  Run "hexdump -C -n 4096 <blockdevice>" to dump the contents of the geometry block.  Run "vdo status", and save the output.  Remove this VDO volume.

3.  Compare the values in 0x48 to 0x4f of the geometry block; they should not be equal.

Actual results:
The values in 0x48 to 0x4f of the geometry block are equal.

Expected results:
The values in 0x48 to 0x4f of the geometry block are not equal (and the integer there is about 10 times larger).

Additional info:

Comment 3 Jakub Krysl 2018-07-03 13:29:38 UTC
Tested on:
RHEL-7.6-20180626.0
kernel-3.10.0-915.el7
kmod-vdo-6.1.1.99-1.el7
vdo-6.1.1.99-2.el7

# vdo create --name vdo --device /dev/sdb
Creating VDO vdo
Starting VDO vdo
Starting compression on VDO vdo
VDO instance 109 volume is ready at /dev/mapper/vdo
# vdo status --name vdo | grep 'Index sparse'
    Index sparse: disabled
# hexdump -C /dev/sdb -n 4K
00000000  64 6d 76 64 6f 30 30 31  05 00 00 00 04 00 00 00  |dmvdo001........|
00000010  00 00 00 00 5d 00 00 00  00 00 00 00 09 01 02 00  |....]...........|
00000020  c5 ac 79 38 18 70 05 00  a3 c1 39 51 1e 12 44 0d  |..y8.p....9Q..D.|
00000030  a7 4a 8c a4 93 44 bd 9a  00 00 00 00 01 00 00 00  |.J...D..........|
00000040  00 00 00 00 01 00 00 00  d8 5c 0a 00 00 00 00 00  |.........\......|
00000050  00 ff ff ff 00 00 00 00  00 dc 64 0f 61 00 00 00  |..........d.a...|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000

0x0a5cd8 * 4096 = 2781708288

# vdo create --name vdo --device /dev/sdb --sparseIndex enabled
Creating VDO vdo
Starting VDO vdo
Starting compression on VDO vdo
VDO instance 110 volume is ready at /dev/mapper/vdo
# vdo status --name vdo | grep 'Index sparse'
    Index sparse: enabled
# hexdump -C /dev/sdb -n 4K
00000000  64 6d 76 64 6f 30 30 31  05 00 00 00 04 00 00 00  |dmvdo001........|
00000010  00 00 00 00 5d 00 00 00  00 00 00 00 09 01 02 00  |....]...........|
00000020  19 0c ce 40 18 70 05 00  7d 53 cd c9 6f c6 42 6f  |...@.p..}S..o.Bo|
00000030  96 d5 62 34 d0 5d b3 47  00 00 00 00 01 00 00 00  |..b4.].G........|
00000040  00 00 00 00 01 00 00 00  2c d7 58 00 00 00 00 00  |........,.X.....|
00000050  00 ff ff ff 00 00 00 00  01 fb 19 c1 7f 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000

0x58d72c * 4096 = 23847944192

23847944192 / 2781708288 = 8.57
So the sparse index on disk is roughly 8.57x bigger than dense index.
Regression testing passed.

Comment 5 errata-xmlrpc 2018-10-30 09:39:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3094


Note You need to log in before you can comment on or make changes to this bug.