Bug 1730680

Summary: VDO should correctly distinguish between DM name and internal VDO name
Product: Red Hat Enterprise Linux 8 Reporter: Jakub Krysl <jkrysl>
Component: vdoAssignee: Ken Raeburn <raeburn>
Status: ASSIGNED --- QA Contact: Filip Suba <fsuba>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.1CC: awalsh, bjohnsto, corwin, jwise
Target Milestone: rc   
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Jakub Krysl 2019-07-17 10:39:19 UTC
Description of problem:
Using directly DM to create VDO device and supplying different DM and VDO name results in faulty behaviour in 'vdostats' command and probably somewhere else too.

# dmsetup create vdo_dm_device --table "0 1000 vdo V2 /dev/loop0 5242880 4096 32768 16380 off sync vdo_instance"
# vdo list
vdo_dm_device
# vdostats
Error sampling device /dev/mapper/vdo_dm_device: [Errno 2] No such file or directory: '/proc/vdo/vdo_dm_device/dedupe_stats'

The issue is "/proc/vdo/vdo_dm_device" dir does not exist, the proper path is "/proc/vdo/vdo_instance", e.g. using VDO internal name instead of DM name which is taken.

Version-Release number of selected component (if applicable):
kernel-4.18.0-107.el8.x86_64
kmod-kvdo-6.2.1.102-53.el8.x86_64
vdo-6.2.1.102-11.el8.x86_64

How reproducible:
100%

Steps to Reproduce:
1. dmsetup create vdo_dm_device --table "0 1000 vdo V2 /dev/loop0 5242880 4096 32768 16380 off sync vdo_instance"
2. vdostats

Actual results:
Error sampling device /dev/mapper/vdo_dm_device: [Errno 2] No such file or directory: '/proc/vdo/vdo_dm_device/dedupe_stats'

Expected results:
No issue found

Additional info:
This came from multi-line DM table testing, where giving different DM and internal VDO name seems to somehow work.

Comment 4 penguinpages 2019-09-30 10:56:18 UTC

I am having this issue on CentOS7.  I fat fingered a volume and made a 1TB drive 30TB usable vs 3TB.  I deleted volume and now cannot remake it with same parameters but size corrected.

[root@odin ~]# vdo create --name=odin_vdo_bay2 --device=/dev/sde --activate=enabled --compression=enabled --deduplication=enabled --vdoLogicalSize=3000G --writePolicy=async --verbose -force --verbose
Creating VDO odin_vdo_bay2
    grep MemAvailable /proc/meminfo
    pvcreate --config devices/scan_lvs=1 -qq --test /dev/sde
    blkid -p /dev/sde
    modprobe kvdo
vdo: ERROR - [Errno 2] No such file or directory: ''
[root@odin ~]#
[root@odin ~]#
[root@odin ~]# vdo create --name=odin_vdo_bay2 --device=/dev/sde --activate=enabled --compression=enabled --deduplication=enabled --vdoLogicalSize=3000G --writePolicy=async --verbose -force --verbose
Creating VDO odin_vdo_bay2
vdo: ERROR - VDO volume odin_vdo_bay2 previous operation (create) is incomplete; recover by performing 'remove --force'
[root@odin ~]# uname -a
Linux odin.penguinpage.local 3.10.0-1062.1.1.el7.x86_64 #1 SMP Fri Sep 13 22:55:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@odin ~]# rpm -qa |grep vdo
vdo-6.1.2.41-4.el7.x86_64
kmod-kvdo-6.1.2.41-5.el7.x86_64


Glad to collect more details as this is impacting the last node I need for creating my GlusterFS volume group.

Comment 5 Andy Walsh 2019-09-30 11:46:47 UTC
(In reply to penguinpages from comment #4)
> 
> I am having this issue on CentOS7.  I fat fingered a volume and made a 1TB
> drive 30TB usable vs 3TB.  I deleted volume and now cannot remake it with
> same parameters but size corrected.

I don't think this is the same issue as what this BZ is referring to.

> 
> [root@odin ~]# vdo create --name=odin_vdo_bay2 --device=/dev/sde
> --activate=enabled --compression=enabled --deduplication=enabled
> --vdoLogicalSize=3000G --writePolicy=async --verbose -force --verbose

I notice only one dash for the force option.

> Creating VDO odin_vdo_bay2
>     grep MemAvailable /proc/meminfo
>     pvcreate --config devices/scan_lvs=1 -qq --test /dev/sde
>     blkid -p /dev/sde
>     modprobe kvdo
> vdo: ERROR - [Errno 2] No such file or directory: ''

This is a strange error to me for this spot.  Not sure if it's failing to load the kernel module or if there's something weird going on due to the '-force' vs '--force'.

> [root@odin ~]#
> [root@odin ~]#
> [root@odin ~]# vdo create --name=odin_vdo_bay2 --device=/dev/sde
> --activate=enabled --compression=enabled --deduplication=enabled
> --vdoLogicalSize=3000G --writePolicy=async --verbose -force --verbose
> Creating VDO odin_vdo_bay2
> vdo: ERROR - VDO volume odin_vdo_bay2 previous operation (create) is
> incomplete; recover by performing 'remove --force'

Still only using one dash for the force option.

> [root@odin ~]# uname -a
> Linux odin.penguinpage.local 3.10.0-1062.1.1.el7.x86_64 #1 SMP Fri Sep 13
> 22:55:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> [root@odin ~]# rpm -qa |grep vdo
> vdo-6.1.2.41-4.el7.x86_64
> kmod-kvdo-6.1.2.41-5.el7.x86_64
> 
> 
> Glad to collect more details as this is impacting the last node I need for
> creating my GlusterFS volume group.

Can you try to do `sudo modprobe kvdo`?  That should ensure the kernel modules will load successfully.

Then, can you please retry the create with '--force' as opposed to '-force'?

Comment 6 penguinpages 2019-10-02 10:57:37 UTC
I did try the --force and same error.   I did a dd  if=/dev/zero of=/dev/sde  to wipe disk.. I assumed "wipefs" was the better / more clean way to clear out disk meta data, but seems that misses things.

That did not fix it.. 


Yes VDO module is present as another VDO volume is on that system and working fine in another cluster volume.


I ended up rebooting about some other issue with gluster ..... and the vdo create --force command worked. 

Not much of a root cause but I will see if I can get more details.

Here is the open community forum posting https://www.centos.org/forums/viewtopic.php?f=47&t=71891  for CentOS

Here is the VDO community on GitHub that was helpful 

https://github.com/dm-vdo/vdo/issues/22#issuecomment-536754654

I will RSVP to this thread if / as I find more.

Comment 7 Jakub Krysl 2019-10-15 14:38:54 UTC
Mass migration to Filip.