Bug 1832882

Summary: [Tracker for bug 1832838] Thinpool with LVM cache throws traceback in /var/log/messages
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: SATHEESARAN <sasundar>
Component: rhhiAssignee: Gobinda Das <godas>
Status: CLOSED CURRENTRELEASE QA Contact: SATHEESARAN <sasundar>
Severity: medium Docs Contact:
Priority: medium    
Version: rhhiv-1.8CC: blivet-maint-list, release-test-team-automation, rhs-bugs
Target Milestone: ---Keywords: ZStream
Target Release: RHHI-V 1.8.z Batch Update 2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: 1832838 Environment:
Last Closed: 2020-12-01 06:58:27 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1832838, 1836189    
Bug Blocks:    

Description SATHEESARAN 2020-05-07 12:36:56 UTC 
+++ This bug was initially created as a clone of Bug #1832838 +++

Description of problem:
-----------------------
This issue is seen with RHHI-V deployment, in which RHVH node is used. When the node has a thinpool device and LVM cache attached to the thinpool

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
python3-blivet-3.1.0-20.el8.noarch
blivet-data-3.1.0-20.el8.noarch
kernel-4.18.0-193.el8.x86_64

How reproducible:
------------------
Always

Steps to Reproduce:
--------------------
1. Create a LVM thinpool with a disk
2. Attach the LVM cache to this thinpool
3. Use blivet to enquire about the devices

Actual results:
---------------
Tracebacks seen in /var/log/messages 

Expected results:
-----------------
No tracebacks should be seen

--- Additional comment from SATHEESARAN on 2020-05-07 11:07:16 UTC ---



--- Additional comment from SATHEESARAN on 2020-05-07 11:22:26 UTC ---

In my setup I have the following disks:


[root@rhsqa-grafton7-nic2 ansible]# dmsetup table
gluster_vg_sdd-cachelv_gluster_thinpool_gluster_vg_sdd_cpool_cdata: 0 461373440 linear 8:96 2048
gluster_vg_sdc-gluster_lv_vmstore: 0 85899345920 thin 253:20 2
3600304801a48610125f9a6311c1a0f61: 0 3905945600 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 2 8:64 1 1 
gluster_vg_sdd-gluster_lv_test: 0 1048576000 thin 253:23 1
gluster_vg_sdd-gluster_thinpool_gluster_vg_sdd-tpool: 0 3893355520 thin-pool 253:21 253:22 5120 0 1 skip_block_zeroing 
gluster_vg_sdd-gluster_thinpool_gluster_vg_sdd_tdata: 0 3893355520 cache 253:30 253:29 253:31 512 2 metadata2 writethrough smq 2 migration_threshold 4096
gluster_vg_sdd-cachelv_gluster_thinpool_gluster_vg_sdd_cpool_cmeta: 0 87040 linear 8:96 461375488
rhvh-var: 0 31457280 thin 253:4 4
gluster_vg_sdd-gluster_thinpool_gluster_vg_sdd_tmeta: 0 6292480 linear 8:48 3899653120
rhvh-rhvh--4.4.0.18--0.20200417.0+1: 0 1652924416 thin 253:4 9
gluster_vg_sdd-gluster_thinpool_gluster_vg_sdd_tdata_corig: 0 3893355520 linear 8:48 6297600
luks-32650b1a-0ed1-4f84-8849-b0c02ea71614: 0 1950316544 crypt aes-xts-plain64 :64:logon:cryptsetup:32650b1a-0ed1-4f84-8849-b0c02ea71614-d0 0 8:2 32768 1 allow_discards
gluster_vg_sdd-gluster_thinpool_gluster_vg_sdd: 0 3893355520 linear 253:23 0
gluster_vg_sdc-gluster_thinpool_gluster_vg_sdc-tpool: 0 343531054080 thin-pool 253:18 253:19 5120 0 1 skip_block_zeroing 
gluster_vg_sdc-gluster_thinpool_gluster_vg_sdc_tdata: 0 343531054080 linear 253:16 33167360
3600304801a48610125f9a69121d4f04d: 0 467795968 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 2 8:112 1 1 
gluster_vg_sdc-gluster_thinpool_gluster_vg_sdc_tmeta: 0 33161216 linear 253:16 343564221440
rhvh-var_log_audit: 0 4194304 thin 253:4 1
gluster_vg_sdc-gluster_lv_data: 0 257698037760 thin 253:20 1
gluster_vg_sdc-gluster_thinpool_gluster_vg_sdc: 0 343531054080 linear 253:20 0
rhvh-pool00: 0 1730519040 linear 253:4 0
vdo_sdc: 0 343597383680 vdo V2 /dev/disk/by-id/scsi-3600304801a48610125f9a5d816c93dc7 4882432000 4096 32768 16380 on auto vdo_sdc maxDiscard 4096 ack 1 bio 4 bioRotationInterval 64 cpu 2 hash 1 logical 1 physical 1
3600304801a48610125f9a6571e612c29: 0 7812939776 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 2 8:80 1 1 
rhvh-home: 0 2097152 thin 253:4 6
rhvh-pool00-tpool: 0 1730519040 thin-pool 253:2 253:3 128 4055904 0 
rhvh-pool00_tdata: 0 1730519040 linear 253:0 862208
gluster_vg_sdb-gluster_lv_engine: 0 209715200 linear 8:16 5120
rhvh-var_log: 0 16777216 thin 253:4 2
rhvh-pool00_tmeta: 0 860160 linear 253:0 1731381248
rhvh-pool00_tmeta: 860160 1236992 linear 253:0 1740630016
rhvh-tmp: 0 2097152 thin 253:4 5
rhvh-var_crash: 0 20971520 thin 253:4 3
rhvh-swap: 0 8388608 linear 253:0 1732241408

-----------

[root@rhsqa-grafton7-nic2 ansible]# python3
Python 3.6.8 (default, Dec  5 2019, 15:45:45) 
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import blivet
>>> blivetEnv=blivet.Blivet()
>>> blivetEnv.reset()
>>> print(blivetEnv.devices)
Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/gi/overrides/BlockDev.py", line 967, in wrapped
    ret = orig_obj(*args, **kwargs)
GLib.Error: g-bd-lvm-error-quark: Failed to get status of the cache map 'gluster_vg_sdd-gluster_thinpool_gluster_vg_sdd':  (4)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 169, in __repr__
    "extents": self.extents, "free_space": self.free_space,
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 479, in free_space
    used = sum((lv.vg_space_used for lv in self.lvs), Size(0))
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 479, in <genexpr>
    used = sum((lv.vg_space_used for lv in self.lvs), Size(0))
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 1850, in decorated
    return meth(self, *args, **kwargs)  # pylint: disable=not-callable
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 1933, in vg_space_used
    return super(LVMLogicalVolumeDevice, self).vg_space_used
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 778, in vg_space_used
    return self.data_vg_space_used + self.metadata_vg_space_used
  File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 784, in data_vg_space_used
    cache_size = self.cache.size
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 2242, in size
    stats = self.stats
  File "/usr/lib/python3.6/site-packages/blivet/devices/lvm.py", line 2267, in stats
    return LVMCacheStats(blockdev.lvm.cache_stats(self._cached_lv.vg.name, self._cached_lv.lvname))
  File "/usr/lib64/python3.6/site-packages/gi/overrides/BlockDev.py", line 993, in wrapped
    raise transform[1](msg)
gi.overrides.BlockDev.LVMError: Failed to get status of the cache map 'gluster_vg_sdd-gluster_thinpool_gluster_vg_sdd': 
>>> exit()

--- Additional comment from SATHEESARAN on 2020-05-07 11:39:41 UTC ---



--- Additional comment from SATHEESARAN on 2020-05-07 12:18:58 UTC ---
Comment 4 SATHEESARAN 2020-11-08 01:45:24 UTC 
Tested with RHVH 4.4.3 ( RHVH-4.4-20201105.3-RHVH-x86_64-dvd1.iso )

There are no such error or traceback messages in /var/log/messages, with thinlv and LVM cache devices.