Bug 1269677
Summary: | cache policy change: NULL pointer dereference at dm_tm_new_block+0x15/0xb0 [dm_persistent_data] | |||
---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Corey Marthaler <cmarthal> | |
Component: | lvm2 | Assignee: | Zdenek Kabelac <zkabelac> | |
lvm2 sub component: | Cache Logical Volumes | QA Contact: | cluster-qe <cluster-qe> | |
Status: | CLOSED ERRATA | Docs Contact: | ||
Severity: | high | |||
Priority: | unspecified | CC: | agk, heinzm, jbrassow, lmiksik, msnitzer, prajnoha, zkabelac | |
Version: | 7.2 | |||
Target Milestone: | rc | |||
Target Release: | --- | |||
Hardware: | x86_64 | |||
OS: | Linux | |||
Whiteboard: | ||||
Fixed In Version: | lvm2-2.02.130-4.el7 | Doc Type: | Bug Fix | |
Doc Text: |
Cause:
Kernel dm-cache module target crashes with NULL pointer dereference when cache cleaner policy has been used with writeback mode.
Consequence:
Unchaching 'writeback' cached LV or using cleaner policy directly with writeback mode resulted into a kernel crash.
Fix:
Code now detects usage of cleaner policy and implicitly changes mode to writethrough.
Result:
System no longer kernel panics, and also it should not endlessly loop when uncaching cached LV.
|
Story Points: | --- | |
Clone Of: | ||||
: | 1269959 (view as bug list) | Environment: | ||
Last Closed: | 2015-11-19 12:48:06 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1269959, 1277134 |
Description
Corey Marthaler
2015-10-07 22:56:26 UTC
This is reproducible when doing the exact setup and changes. Yet when just randomly looping over cachepolicy changes, I'm unable to hit this. lvcreate -L 4G -n display_cache cache_sanity /dev/sdg1 lvcreate -L 4G -n pool cache_sanity /dev/sdf1 lvcreate -L 12M -n pool_meta cache_sanity /dev/sdf1 lvconvert --yes --type cache-pool --cachepolicy cleaner --cachemode writeback -c 64 --poolmetadata cache_sanity/pool_meta cache_sanity/pool lvconvert --yes --type cache --cachepool cache_sanity/pool cache_sanity/display_cache lvchange --cachepolicy smq cache_sanity/display_cache lvchange --cachepolicy mq cache_sanity/display_cache Lvm2 code now enforces 'writethrough' mode for cleaner policy: https://www.redhat.com/archives/lvm-devel/2015-October/msg00034.html This fix didn't make the 130-5 rpm. 3.10.0-322.el7.x86_64 lvm2-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 lvm2-libs-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 lvm2-cluster-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-1.02.107-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-libs-1.02.107-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-event-1.02.107-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-event-libs-1.02.107-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-persistent-data-0.5.5-1.el7 BUILT: Thu Aug 13 09:58:10 CDT 2015 cmirror-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 sanlock-3.2.4-1.el7 BUILT: Fri Jun 19 12:48:49 CDT 2015 sanlock-lib-3.2.4-1.el7 BUILT: Fri Jun 19 12:48:49 CDT 2015 lvm2-lockd-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 [root@host-082 ~]# vgcreate cache_sanity /dev/sd[abcdefgh]1 Volume group "cache_sanity" successfully created [root@host-082 ~]# lvcreate -L 4G -n display_cache cache_sanity /dev/sdg1 Logical volume "display_cache" created. [root@host-082 ~]# lvcreate -L 4G -n pool cache_sanity /dev/sdf1 Logical volume "pool" created. [root@host-082 ~]# lvcreate -L 12M -n pool_meta cache_sanity /dev/sdf1 Logical volume "pool_meta" created. [root@host-082 ~]# lvconvert --yes --type cache-pool --cachepolicy cleaner --cachemode writeback -c 64 --poolmetadata cache_sanity/pool_meta cache_sanity/pool WARNING: Converting logical volume cache_sanity/pool and cache_sanity/pool_meta to pool's data and metadata volumes. THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) Converted cache_sanity/pool to cache pool. [root@host-082 ~]# lvconvert --yes --type cache --cachepool cache_sanity/pool cache_sanity/display_cache Logical volume cache_sanity/display_cache is now cached. [root@host-082 ~]# lvs -a -o +devices,cache_policy,cache_mode LV VG Attr LSize Pool Origin Data% Meta% Cpy%Sync Devices Cache Policy Cachemode display_cache cache_sanity Cwi-a-C--- 4.00g [pool] [display_cache_corig] 0.00 4.39 100.00 display_cache_corig(0) cleaner writeback [display_cache_corig] cache_sanity owi-aoC--- 4.00g /dev/sdg1(0) [lvol0_pmspare] cache_sanity ewi------- 12.00m /dev/sda1(0) [pool] cache_sanity Cwi---C--- 4.00g 0.00 4.39 100.00 pool_cdata(0) cleaner writeback [pool_cdata] cache_sanity Cwi-ao---- 4.00g /dev/sdf1(0) [pool_cmeta] cache_sanity ewi-ao---- 12.00m /dev/sdf1(1024) [root@host-082 ~]# lvchange --cachepolicy smq cache_sanity/display_cache Logical volume "display_cache" changed. [root@host-082 ~]# lvchange --cachepolicy mq cache_sanity/display_cache [ 266.840237] device-mapper: cache cleaner: version 1.0.0 loaded Oct 15 14:43:58 host-082 kernel: device-mapper: cache cleaner: version 1.0.0 loaded [ 301.147662] BUG: unable to handle kernel NULL pointer dereference at (null) [ 301.148017] IP: [<ffffffffa034c135>] dm_tm_new_block+0x15/0xb0 [dm_persistent_data] [ 301.148017] PGD 0 [ 301.148017] Oops: 0000 [#1] SMP [...] Also needs kernel fix. For lvm2 side - 'dmsetup table' should be now showing 'writethrough' mode when 'cleaner' policy is in use. (Later we will provide more command warnings about enforced usage of 'writethrough' mode when cleaner policy is being set). This is fixed with the latest kernel. Even though "writeback" mode was specified on the cmdline, "writethrough" is shown through 'dmsetup table'. [root@host-113 ~]# lvconvert --yes --type cache-pool --cachepolicy cleaner --cachemode writeback -c 64 --poolmetadata cache_sanity/pool_meta cache_sanity/pool WARNING: Converting logical volume cache_sanity/pool and cache_sanity/pool_meta to pool's data and metadata volumes. THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) Converted cache_sanity/pool to cache pool. [root@host-113 ~]# lvconvert --yes --type cache --cachepool cache_sanity/pool cache_sanity/display_cache Logical volume cache_sanity/display_cache is now cached. [root@host-113 ~]# lvs -a -o +devices,cache_policy,cache_mode LV Attr LSize Pool Origin Data% Meta% Cpy%Sync Devices Cache Policy Cachemode display_cache Cwi-a-C--- 4.00g [pool] [display_cache_corig] 0.00 4.39 100.00 display_cache_corig(0) cleaner writeback [display_cache_corig] owi-aoC--- 4.00g /dev/sda1(0) [lvol0_pmspare] ewi------- 12.00m /dev/sda1(1024) [pool] Cwi---C--- 4.00g 0.00 4.39 100.00 pool_cdata(0) cleaner writeback [pool_cdata] Cwi-ao---- 4.00g /dev/sdb1(0) [pool_cmeta] ewi-ao---- 12.00m /dev/sdb1(1024) # cache mode is "writethrough" even though "writeback" was specified: [root@host-113 ~]# dmsetup table cache_sanity-display_cache: 0 8388608 cache 253:4 253:3 253:5 128 1 writethrough cleaner 0 cache_sanity-display_cache_corig: 0 8388608 linear 8:1 2048 cache_sanity-pool_cdata: 0 8388608 linear 8:17 2048 cache_sanity-pool_cmeta: 0 24576 linear 8:17 8390656 [root@host-113 ~]# lvchange --cachepolicy smq cache_sanity/display_cache Logical volume "display_cache" changed. [root@host-113 ~]# lvchange --cachepolicy mq cache_sanity/display_cache Logical volume "display_cache" changed. [root@host-113 ~]# dmsetup table cache_sanity-display_cache: 0 8388608 cache 253:4 253:3 253:5 128 1 writeback mq 0 cache_sanity-display_cache_corig: 0 8388608 linear 8:1 2048 cache_sanity-pool_cdata: 0 8388608 linear 8:17 2048 cache_sanity-pool_cmeta: 0 24576 linear 8:17 8390656 3.10.0-325.el7.x86_64 lvm2-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 lvm2-libs-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 lvm2-cluster-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-1.02.107-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-libs-1.02.107-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-event-1.02.107-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-event-libs-1.02.107-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 device-mapper-persistent-data-0.5.5-1.el7 BUILT: Thu Aug 13 09:58:10 CDT 2015 cmirror-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 sanlock-3.2.4-1.el7 BUILT: Fri Jun 19 12:48:49 CDT 2015 sanlock-lib-3.2.4-1.el7 BUILT: Fri Jun 19 12:48:49 CDT 2015 lvm2-lockd-2.02.130-5.el7 BUILT: Wed Oct 14 08:27:29 CDT 2015 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2147.html |