1080894 – dm-cache: crash on creating cache

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1080894 - dm-cache: crash on creating cache

Summary: dm-cache: crash on creating cache

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	kernel
Sub Component:
Version:	7.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	urgent
Target Milestone:	rc
Target Release:	---
Assignee:	Mike Snitzer
QA Contact:	XiaoNi
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1081934 (view as bug list)
Depends On:
Blocks:	1113511 1119326 1159001
TreeView+	depends on / blocked

Reported:	2014-03-26 09:46 UTC by Heinz Mauelshagen
Modified:	2018-12-09 17:40 UTC (History)
CC List:	7 users (show)
Fixed In Version:	kernel-3.10.0-210.el7
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2015-03-05 11:46:35 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Knowledge Base (Solution)	912953	0	None	None	None	Never
Red Hat Product Errata	RHSA-2015:0290	0	normal	SHIPPED_LIVE	Important: kernel security, bug fix, and enhancement update	2015-03-05 16:13:58 UTC

Description Heinz Mauelshagen 2014-03-26 09:46:50 UTC

Description of problem:
dm-cache setup crashs for certain configuration (see below).

Version-Release number of selected component (if applicable):
Kernel 3.10.0-113.el7.x86_6

How reproducible:
Always

Steps to Reproduce:
[root@storageqe-19 home]# vgcreate vgorig /dev/sdb /dev/sdc /dev/sdd /dev/sde
  Volume group "vgorig" successfully created
[root@storageqe-19 home]# vgcreate vgssd /dev/sd[f-h]
  Volume group "vgssd" successfully created
[root@storageqe-19 home]# vgs
  VG     #PV #LV #SN Attr   VSize VFree
  vgorig   4   0   0 wz--n- 3.64t 3.64t
  vgssd    3   0   0 wz--n- 2.73t 2.73t
[root@storageqe-19 home]# lvcreate -L 2.5T -n ssd vgssd
  Logical volume "ssd" created
[root@storageqe-19 home]# lvcreate -L 3.5T -n orig vgorig
  Logical volume "orig" created
[root@storageqe-19 home]# dmsetup create metadata --table "0 4194304 linear /dev/mapper/vgssd-ssd 0"
[root@storageqe-19 home]# dd if=/dev/zero of=/dev/mapper/metadata bs=4k count=1 conv=notrunc
1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 0.000914217 s, 4.5 MB/s
[root@storageqe-19 home]# dmsetup create ssd --table "0 4504174592 linear /dev/mapper/vgssd-ssd 4194304"
[root@storageqe-19 home]# dmsetup create cache  --table "0 6442450944 cache /dev/mapper/metadata /dev/mapper/ssd $ORIGIN 128 0 mq 0"



Actual results:
CRash

Expected results:
Should succeed

Additional info:
   [ 6971.475286] ------------[ cut here ]------------
[ 6971.479904] kernel BUG at drivers/md/persistent-data/dm-btree-spine.c:169!
[ 6971.486773] invalid opcode: 0000 [#1] SMP
[ 6971.490899] Modules linked in: dm_cache_mq dm_cache() dm_persistent_data dm_bio_prison dm_bufio sg dm_mod e1000e ptp iTCO_wdt iTCO_vendor_support coretemp ses pps_core i2c_i801 kvm_intel serio_raw kvm crc32c_intel pcspkr lpc_ich mfd_core enclosure ioatdma i7core_edac edac_core shpchp dca acpi_cpufreq mperf nfsd auth_rpcgss nfs_acl lockd sunrpc xfs libcrc32c sd_mod crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect ata_generic sysimgblt pata_acpi i2c_algo_bit drm_kms_helper ttm drm ata_piix libata i2c_core aacraid floppy
[ 6971.538602] CPU: 0 PID: 2600 Comm: dmsetup Tainted: G            -------------- T 3.10.0-113.el7.x86_64 #1
[ 6971.548247] Hardware name: Supermicro X8STi/X8STi, BIOS 2.0        09/17/10
[ 6971.555371] task: ffff8800d9705b00 ti: ffff880212480000 task.ti: ffff880212480000
[ 6971.562840] RIP: 0010:[<ffffffffa048976a>]  [<ffffffffa048976a>] ro_pop+0x2a/0x30 [dm_persistent_data]
[ 6971.572159] RSP: 0018:ffff880212481b20  EFLAGS: 00010246
[ 6971.577471] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000000
[ 6971.584594] RDX: 0000000000000000 RSI: 0000000000000246 RDI: ffff880212481b80
[ 6971.591719] RBP: ffff880212481b70 R08: 0000000000000000 R09: 0000000000000000
[ 6971.598842] R10: 0000000000000004 R11: ffff880170f5dff8 R12: 0000000000000004
[ 6971.605965] R13: ffffffffa04827a0 R14: ffff880212481bd0 R15: ffff880177b3c000
[ 6971.613092] FS:  00007f1537af0880(0000) GS:ffff88021fc00000(0000) knlGS:0000000000000000
[ 6971.621177] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 6971.626915] CR2: 00007f446fddb000 CR3: 00000001fb75e000 CR4: 00000000000007f0
[ 6971.634039] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6971.641162] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 6971.648285] Stack:
[ 6971.650296]  ffffffffa04875d2 00000000000001fd ffff880212481b80 ffff880212481ba0
[ 6971.657750]  00000000f7741310 0000000000000028 ffffffffa04827a0 ffff880212481bd0
[ 6971.665203]  ffff8800ca398400 ffff880213398800 ffff880212481bc0 ffffffffa048824e
[ 6971.672656] Call Trace:
[ 6971.675107]  [<ffffffffa04875d2>] ? walk_node+0xc2/0x100 [dm_persistent_data]
[ 6971.682236]  [<ffffffffa04827a0>] ? block_dec+0x160/0x160 [dm_persistent_data]
[ 6971.689457]  [<ffffffffa048824e>] dm_btree_walk+0x4e/0x80 [dm_persistent_data]
[ 6971.696674]  [<ffffffffa0494a80>] ? complete_migration+0x30/0x30 [dm_cache]
[ 6971.703635]  [<ffffffffa04825dc>] dm_array_walk+0x3c/0x60 [dm_persistent_data]
[ 6971.710853]  [<ffffffffa04977b0>] ? blocks_are_unmapped_or_clean+0xd0/0xd0 [dm_cache]
[ 6971.718678]  [<ffffffffa049853f>] dm_cache_load_mappings+0x7f/0xe0 [dm_cache]
[ 6971.725812]  [<ffffffffa0494a80>] ? complete_migration+0x30/0x30 [dm_cache]
[ 6971.732770]  [<ffffffffa0497019>] cache_preresume+0xf9/0x1a0 [dm_cache]
[ 6971.739387]  [<ffffffffa0416ff9>] dm_table_resume_targets+0x49/0xe0 [dm_mod]
[ 6971.746432]  [<ffffffffa041489c>] dm_resume+0x4c/0xd0 [dm_mod]
[ 6971.752264]  [<ffffffffa0419bbb>] dev_suspend+0x12b/0x250 [dm_mod]
[ 6971.758444]  [<ffffffffa0419a90>] ? table_load+0x380/0x380 [dm_mod]
[ 6971.764708]  [<ffffffffa041a4d5>] ctl_ioctl+0x255/0x500 [dm_mod]
[ 6971.770716]  [<ffffffffa041a793>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[ 6971.776807]  [<ffffffff811c2dd5>] do_vfs_ioctl+0x2e5/0x4c0
[ 6971.782291]  [<ffffffff8125798e>] ? file_has_perm+0xae/0xc0
[ 6971.787865]  [<ffffffff811c3051>] SyS_ioctl+0xa1/0xc0
[ 6971.792918]  [<ffffffff815f4925>] ? do_device_not_available+0x35/0x60
[ 6971.799358]  [<ffffffff815fc719>] system_call_fastpath+0x16/0x1b
[ 6971.805359] Code: 90 66 66 66 66 90 8b 47 08 85 c0 74 1e 83 e8 01 55 89 47 08 48 98 48 8b 74 c7 10 48 8b 07 48 89 e5 48 8b 38 e8 78 d4 ff ff 5d c3 <0f> 0b 0f 1f 40 00 66 66 66 66 90 8b 47 08 85 c0 74 15 83 e8 01
[ 6971.825312] RIP  [<ffffffffa048976a>] ro_pop+0x2a/0x30 [dm_persistent_data]
[ 6971.832289]  RSP <ffff880212481b20>

Comment 1 Heinz Mauelshagen 2014-03-28 13:26:06 UTC

*** Bug 1081934 has been marked as a duplicate of this bug. ***

Comment 2 Petr Rockai 2014-10-22 09:05:44 UTC

Still happens with 3.10.0-123.el7.x86_64:

[   14.020028] ------------[ cut here ]------------
[   14.021004] kernel BUG at drivers/md/persistent-data/dm-btree-spine.c:169!
[   14.021004] invalid opcode: 0000 [#1] SMP 
[   14.021004] Modules linked in: dm_cache_mq dm_cache() nls_utf8 dm_thin_pool dm_bio_prison dm_persistent_data libcrc32c dm_raid raid10 raid1 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx dm_zero dm_mirror dm_region_hash dm_log dm_snapshot dm_bufio dm_mod loop sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_piix libata des_generic md4 virtio_net cifs dns_resolver ext4 jbd2 mbcache virtio_balloon virtio_blk virtio_pci virtio_ring virtio
[   14.021004] CPU: 0 PID: 508 Comm: dmsetup Tainted: G            -------------- T 3.10.0-123.el7.x86_64 #1
[   14.021004] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
[   14.021004] task: ffff8801ad8bcfa0 ti: ffff8801ab83e000 task.ti: ffff8801ab83e000
[   14.021004] RIP: 0010:[<ffffffffa029273a>]  [<ffffffffa029273a>] ro_pop+0x2a/0x30 [dm_persistent_data]
[   14.021004] RSP: 0018:ffff8801ab83fb20  EFLAGS: 00010246
[   14.021004] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000000
[   14.021004] RDX: 0000000000000000 RSI: 0000000000000246 RDI: ffff8801ab83fb80
[   14.021004] RBP: ffff8801ab83fb70 R08: 0000000000000000 R09: 0000000000000000
[   14.021004] R10: 0000000000000004 R11: ffff88012c5e2ff8 R12: 0000000000000004
[   14.021004] R13: ffffffffa028b7a0 R14: ffff8801ab83fbd0 R15: ffff88003704a000
[   14.021004] FS:  00007f88415e8800(0000) GS:ffff8801b6c00000(0000) knlGS:0000000000000000
[   14.021004] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[   14.021004] CR2: 00007fd9471812e0 CR3: 00000001ab28a000 CR4: 00000000000006f0
[   14.021004] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   14.021004] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[   14.021004] Stack:
[   14.021004]  ffffffffa02905a2 00000000000001fd ffff8801ab83fb80 ffff8801ab83fba0
[   14.021004]  0000000073895023 0000000000000028 ffffffffa028b7a0 ffff8801ab83fbd0
[   14.021004]  ffff88007ff78800 ffff8801aafc4000 ffff8801ab83fbc0 ffffffffa029121e
[   14.021004] Call Trace:
[   14.021004]  [<ffffffffa02905a2>] ? walk_node+0xc2/0x100 [dm_persistent_data]
[   14.021004]  [<ffffffffa028b7a0>] ? block_dec+0x160/0x160 [dm_persistent_data]
[   14.021004]  [<ffffffffa029121e>] dm_btree_walk+0x4e/0x80 [dm_persistent_data]
[   14.021004]  [<ffffffffa02c1a30>] ? complete_migration+0x30/0x30 [dm_cache]
[   14.021004]  [<ffffffffa028b5dc>] dm_array_walk+0x3c/0x60 [dm_persistent_data]
[   14.021004]  [<ffffffffa02c4700>] ? blocks_are_unmapped_or_clean+0xd0/0xd0 [dm_cache]
[   14.021004]  [<ffffffffa02c551f>] dm_cache_load_mappings+0x7f/0xe0 [dm_cache]
[   14.021004]  [<ffffffffa02c1a30>] ? complete_migration+0x30/0x30 [dm_cache]
[   14.021004]  [<ffffffff810f0001>] ? kdb_register+0x1/0x20
[   14.021004]  [<ffffffffa02c3ef9>] cache_preresume+0xf9/0x1a0 [dm_cache]
[   14.021004]  [<ffffffffa01d2ff9>] dm_table_resume_targets+0x49/0xe0 [dm_mod]
[   14.021004]  [<ffffffffa01d089c>] dm_resume+0x4c/0xd0 [dm_mod]
[   14.021004]  [<ffffffffa01d5bcb>] dev_suspend+0x12b/0x250 [dm_mod]
[   14.021004]  [<ffffffffa01d5aa0>] ? table_load+0x380/0x380 [dm_mod]
[   14.021004]  [<ffffffffa01d64e5>] ctl_ioctl+0x255/0x500 [dm_mod]
[   14.021004]  [<ffffffffa01d67a3>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[   14.021004]  [<ffffffff811c2f25>] do_vfs_ioctl+0x2e5/0x4c0
[   14.021004]  [<ffffffff81257a2e>] ? file_has_perm+0xae/0xc0
[   14.021004]  [<ffffffff811c31a1>] SyS_ioctl+0xa1/0xc0
[   14.021004]  [<ffffffff815ea325>] ? do_device_not_available+0x35/0x60
[   14.021004]  [<ffffffff815f2119>] system_call_fastpath+0x16/0x1b
[   14.021004] Code: 90 0f 1f 44 00 00 8b 47 08 85 c0 74 1e 83 e8 01 55 89 47 08 48 98 48 8b 74 c7 10 48 8b 07 48 89 e5 48 8b 38 e8 38 d4 ff ff 5d c3 <0f> 0b 0f 1f 40 00 0f 1f 44 00 00 8b 47 08 85 c0 74 15 83 e8 01 
[   14.021004] RIP  [<ffffffffa029273a>] ro_pop+0x2a/0x30 [dm_persistent_data]
[   14.021004]  RSP <ffff8801ab83fb20>
[   14.074847] ---[ end trace 2f328e1677444d10 ]---
[   14.075464] Kernel panic - not syncing: Fatal exception

I am going to try with a more recent kernel build (but it's going to take a while). Also, with 4G of RAM, the kernel runs out of memory creating the 2.5T cache device. It gives the above panic with 6G of RAM.

Comment 3 Petr Rockai 2014-10-22 11:22:14 UTC

Also happens with latest RHEL 7.1 compose (20141021):

[   23.990481] ------------[ cut here ]------------
[   23.991004] kernel BUG at drivers/md/persistent-data/dm-btree-spine.c:169!
[   23.991004] invalid opcode: 0000 [#1] SMP 
[   23.991004] Modules linked in: dm_cache_mq dm_cache() nls_utf8 dm_thin_pool dm_bio_prison dm_persistent_data libcrc32c dm_raid raid10 raid1 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx dm_zero dm_mirror dm_region_hash dm_log dm_snapshot dm_bufio dm_mod loop sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_piix libata des_generic md4 virtio_net cifs dns_resolver ext4 jbd2 mbcache virtio_balloon virtio_blk virtio_pci virtio_ring virtio
[   23.991004] CPU: 0 PID: 541 Comm: dmsetup Tainted: G            -------------- T 3.10.0-189.el7.x86_64 #1
[   23.991004] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
[   23.991004] task: ffff8800ba9116c0 ti: ffff8800ba924000 task.ti: ffff8800ba924000
[   23.991004] RIP: 0010:[<ffffffffa029a6fa>]  [<ffffffffa029a6fa>] ro_pop+0x2a/0x30 [dm_persistent_data]
[   23.991004] RSP: 0018:ffff8800ba927b20  EFLAGS: 00010246
[   23.991004] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000000
[   23.991004] RDX: 0000000000000000 RSI: 0000000000000246 RDI: ffff8800ba927b80
[   23.991004] RBP: ffff8800ba927b70 R08: 0000000000000000 R09: 0000000000000000
[   23.991004] R10: 0000000000000004 R11: ffff880034f1eff8 R12: 0000000000000004
[   23.991004] R13: ffffffffa02937a0 R14: ffff8800ba927bd0 R15: ffff88012ebc3000
[   23.991004] FS:  00007f22b5681800(0000) GS:ffff8801b6c00000(0000) knlGS:0000000000000000
[   23.991004] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[   23.991004] CR2: 00007f8ccdb76d3c CR3: 00000000ba8c4000 CR4: 00000000000006f0
[   23.991004] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   23.991004] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[   23.991004] Stack:
[   23.991004]  ffffffffa0298562 00000000000001fd ffff8800ba927b80 ffff8800ba927ba0
[   23.991004]  00000000d698142d 0000000000000028 ffffffffa02937a0 ffff8800ba927bd0
[   23.991004]  ffff8800ba920400 ffff8801adb7f800 ffff8800ba927bc0 ffffffffa02991de
[   23.991004] Call Trace:
[   23.991004]  [<ffffffffa0298562>] ? walk_node+0xc2/0x100 [dm_persistent_data]
[   23.991004]  [<ffffffffa02937a0>] ? block_dec+0x160/0x160 [dm_persistent_data]
[   23.991004]  [<ffffffffa02991de>] dm_btree_walk+0x4e/0x80 [dm_persistent_data]
[   23.991004]  [<ffffffffa02c99f0>] ? complete_migration+0x30/0x30 [dm_cache]
[   23.991004]  [<ffffffffa02935dc>] dm_array_walk+0x3c/0x60 [dm_persistent_data]
[   23.991004]  [<ffffffffa02cc640>] ? blocks_are_unmapped_or_clean+0xd0/0xd0 [dm_cache]
[   23.991004]  [<ffffffffa02cd50f>] dm_cache_load_mappings+0x7f/0xe0 [dm_cache]
[   23.991004]  [<ffffffffa02c99f0>] ? complete_migration+0x30/0x30 [dm_cache]
[   23.991004]  [<ffffffff81110001>] ? irq_create_mapping+0x211/0x240
[   23.991004]  [<ffffffffa02cbe69>] cache_preresume+0xf9/0x1a0 [dm_cache]
[   23.991004]  [<ffffffffa01d84a9>] dm_table_resume_targets+0x49/0xe0 [dm_mod]
[   23.991004]  [<ffffffffa01d592c>] dm_resume+0x4c/0xd0 [dm_mod]
[   23.991004]  [<ffffffffa01daccb>] dev_suspend+0x12b/0x250 [dm_mod]
[   23.991004]  [<ffffffffa01daba0>] ? table_load+0x380/0x380 [dm_mod]
[   23.991004]  [<ffffffffa01db5e5>] ctl_ioctl+0x255/0x500 [dm_mod]
[   23.991004]  [<ffffffffa01db8a3>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[   23.991004]  [<ffffffff811d9205>] do_vfs_ioctl+0x2e5/0x4c0
[   23.991004]  [<ffffffff8126e0fe>] ? file_has_perm+0xae/0xc0
[   23.991004]  [<ffffffff811d9481>] SyS_ioctl+0xa1/0xc0
[   23.991004]  [<ffffffff8160baa5>] ? do_device_not_available+0x35/0x60
[   23.991004]  [<ffffffff816134e9>] system_call_fastpath+0x16/0x1b
[   23.991004] Code: 90 0f 1f 44 00 00 8b 47 08 85 c0 74 1e 83 e8 01 55 89 47 08 48 98 48 8b 74 c7 10 48 8b 07 48 89 e5 48 8b 38 e8 38 d4 ff ff 5d c3 <0f> 0b 0f 1f 40 00 0f 1f 44 00 00 8b 47 08 85 c0 74 15 83 e8 01 
[   23.991004] RIP  [<ffffffffa029a6fa>] ro_pop+0x2a/0x30 [dm_persistent_data]
[   23.991004]  RSP <ffff8800ba927b20>
[   24.030925] ---[ end trace 5b03bba261d2c923 ]---
[   24.031260] Kernel panic - not syncing: Fatal exception

The kernel is this build: [    0.000000] Linux version 3.10.0-189.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-7) (GCC) ) #1 SMP Fri Oct 17 11:04:19 EDT 2014

Comment 4 Petr Rockai 2014-10-28 12:27:27 UTC

This appears to be directly linked to the number of blocks in the “ssd” device. Increasing block size or shrinking the ssd device both make the tripped BUG_ON go away. The threshold appears to be somewhere between 2^22 and 2^23 blocks. I'm reading the source code trying to narrow the problem down.

Comment 5 Petr Rockai 2014-10-29 14:50:04 UTC

I have cut down the threshold to be between 2^23 - 2^13 and 2^23 - 2^14 cache-device blocks. That is definitely an odd number. Just FYI. I need move to other things, but ping me if you need any other details I could provide.

Comment 6 Joe Thornber 2014-11-04 11:00:58 UTC

Reproduced with the following dmtest test:

https://github.com/jthornber/device-mapper-test-suite/blob/master/lib/dmtest/tests/cache/large_cache_tests.rb#L139

The real problem here is commit 64ab346a360a4b15c28fb8531918d4a01f4eabd9 made at the end of March.

Keeping track of which blocks in on the origin have been discarded allows us to optimise migration to/from the cache by avoiding a copy (no point copying discarded data).  Originally the discard block size was a large multiple of the cache block size, because the discard bitset size depends on the size of the _origin_ rather than the fast ssd device.  The offending patch makes these two block sizes the same; when the origin is large and the cache block size is small this causes an outrageous amount of metadata and memory to be used to store the discard bitset.  We just can't go live with this patch.  For the record the testing that NA did with v. large setups was done before this patch went in.

So I'm backing this patch out and investigating the issue that caused it to go in in the first place.

Comment 7 Joe Thornber 2014-11-10 15:47:59 UTC

2 patches are making their way upstream.

https://github.com/jthornber/linux-2.6/commit/e28c5a7e0c8208b7f3080744c9d90ce97b359f09

https://github.com/jthornber/linux-2.6/commit/c84dd912f9fef71ce8f1d3d801ee7cacf9412635

Which fix this test case:

https://github.com/jthornber/device-mapper-test-suite/blob/master/lib/dmtest/tests/cache/large_cache_tests.rb#L139

Handing over to Mike Snitzer to roll a RHEL kernel for testing.

Comment 8 Jarod Wilson 2014-11-25 13:16:14 UTC

Patch(es) available on kernel-3.10.0-210.el7

Comment 11 XiaoNi 2014-12-17 07:49:17 UTC

Hi all

   The problem is fixed in kernel-3.10.0-210.el7. Set Verified.

Thanks
Xiao

Comment 13 errata-xmlrpc 2015-03-05 11:46:35 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0290.html

Note You need to log in before you can comment on or make changes to this bug.