| Summary: | "fstrim/discard" operation is not supported for lvm RAID1 in RHEL7.3 | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Ranjith ML <rmadhuso> | |
| Component: | lvm2 | Assignee: | Heinz Mauelshagen <heinzm> | |
| lvm2 sub component: | Thin Provisioning | QA Contact: | cluster-qe <cluster-qe> | |
| Status: | CLOSED ERRATA | Docs Contact: | Steven J. Levine <slevine> | |
| Severity: | unspecified | |||
| Priority: | unspecified | CC: | agk, cmarthal, coughlan, heinzm, jbrassow, msnitzer, prajnoha, prockai, rbednar, thornber, zkabelac | |
| Version: | 7.3 | Keywords: | Regression | |
| Target Milestone: | rc | |||
| Target Release: | --- | |||
| Hardware: | All | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | lvm2-2.02.169-1.el7 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1474462 (view as bug list) | Environment: | ||
| Last Closed: | 2017-08-01 21:49:49 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Bug Depends On: | ||||
| Bug Blocks: | 1385242 | |||
|
Description
Ranjith ML
2016-12-02 07:49:14 UTC
Works on kernel 3.10.0-587.el7.x86_64: # lvs -aoname,size,segtype,devices LV LSize Type Devices root 45.12g linear /dev/vda2(0) swap 3.88g linear /dev/vda2(11550) r 200.00t raid1 r_rimage_0(0),r_rimage_1(0) [r_rimage_0] 200.00t linear /dev/sdb(1) [r_rimage_1] 200.00t linear /dev/sdc(1) [r_rmeta_0] 4.00m linear /dev/sdb(0) [r_rmeta_1] 4.00m linear /dev/sdc(0) # time mkfs -t ext4 -E nodiscard /dev/t/r mke2fs 1.42.9 (28-Dec-2013) Filesystem label= OS type: Linux ... real 1m36.392s user 1m29.085s sys 0m3.799s # mount /dev/t/r /mnt/r [root@rhel-7-4 ~]# df -h /mnt/r Filesystem Size Used Avail Use% Mounted on /dev/mapper/t-r 200T 20K 190T 1% /mnt/r # time fstrim -v /mnt/r /mnt/r: 199.2 TiB (219026630234112 bytes) trimmed real 1m37.915s user 0m0.001s sys 0m39.112s Does not work with 3.10.0-663.el7.x86_64 Tried fstrim with linear, raid1, thin_lv and thin_lv on raid1 thinpool, on both ext4 and xfs. # lvs -aoname,pool_lv,segtype,devices LV Pool Type Devices root linear /dev/vda2(205) swap linear /dev/vda2(0) linear linear /dev/sda(257) [lvol0_pmspare] linear /dev/sda(513) newraid1 raid1 newraid1_rimage_0(0),newraid1_rimage_1(0) [newraid1_rimage_0] linear /dev/sda(1029) [newraid1_rimage_1] linear /dev/sdb(515) [newraid1_rmeta_0] linear /dev/sda(1028) [newraid1_rmeta_1] linear /dev/sdb(514) raid1 raid1 raid1_rimage_0(0),raid1_rimage_1(0) [raid1_rimage_0] linear /dev/sda(772) [raid1_rimage_1] linear /dev/sdb(258) [raid1_rmeta_0] linear /dev/sda(771) [raid1_rmeta_1] linear /dev/sdb(257) thinlv thinp thin thinp thin-pool thinp_tdata(0) thinp_on_raid1 thin-pool thinp_on_raid1_tdata(0) [thinp_on_raid1_tdata] raid1 thinp_on_raid1_tdata_rimage_0(0),thinp_on_raid1_tdata_rimage_1(0) [thinp_on_raid1_tdata_rimage_0] linear /dev/sda(1) [thinp_on_raid1_tdata_rimage_1] linear /dev/sdb(1) [thinp_on_raid1_tdata_rmeta_0] linear /dev/sda(0) [thinp_on_raid1_tdata_rmeta_1] linear /dev/sdb(0) [thinp_on_raid1_tmeta] linear /dev/sda(770) [thinp_tdata] linear /dev/sda(514) [thinp_tmeta] linear /dev/sdj(0) virt_raid1 thinp_on_raid1 thin # mount ... /dev/mapper/vg-raid1 on /mnt/raid1 type xfs (rw,relatime,seclabel,attr2,inode64,noquota) /dev/mapper/vg-thinlv on /mnt/thinlv type xfs (rw,relatime,seclabel,attr2,inode64,noquota) /dev/mapper/vg-linear on /mnt/linear type xfs (rw,relatime,seclabel,attr2,inode64,noquota) /dev/mapper/vg-virt_raid1 on /mnt/virt_raid1 type xfs (rw,relatime,seclabel,attr2,inode64,logbsize=64k,sunit=128,swidth=128,noquota) ... # fstrim -v /mnt/linear fstrim: /mnt/linear: the discard operation is not supported # fstrim -v /mnt/raid1 fstrim: /mnt/raid1: the discard operation is not supported # fstrim -v /mnt/thinlv /mnt/thinlv: 5 MiB (5267456 bytes) trimmed # fstrim -v /mnt/virt_raid1 /mnt/virt_raid1: 96.5 MiB (101220352 bytes) trimmed lvm2-2.02.171-1.el7 BUILT: Wed May 3 14:05:13 CEST 2017 lvm2-libs-2.02.171-1.el7 BUILT: Wed May 3 14:05:13 CEST 2017 lvm2-cluster-2.02.171-1.el7 BUILT: Wed May 3 14:05:13 CEST 2017 device-mapper-1.02.140-1.el7 BUILT: Wed May 3 14:05:13 CEST 2017 device-mapper-libs-1.02.140-1.el7 BUILT: Wed May 3 14:05:13 CEST 2017 device-mapper-event-1.02.140-1.el7 BUILT: Wed May 3 14:05:13 CEST 2017 device-mapper-event-libs-1.02.140-1.el7 BUILT: Wed May 3 14:05:13 CEST 2017 device-mapper-persistent-data-0.7.0-0.1.rc6.el7 BUILT: Mon Mar 27 17:15:46 CEST 2017 cmirror-2.02.171-1.el7 BUILT: Wed May 3 14:05:13 CEST 2017 In reply to comment#7, and in speaking with Roman on irc, it looks like the underlying storage does _not_ support discard (at least not all devices). DM thinp will disable discard passdown if the underlying storage doesn't support discards but thinp itself will handle the discards at the logical layer. Please re-test lvm RAID1 only against storage that supports discards. If you don't have physical storage that supports discards you can use scsi_debug to create ramdisk-based test devices that support discard, e.g.: modprobe scsi_debug lbpws=1 dev_size_mb=100 Thanks Mike, it was exactly the case. I did the previous testing with VMs on host that actually uses SSDs as backing devices, but then on the VMs scsi targets don't appear to have discard support as shown below. Marking verified, fstrim/discard on raid1 volumes is supported with latest rpms. ============================================================== # modprobe scsi_debug lbpws=1 dev_size_mb=100 # lsscsi ... [62:0:0:0] disk LIO-ORG cluster62595-di 4.0 /dev/sdj [70:0:0:0] disk LIO-ORG cluster62595-di 4.0 /dev/sdi ... [72:0:0:0] disk Linux scsi_debug 0004 /dev/sdk ... ## Partition /dev/sdk # lsblk -D NAME DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO ... sdi 0 0B 0B 0 └─sdi1 0 0B 0B 0 sdj 0 0B 0B 0 └─sdj1 0 0B 0B 0 sdk 0 512B 32M 1 ├─sdk1 0 512B 32M 1 ├─sdk2 0 512B 32M 1 └─sdk3 0 512B 32M 1 vda 0 0B 0B 0 ├─vda1 0 0B 0B 0 ... # lvs -a -o lv_name,devices LV Devices root /dev/vda2(205) swap /dev/vda2(0) raid1 raid1_rimage_0(0),raid1_rimage_1(0) [raid1_rimage_0] /dev/sdk1(1) [raid1_rimage_1] /dev/sdk2(1) [raid1_rmeta_0] /dev/sdk1(0) [raid1_rmeta_1] /dev/sdk2(0) # mount | grep /mnt/raid1 /dev/mapper/vg-raid1 on /mnt/raid1 type ext4 (rw,relatime,seclabel,data=ordered) # fstrim -v /mnt/raid1 /mnt/raid1: 18 MiB (18914304 bytes) trimmed ============================================================== 3.10.0-675.el7.x86_64 lvm2-2.02.171-3.el7 BUILT: Wed May 31 15:36:29 CEST 2017 lvm2-libs-2.02.171-3.el7 BUILT: Wed May 31 15:36:29 CEST 2017 lvm2-cluster-2.02.171-3.el7 BUILT: Wed May 31 15:36:29 CEST 2017 device-mapper-1.02.140-3.el7 BUILT: Wed May 31 15:36:29 CEST 2017 device-mapper-libs-1.02.140-3.el7 BUILT: Wed May 31 15:36:29 CEST 2017 device-mapper-event-1.02.140-3.el7 BUILT: Wed May 31 15:36:29 CEST 2017 device-mapper-event-libs-1.02.140-3.el7 BUILT: Wed May 31 15:36:29 CEST 2017 device-mapper-persistent-data-0.7.0-0.1.rc6.el7 BUILT: Mon Mar 27 17:15:46 CEST 2017 cmirror-2.02.171-3.el7 BUILT: Wed May 31 15:36:29 CEST 2017 Putting in NEEDINFO re: Comment 17, to settle for sure that this does not belong in the release notes as a new feature. Okay, I read through this more carefully and I think I understand the situation: 1. Support for discard on LVM RAID 1 is not a new feature in 7.4. So we do not need a new-feature release note for that. 2. The reason several customers raised the question is because they received the message "discard operation is not supported" when they ran fstrim on an LVM RAID 1 device. This error was apparently caused by the fact that one or more of the underlying hardware devices do not support discard. It was just a coincidence that RAID 1 was in the mix. 3. There is no need to single out the RAID section of the LVM guide for a discussion about the reason for discard, or instructions on how to use it. It is a generic storage management function, not tied to LVM RAID. 4. Discard is covered in the Storage Management Guide: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Storage_Administration_Guide/ch02s04.html It seems to me the best way to avoid customer cases such as the ones attached to this BZ is to enhance that section of the Guide. For example, after the text: "Physical discard operations are supported if ... discard_max_bytes is not zero." we could add "If you issue the fstrim command on a device that does not support discard, the message "the discard operation is not supported" will be displayed. You will also get this message if you use the fstrim command on a logical device (LVM or MD) comprised of multiple devices where any one of the devices does not support discard." Then, someone will need to determine what message is displayed of you try to mount such a device with the -o discard option, and then add a similar paragraph to cover that scenario as well. (Or rearrange to provide one new paragraph that covers both batch discard (fstrim) and on-line discard (mount -o discard), assuming the behavior is similar. I am removing the Doc Type flag from theis BZ. John, Steven, if you agree the proposed change is useful, then clone this for the Storage Mgmt. Guide. I have cloned this as a documentation bug for the Storage Administration Guide, as per Comment 19. The needinfo in Comment 18 was addressed in Comment 19 so I'm clearing it. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:2222 |