Bug 1889146

Summary: fstrim fails to trim file systems on LVM raid volumes
Product: [Fedora] Fedora Reporter: Thomas Köller <thomas>
Component: dmraidAssignee: LVM and device-mapper development team <lvm-team>
Status: CLOSED UPSTREAM QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 32CC: agk, bmr, cfeist, heinzm, jonathan, kzak, lvm-team, mcsontos, prajnoha, prockai, zkabelac
Target Milestone: ---Flags: thomas: needinfo-
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 18:35:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Thomas Köller 2020-10-18 12:32:05 UTC
fstrim only works on file systems contained in LVM raid volumes if explicitly instructed to do so, neither the -a or -A options cause it to trim these:

[root@sarkovy thomas]# fstrim -nva
/boot/efi: 0 B (dry run) trimmed on /dev/sda1
/boot: 0 B (dry run) trimmed on /dev/sdb3
[root@sarkovy thomas]# fstrim -nvA
/boot/efi: 0 B (dry run) trimmed on /dev/sda1
/boot: 0 B (dry run) trimmed on /dev/sdb3
[root@sarkovy thomas]# fstrim -nv /
/: 0 B (dry run) trimmed
[root@sarkovy thomas]# fstrim -nv /home
/home: 0 B (dry run) trimmed

These are the contents of /etc/fstab:

LABEL=ROOT	/		ext4	defaults,x-systemd.device-timeout=0	1 1
LABEL=BOOT	/boot		ext4	defaults				1 2
LABEL=HOME	/home		ext4	defaults,nodev				1 2
LABEL=EFISYS	/boot/efi	vfat	defaults,nodev,dmask=0022,fmask=0133	1 3
LABEL=SWAP	swap		swap	defaults				0 0
htpc:/video	/mnt/video     	nfs	defaults,noauto,proto=tcp,_netdev	0 0

An these are the mounted file systems:

[root@sarkovy thomas]# mount -t ext4
/dev/mapper/SysStorage-RootFS on / type ext4 (rw,relatime,seclabel)
/dev/sdb3 on /boot type ext4 (rw,relatime,seclabel)
/dev/mapper/SysStorage-HomeFS on /home type ext4 (rw,nodev,relatime,seclabel)

Comment 1 Karel Zak 2020-10-19 09:22:49 UTC
Please, try 

   cat /sys/block/<name>/queue/discard_granularity

with proper kernel device name for /dev/mapper/SysStorage-HomeFS (for example "lsblk -o KNAME /dev/mapper/SysStorage-HomeFS") retruns the name,

or you can try "lsblk --discard" to get complete overview.

Note that fstrim ignores devices with zero discard_granularity (or DISC-GRAN in lsblk output).

Comment 2 Thomas Köller 2020-10-19 20:17:02 UTC
(In reply to Karel Zak from comment #1)
> Please, try 
> 
>    cat /sys/block/<name>/queue/discard_granularity
> 
> with proper kernel device name for /dev/mapper/SysStorage-HomeFS (for
> example "lsblk -o KNAME /dev/mapper/SysStorage-HomeFS") retruns the name,
> 

[thomas@sarkovy ~]$ lsblk -o KNAME /dev/mapper/SysStorage-HomeFS
KNAME
dm-9
[thomas@sarkovy ~]$ cat /sys/block/dm-9/queue/discard_granularity
0
[thomas@sarkovy ~]$ lsblk -o KNAME /dev/mapper/SysStorage-RootFS
KNAME
dm-4
[thomas@sarkovy ~]$ cat /sys/block/dm-4/queue/discard_granularity
0

> or you can try "lsblk --discard" to get complete overview.
> 

[thomas@sarkovy ~]$ lsblk --discard
NAME                           DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda                                   0      512B       2G         0
├─sda1                                0      512B       2G         0
└─sda2                                0      512B       2G         0
  ├─SysStorage-RootFS_rmeta_0         0      512B       2G         0
  │ └─SysStorage-RootFS               0        0B       0B         0
  ├─SysStorage-RootFS_rimage_0        0      512B       2G         0
  │ └─SysStorage-RootFS               0        0B       0B         0
  ├─SysStorage-HomeFS_rmeta_0         0      512B       2G         0
  │ └─SysStorage-HomeFS               0        0B       0B         0
  └─SysStorage-HomeFS_rimage_0        0      512B       2G         0
    └─SysStorage-HomeFS               0        0B       0B         0
sdb                                   0      512B       2G         0
├─sdb1                                0      512B       2G         0
├─sdb2                                0      512B       2G         0
│ ├─SysStorage-RootFS_rmeta_1         0      512B       2G         0
│ │ └─SysStorage-RootFS               0        0B       0B         0
│ ├─SysStorage-RootFS_rimage_1        0      512B       2G         0
│ │ └─SysStorage-RootFS               0        0B       0B         0
│ ├─SysStorage-HomeFS_rmeta_1         0      512B       2G         0
│ │ └─SysStorage-HomeFS               0        0B       0B         0
│ └─SysStorage-HomeFS_rimage_1        0      512B       2G         0
│   └─SysStorage-HomeFS               0        0B       0B         0
└─sdb3                                0      512B       2G         0
sdc                                   0        0B       0B         0
└─sdc2                                0        0B       0B         0
sdd                                   0        0B       0B         0
sde                                   0        0B       0B         0
└─sde2                                0        0B       0B         0
sr0                                   0        0B       0B         0

Comment 3 Karel Zak 2020-10-21 10:24:28 UTC
Thanks for details.

It's obvious (from lsblk --discard) that LVM does not follow discard_granularity from original devices. Reassigning to LVM team.

Comment 4 Zdenek Kabelac 2020-10-21 15:19:06 UTC
I'm wondering how is this related to this kernel commit: f0e90b6c663a7e3b4736cb318c6c7c589f152c28

https://github.com/torvalds/linux/commit/f0e90b6c663a7e3b4736cb318c6c7c589f152c28

Please retest with kernel 5.9 which should have this fix included.

Comment 5 Zdenek Kabelac 2020-10-21 17:26:36 UTC
For completeness this patch was added to this earlier one - e0910c8e4f87bb9f767e61a778b0d9271c4dc512

https://github.com/torvalds/linux/commit/e0910c8e4f87bb9f767e61a778b0d9271c4dc512

Comment 6 Thomas Köller 2020-10-21 23:06:14 UTC
(In reply to Zdenek Kabelac from comment #4)
> I'm wondering how is this related to this kernel commit:
> f0e90b6c663a7e3b4736cb318c6c7c589f152c28
> 
> https://github.com/torvalds/linux/commit/
> f0e90b6c663a7e3b4736cb318c6c7c589f152c28
> 
> Please retest with kernel 5.9 which should have this fix included.

I have a raid1 configuration, so this is most likely unrelated.

Comment 7 Zdenek Kabelac 2020-10-22 08:39:06 UTC
(In reply to Thomas Köller from comment #6)
> (In reply to Zdenek Kabelac from comment #4)
> > I'm wondering how is this related to this kernel commit:
> > f0e90b6c663a7e3b4736cb318c6c7c589f152c28
> > 
> > https://github.com/torvalds/linux/commit/
> > f0e90b6c663a7e3b4736cb318c6c7c589f152c28
> > 
> > Please retest with kernel 5.9 which should have this fix included.
> 
> I have a raid1 configuration, so this is most likely unrelated.

See the 'starting' patch in comment 5, (in comment 4 there was the final patch of this).

Please retest with 5.9 and let us know.

Comment 8 Thomas Köller 2020-10-26 11:01:02 UTC
> Please retest with 5.9 and let us know.

Seems there is currently no 5.9 kernel for fedora, not even in rawhide, or am I missing something?

Comment 9 Zdenek Kabelac 2020-10-26 11:25:22 UTC
(In reply to Thomas Köller from comment #8)
> > Please retest with 5.9 and let us know.
> 
> Seems there is currently no 5.9 kernel for fedora, not even in rawhide, or
> am I missing something?

I'm running myself no-debug repo kernel: 5.9.0-36.fc34.x86_64

See the outcome of this link:

https://koji.fedoraproject.org/koji/packageinfo?packageID=8

Comment 10 Thomas Köller 2020-10-27 16:28:51 UTC
Problem still exists in kernel 5.9.0:

[root@sarkovy ~]# uname -a
Linux sarkovy 5.9.0-36.fc34.x86_64 #1 SMP Mon Oct 12 13:40:33 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@sarkovy ~]# fstrim -vna
/boot/efi: 0 B (dry run) trimmed on /dev/sda1
/boot: 0 B (dry run) trimmed on /dev/sdb3
[root@sarkovy ~]# lsblk --discard
NAME                           DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda                                   0      512B       2G         0
├─sda1                                0      512B       2G         0
└─sda2                                0      512B       2G         0
  ├─SysStorage-RootFS_rmeta_0         0      512B       2G         0
  │ └─SysStorage-RootFS               0        0B       0B         0
  ├─SysStorage-RootFS_rimage_0        0      512B       2G         0
  │ └─SysStorage-RootFS               0        0B       0B         0
  ├─SysStorage-HomeFS_rmeta_0         0      512B       2G         0
  │ └─SysStorage-HomeFS               0        0B       0B         0
  └─SysStorage-HomeFS_rimage_0        0      512B       2G         0
    └─SysStorage-HomeFS               0        0B       0B         0
sdb                                   0      512B       2G         0
├─sdb1                                0      512B       2G         0
├─sdb2                                0      512B       2G         0
│ ├─SysStorage-RootFS_rmeta_1         0      512B       2G         0
│ │ └─SysStorage-RootFS               0        0B       0B         0
│ ├─SysStorage-RootFS_rimage_1        0      512B       2G         0
│ │ └─SysStorage-RootFS               0        0B       0B         0
│ ├─SysStorage-HomeFS_rmeta_1         0      512B       2G         0
│ │ └─SysStorage-HomeFS               0        0B       0B         0
│ └─SysStorage-HomeFS_rimage_1        0      512B       2G         0
│   └─SysStorage-HomeFS               0        0B       0B         0
└─sdb3                                0      512B       2G         0
sdc                                   0        0B       0B         0
└─sdc2                                0        0B       0B         0
sdd                                   0        0B       0B         0
sde                                   0        0B       0B         0
└─sde2                                0        0B       0B         0
sr0                                   0        0B       0B         0

Comment 11 Zdenek Kabelac 2020-10-27 17:07:00 UTC
So now - lvm2  --type raid0  is supported.
And quite likely  --type raid1  seems still unsupported.

Passing to Heinz for more insight comments (as I've believe it's been modified in past already).

Comment 12 Zdenek Kabelac 2020-10-27 17:40:23 UTC
Ok - seems the patch has not landed in  5.9

Please retry with now available:  5.10.0-0.rc1.56.fc34.x86_64

This should give you non-zero granularity.

Comment 13 Heinz Mauelshagen 2020-10-27 18:34:27 UTC
As Zdenek mentioned in the previous comment, with recent upstream kernels raid0/1/10 RaidLVs allow to process discards, thus fstrim will work.

Nonetheless, raid4/5/6 RaidLVs would need the dm-raid module parameter devices_handle_discard_safely set to 'Y'
but that presumes _knowing_ all PVs with such RaidLV allocations return zeroes reliably on reads from discarded blocks!

Besides, the same module paramter setting is needed for native MD's raid456 module to allow for discards.