Red Hat Bugzilla – Bug 1009546
fstrim on ext4 without journal
Last modified: 2014-01-12 10:19:07 EST
Description of problem: On a thinly-provisioned logical volume, when you run fstrim AFTER the first time, it only trims 124 MB of data even when you delete GB's of data. Version-Release number of selected component (if applicable): kernel-3.10.11-200.fc19.x86_64 kernel-modules-extra-3.10.11-200.fc19.x86_64 lvm2-2.02.98-12.fc19.x86_64 lvm2-libs-2.02.98-12.fc19.x86_64 util-linux-2.23.2-4.fc19.x86_64 How reproducible: Always Steps to Reproduce: This is a 100GB partition: # pvcreate /dev/sda3 # vgcreate vgthin1 # lvcreate --thinpool lvthinpool -l 25774 vgthin1 # lvcreate --name lvol1 --virtualsize 100.78GB --thin vgthin1/lvthinpool # mkfs.ext4 /dev/vgthin1/lvol1 # mount /dev/vgthin1/lvol1 /var/lib/libvirt/images # restorecon -R /var/lib/libvirt/images Actual results: # mount /var/lib/libvirt/images # fstrim -v /var/lib/libvirt/images /var/lib/libvirt/images: 99 GiB (106316812288 bytes) trimmed # dd if=/dev/zero of=/var/lib/libvirt/images/tempfile bs=1M count=7000 7000+0 records in 7000+0 records out 7340032000 bytes (7.3 GB) copied, 17.1512 s, 428 MB/s # rm /var/lib/libvirt/images/tempfile # sync # fstrim -v /var/lib/libvirt/images/tempfile /var/lib/libvirt/images: 124 MiB (130019328 bytes) trimmed Expected results: # fstrim -v /var/lib/libvirt/images/tempfile /var/lib/libvirt/images: 7 GiB (7516192768 bytes) trimmed Additional info: When I monitor the above steps (using "lvs vgthin1") I see the Data% usage grow as I create files but almost never shrinks back. I need to unmount & mount the filesystem so I can run fstrim for the first time so that it discards all free space (and eventually I'll see the thinpool data-usage statistics reflect that). As a side note: when I perform the steps described in this bug report using logical volumes (not thinly provisioned) I get the expected result. Finally, more info here: # ls -l /dev/mapper/vgthin1-lvol1 lrwxrwxrwx. 1 root root 8 Sep 18 10:41 /dev/mapper/vgthin1-lvol1 -> ../dm-14 # grep . /sys/block/dm-14/queue/* /sys/block/dm-14/queue/add_random:0 /sys/block/dm-14/queue/discard_granularity:512 /sys/block/dm-14/queue/discard_max_bytes:65536 /sys/block/dm-14/queue/discard_zeroes_data:0 /sys/block/dm-14/queue/hw_sector_size:512 /sys/block/dm-14/queue/iostats:0 /sys/block/dm-14/queue/logical_block_size:512 /sys/block/dm-14/queue/max_hw_sectors_kb:32767 /sys/block/dm-14/queue/max_integrity_segments:0 /sys/block/dm-14/queue/max_sectors_kb:512 /sys/block/dm-14/queue/max_segments:168 /sys/block/dm-14/queue/max_segment_size:65536 /sys/block/dm-14/queue/minimum_io_size:512 /sys/block/dm-14/queue/nomerges:0 /sys/block/dm-14/queue/nr_requests:128 /sys/block/dm-14/queue/optimal_io_size:65536 /sys/block/dm-14/queue/physical_block_size:512 /sys/block/dm-14/queue/read_ahead_kb:128 /sys/block/dm-14/queue/rotational:0 /sys/block/dm-14/queue/rq_affinity:0 /sys/block/dm-14/queue/scheduler:none /sys/block/dm-14/queue/write_same_max_bytes:0
Had some typos: fstrim -v /var/lib/libvirt/images/tempfile should be of course: fstrim -v /var/lib/libvirt/images/
I forgot to mention that all this was without using the "discard" mount option. If I use "discard" everything works as expected. As soon as I delete a file I see that the space is reclaimed in the thin pool (lvs vgthin1).
In the email you've mentioned that you're using ext4 without journal. I can not see it in the steps to reproduce, does it mean that you have it defined in the /etc/mke2fs.conf, or you're using journal after all ? Please confirm. In no journal mode there is a bug which prevents fstrim to do anything after the first run. Patch has already been send upstream. -Lukas
I wrote the steps on how to redproduce out of my mind (based on what I remembered). I totally forgot that for this logical volume I created it without journal...so to answer your question the steps to reproduce should include: mkfs.ext4 -O ^has_journal /dev/vgthin1/lvol2 ...instead of the plain vanilla mkfs.ext4
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There is a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 19 kernel bugs. Fedora 19 has now been rebased to 3.12.6-200.fc19. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 20, and are still experiencing this issue, please change the version to Fedora 20. If you experience different issues, please open a new bug report for those.
Patch accepted upstream (commit 8f9ff189205a6817aee5a1f996f876541f86e07c) and submitted for stable 3.16.y Thanks! -Lukas
(In reply to Lukáš Czerner from comment #6) > Patch accepted upstream (commit 8f9ff189205a6817aee5a1f996f876541f86e07c) > and submitted for stable 3.16.y > > Thanks! > -Lukas Er... I think you mean 3.12.y? If so, it should make the next 3.12.y release, so we'll leave this in POST for right now and pick it up via that.
Yeah, of course I meant 3.12.y :) Btw, it's also submitted for 3.10.y. -Lukas
3.12.7 is in updates-testing now.
Thanks Josh. I did an update of my Fedora 20 system with: yum update --enablerepo=updates-testing kernel ...and installed 3.12.7. I then removed the discard option for my non-journal-EXT4 filesystem and proceeded with the fstrim tests. It works perfectly fine now. Thank you all & specially to Lukáš Czerner.