Bug 1249623 - LVM thinp: lvextend fails after thinpool is full
LVM thinp: lvextend fails after thinpool is full
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: lvm2 (Show other bugs)
7.2
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: Zdenek Kabelac
cluster-qe@redhat.com
: Regression, ZStream
Depends On:
Blocks: 1285218
  Show dependency treegraph
 
Reported: 2015-08-03 08:35 EDT by Bruno Goncalves
Modified: 2015-12-01 08:11 EST (History)
11 users (show)

See Also:
Fixed In Version: lvm2-2.02.130-2.el7
Doc Type: Bug Fix
Doc Text:
This is in-release bug, no doc text required.
Story Points: ---
Clone Of:
: 1285218 (view as bug list)
Environment:
Last Closed: 2015-11-19 07:47:52 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Bruno Goncalves 2015-08-03 08:35:28 EDT
Description of problem:
After creating a thin volume and using all the space on thinpool it was not possible to run lvextend.

Version-Release number of selected component (if applicable):
lvm2-2.02.126-1.el7.x86_64
3.10.0-229.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.# dd if=/dev/zero of=thin_test.img count=0 seek=2048 bs=1M

2.# losetup /dev/loop0 thin_test.img

3.# vgcreate vgtest /dev/loop0

4.# lvcreate -L 200M -T vgtest/mythinpool -V1G -n thin1
  Logical volume "thin1" created.

5.Fill up thin pool by writing on thin1
# dd if=/dev/zero of=/dev/mapper/vgtest-thin1 bs=1M count=1K
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 69.0271 s, 15.6 MB/s

6.Create new LV, as there is no space on pool it SHOULD FAIL
# lvcreate -n thin2 -V 1G -T vgtest/mythinpool
  WARNING: Sum of all thin volume sizes (2.00 GiB) exceeds the size of thin pool vgtest/mythinpool and the size of whole volume group (2.00 GiB)!
  For thin pool auto extension activation/thin_pool_autoextend_threshold should be below 100.
  Free space in pool vgtest/mythinpool is above threshold, new volumes are not allowed.

7.# lvextend -L+200M vgtest/mythinpool
  For thin pool auto extension activation/thin_pool_autoextend_threshold should be below 100.
  Size of logical volume vgtest/mythinpool_tdata changed from 200.00 MiB (50 extents) to 400.00 MiB (100 extents).
  WARNING: Sum of all thin volume sizes (1.00 GiB) exceeds the size of thin pools (200.00 MiB)!
  Thin pool vgtest-mythinpool-tpool (253:5) transaction_id is 2, while expected 1.
  Problem reactivating logical volume vgtest/thin1.
  Releasing activation in critical section.
  libdevmapper exiting with 1 device(s) still suspended.
Actual results:


Expected results:
# lvextend -L+200M vgtest/mythinpool
  Size of logical volume vgtest/mythinpool_tdata changed from 200.00 MiB (50 extents) to 400.00 MiB (100 extents).
  Logical volume mythinpool successfully resized
Comment 2 Bruno Goncalves 2015-08-03 08:37:08 EDT
With lvm2-2.02.115-3.el7.x86_64 (default of RHEL-7.1) it works well.



Trying to create new LV, as there is no space on pool it SHOULD FAIL
# lvcreate -n thin2 -V 1G -T vgtest/mythinpool
  device-mapper: message ioctl on  failed: Invalid argument
  Failed to resume mythinpool.

# dmsetup status
vgtest-thin1: 0 2097152 thin 409600 409599
vgtest-mythinpool-tpool: 0 409600 thin-pool 1 35/1024 3200/3200 - ro discard_passdown queue_if_no_space 
vgtest-mythinpool_tdata: 0 409600 linear 
vgtest-mythinpool_tmeta: 0 8192 linear 
vgtest-mythinpool: 0 409600 linear 
rhel_dell--pe--sc1435--02-home: 0 862363648 linear 
rhel_dell--pe--sc1435--02-swap: 0 8388608 linear 
rhel_dell--pe--sc1435--02-root: 0 104857600 linear 

Going to extend thin pool
# lvextend -L+200M vgtest/mythinpool
  Size of logical volume vgtest/mythinpool_tdata changed from 200.00 MiB (50 extents) to 400.00 MiB (100 extents).
  Logical volume mythinpool successfully resized
Comment 9 Zdenek Kabelac 2015-09-15 05:02:50 EDT
This particular case should be fixed upstream with commit:
https://www.redhat.com/archives/lvm-devel/2015-September/msg00106.html
Comment 11 Corey Marthaler 2015-10-07 18:32:15 EDT
Fixed verified in the latest rpms.

3.10.0-319.el7.x86_64
lvm2-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
lvm2-libs-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
lvm2-cluster-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-libs-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-event-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-event-libs-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-persistent-data-0.5.5-1.el7    BUILT: Thu Aug 13 09:58:10 CDT 2015
cmirror-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
sanlock-3.2.4-1.el7    BUILT: Fri Jun 19 12:48:49 CDT 2015
sanlock-lib-3.2.4-1.el7    BUILT: Fri Jun 19 12:48:49 CDT 2015
lvm2-lockd-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015


[root@host-116 ~]# lvs -a -o +devices
  LV              VG            Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices       
  POOL            snapper_thinp twi-aot---   1.00g             0.04   1.56                             POOL_tdata(0) 
  [POOL_tdata]    snapper_thinp Twi-ao----   1.00g                                                     /dev/sdb1(1)  
  [POOL_tmeta]    snapper_thinp ewi-ao----   4.00m                                                     /dev/sdf1(0)  
  display_snap    snapper_thinp Vwi-a-t---   1.00g POOL origin 0.01                                                  
  [lvol0_pmspare] snapper_thinp ewi-------   4.00m                                                     /dev/sdb1(0)  
  origin          snapper_thinp Vwi-a-t---   1.00g POOL        0.01                                                  
  other1          snapper_thinp Vwi-a-t---   1.00g POOL        0.01                                                  
  other2          snapper_thinp Vwi-a-t---   1.00g POOL        0.01                                                  
  other3          snapper_thinp Vwi-a-t---   1.00g POOL        0.01                                                  
  other4          snapper_thinp Vwi-a-t---   1.00g POOL        0.01                                                  
  other5          snapper_thinp Vwi-a-t---   1.00g POOL        0.01                                                  

[root@host-116 ~]# dd if=/dev/zero of=/dev/snapper_thinp/other1 bs=1M count=1100
dd: error writing â/dev/snapper_thinp/other1â: No space left on device
1025+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 65.7462 s, 16.3 MB/s
dd: closing input file â/dev/zeroâ: Bad file descriptor

Oct  7 17:22:23 host-116 lvm[1486]: Thin snapper_thinp-POOL-tpool is now 97% full.
[779880.641370] device-mapper: thin: 253:4: reached low water mark for data device: sending event.
Oct  7 17:22:55 host-116 kernel: device-mapper: thin: 253:4: reached low water mark for data device: sending event.
[779880.650264] device-mapper: thin: 253:4: switching pool to out-of-data-space (queue IO) mode
Oct  7 17:22:55 host-116 kernel: device-mapper: thin: 253:4: switching pool to out-of-data-space (queue IO) mode
Oct  7 17:22:55 host-116 lvm[1486]: Thin snapper_thinp-POOL-tpool is now 100% full.
Oct  7 17:23:55 host-116 kernel: device-mapper: thin: 253:4: switching pool to out-of-data-space (error IO) mode
Oct  7 17:23:55 host-116 kernel: Buffer I/O error on device dm-7, logical block 262064
Oct  7 17:23:55 host-116 kernel: lost page write due to I/O error on dm-7
Oct  7 17:23:55 host-116 kernel: Buffer I/O error on device dm-7, logical block 262065
Oct  7 17:23:55 host-116 kernel: lost page write due to I/O error on dm-7
Oct  7 17:23:55 host-116 kernel: Buffer I/O error on device dm-7, logical block 262066
Oct  7 17:23:55 host-116 kernel: lost page write due to I/O error on dm-7
Oct  7 17:23:55 host-116 kernel: Buffer I/O error on device dm-7, logical block 262067
Oct  7 17:23:55 host-116 kernel: lost page write due to I/O error on dm-7
Oct  7 17:23:55 host-116 kernel: Buffer I/O error on device dm-7, logical block 262068
Oct  7 17:23:55 host-116 kernel: lost page write due to I/O error on dm-7
Oct  7 17:23:55 host-116 kernel: Buffer I/O error on device dm-7, logical block 262069

[root@host-116 ~]# lvs -a -o +devices
  LV              VG            Attr       LSize Pool Origin Data%  Meta%  Devices
  POOL            snapper_thinp twi-aot-D- 1.00g             100.00 14.16  POOL_tdata(0)
  [POOL_tdata]    snapper_thinp Twi-ao---- 1.00g                           /dev/sdb1(1)
  [POOL_tmeta]    snapper_thinp ewi-ao---- 4.00m                           /dev/sdf1(0)
  display_snap    snapper_thinp Vwi-a-t--- 1.00g POOL origin 0.01
  [lvol0_pmspare] snapper_thinp ewi------- 4.00m                           /dev/sdb1(0)
  origin          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other1          snapper_thinp Vwi-a-t--- 1.00g POOL        99.97
  other2          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other3          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other4          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other5          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01

[root@host-116 ~]# lvcreate  --virtualsize 1G -T snapper_thinp/POOL -n other6
  WARNING: Sum of all thin volume sizes (8.00 GiB) exceeds the size of thin pool snapper_thinp/POOL (1.00 GiB)!
  For thin pool auto extension activation/thin_pool_autoextend_threshold should be below 100.
  Free space in pool snapper_thinp/POOL is above threshold, new volumes are not allowed.

[root@host-116 ~]# lvextend -L+500M snapper_thinp/POOL
  WARNING: Sum of all thin volume sizes (7.00 GiB) exceeds the size of thin pools (1.00 GiB)!
  For thin pool auto extension activation/thin_pool_autoextend_threshold should be below 100.
  Size of logical volume snapper_thinp/POOL_tdata changed from 1.00 GiB (256 extents) to 1.49 GiB (381 extents).
  Logical volume POOL successfully resized.

[root@host-116 ~]# lvcreate  --virtualsize 1G -T snapper_thinp/POOL -n other6
  WARNING: Sum of all thin volume sizes (8.00 GiB) exceeds the size of thin pool snapper_thinp/POOL (1.49 GiB)!
  For thin pool auto extension activation/thin_pool_autoextend_threshold should be below 100.
  Logical volume "other6" created.

[root@host-116 ~]# lvs -a -o +devices
  LV              VG            Attr       LSize Pool Origin Data%  Meta%  Devices
  POOL            snapper_thinp twi-aot--- 1.49g             67.20  14.26  POOL_tdata(0)
  [POOL_tdata]    snapper_thinp Twi-ao---- 1.49g                           /dev/sdb1(1)
  [POOL_tmeta]    snapper_thinp ewi-ao---- 4.00m                           /dev/sdf1(0)
  display_snap    snapper_thinp Vwi-a-t--- 1.00g POOL origin 0.01
  [lvol0_pmspare] snapper_thinp ewi------- 4.00m                           /dev/sdb1(0)
  origin          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other1          snapper_thinp Vwi-a-t--- 1.00g POOL        99.97
  other2          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other3          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other4          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other5          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
  other6          snapper_thinp Vwi-a-t--- 1.00g POOL        0.01
Comment 12 errata-xmlrpc 2015-11-19 07:47:52 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2147.html

Note You need to log in before you can comment on or make changes to this bug.