Bug 1747153

Summary: Mark remaining dm-linear mapping for auto-remove after decryption is finished
Product: Red Hat Enterprise Linux 8 Reporter: Corey Marthaler <cmarthal>
Component: cryptsetupAssignee: Ondrej Kozina <okozina>
Status: CLOSED ERRATA QA Contact: guazhang <guazhang>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 8.1CC: agk, guazhang, jbrassow, mbroz, okozina, prajnoha, rhandlin
Target Milestone: rc   
Target Release: 8.2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: cryptsetup-2.2.2-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-04-28 16:54:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1757783    
Bug Blocks:    

Description Corey Marthaler 2019-08-29 18:54:23 UTC
Description of problem:
According to the feature request:
https://bugzilla.redhat.com/show_bug.cgi?id=1676622#c8

"If the decryption was run in online mode, remaining dm-linear mapping disappears when last user is gone (fs umount)." 

But it appears the only thing that removes the luks dm device is a luksClose operation which raises the question, isn't it dangerous to have this device still around and usable once the underlying device has been decrypted?

[root@hayes-02 ~]# lvs -a -o +devices
  LV              VG            Attr       LSize Pool Origin Data%  Meta%   Devices      
  POOL            snapper_thinp twi-aotz-- 4.00g             29.06  26.56   POOL_tdata(0)
  [POOL_tdata]    snapper_thinp Twi-ao---- 4.00g                            /dev/sdo1(1) 
  [POOL_tmeta]    snapper_thinp ewi-ao---- 4.00m                            /dev/sdn1(0) 
  fs_snap1        snapper_thinp Vwi-a-tz-- 1.00g POOL origin 16.24
  [lvol0_pmspare] snapper_thinp ewi------- 4.00m                            /dev/sdo1(0) 
  origin          snapper_thinp Vwi-a-tz-- 1.00g POOL        100.00
  other1          snapper_thinp Vwi-a-tz-- 1.00g POOL        0.00
  other2          snapper_thinp Vwi-a-tz-- 1.00g POOL        0.00
  other3          snapper_thinp Vwi-a-tz-- 1.00g POOL        0.00
  other4          snapper_thinp Vwi-a-tz-- 1.00g POOL        0.00
  other5          snapper_thinp Vwi-a-tz-- 1.00g POOL        0.00

# Encrypt existing filesystem using a detached header file (RFE 1566005)
cryptsetup reencrypt --encrypt /dev/snapper_thinp/origin --header /tmp/luks_detachedheader.20512                                                                                                                                             
cryptsetup luksOpen --disable-keyring /dev/snapper_thinp/origin luks_origin --header /tmp/luks_detachedheader.20512                                                                                                                          
mount /dev/mapper/luks_origin /mnt/origin                                                                                                                                                                                                    
                                                                                                                                                                                                                                             
# Checking files on /mnt/origin                                                                                                                                                                                                              
                                                                                                                                                                                                                                             
# Now decrypting filesystem                                                                                                                                                                                                                  
[root@hayes-02 ~]# echo "Str0ngP455w0rd###" | cryptsetup reencrypt --decrypt /dev/snapper_thinp/origin --header /tmp/luks_detachedheader.20512                                                                                               
Auto-detected active dm device 'luks_origin' for data device /dev/snapper_thinp/origin.                                                                                                                                                      
Finished, time 00:08.894, 1024 MiB written, speed 115.1 MiB/s                                                                                                                                                                                
                                                                                                                                                                                                                                             
[root@hayes-02 ~]# umount /mnt/origin                                                                                                                                                                                                        
Aug 29 11:41:01 hayes-02 kernel: XFS (dm-11): Unmounting Filesystem                                                                                                                                                                          
                                                                                                                                                                                                                                             
# dm mapping still exists, shouldn't a decrypt rmeove that mapping? Is it even valid anymore now?                                                                                                                                            
[root@hayes-02 ~]# dmsetup ls                                                                                                                                                                                                                
snapper_thinp-origin    (253:4)                                                                                                                                                                                                              
snapper_thinp-POOL      (253:3)
snapper_thinp-fs_snap1  (253:10)
snapper_thinp-other5    (253:9)
snapper_thinp-other4    (253:8)
snapper_thinp-other3    (253:7)
snapper_thinp-POOL-tpool        (253:2)
snapper_thinp-POOL_tdata        (253:1)
snapper_thinp-other2    (253:6)
luks_origin     (253:11)
snapper_thinp-POOL_tmeta        (253:0)
snapper_thinp-other1    (253:5)

[root@hayes-02 ~]# fuser /mnt/origin
[root@hayes-02 ~]# fuser /dev/mapper/luks_origin 

# It can be mounted again
[root@hayes-02 ~]# mount /dev/mapper/luks_origin /mnt/origin/
[root@hayes-02 ~]# umount /mnt/origin

# Once closed, it's no longer valid for opening
[root@hayes-02 ~]# cryptsetup luksClose luks_origin
[root@hayes-02 ~]# echo Str0ngP455w0rd### | cryptsetup luksOpen --disable-keyring /dev/snapper_thinp/origin luks_origin --header /tmp/luks_detachedheader.20512
No usable keyslot is available.


Version-Release number of selected component (if applicable):
kernel-4.18.0-139.el8    BUILT: Tue Aug 27 17:11:00 CDT 2019
cryptsetup-2.2.0-1.el8    BUILT: Fri Aug 16 01:22:41 CDT 2019
cryptsetup-libs-2.2.0-1.el8    BUILT: Fri Aug 16 01:22:41 CDT 2019

How reproducible:
Everytime

Comment 1 Ondrej Kozina 2019-08-30 07:59:10 UTC
(In reply to Corey Marthaler from comment #0)
> Description of problem:
> According to the feature request:
> https://bugzilla.redhat.com/show_bug.cgi?id=1676622#c8
> 
> "If the decryption was run in online mode, remaining dm-linear mapping
> disappears when last user is gone (fs umount)." 
> 
> But it appears the only thing that removes the luks dm device is a luksClose
> operation which raises the question, isn't it dangerous to have this device
> still around and usable once the underlying device has been decrypted?

I think we can close remaining dm-linear mapping (after decryption) with deferred flag so that it gets automatically removed after last close().

On the other hand I have to investigate first what impact it could have on udev. The device may remain in system for a long time if it has i.e. rootfs on top.

> (...)
> 
> # Once closed, it's no longer valid for opening
> [root@hayes-02 ~]# cryptsetup luksClose luks_origin
> [root@hayes-02 ~]# echo Str0ngP455w0rd### | cryptsetup luksOpen
> --disable-keyring /dev/snapper_thinp/origin luks_origin --header
> /tmp/luks_detachedheader.20512
> No usable keyslot is available.
> 

This requires some thinking. I'm not sure if we want to allow luksOpen at all. Theoretically we could allow it (with some warning: "This actually is *not* encrypted device anymore!"
Anyway the error message should be different more descriptive.

Comment 3 guazhang@redhat.com 2019-11-25 12:29:43 UTC
Hello


# rpm -qa |grep crypt
libxcrypt-4.1.1-4.el8.x86_64
crypto-policies-20191022-1.gite17cc3a.el8.noarch
libxcrypt-devel-4.1.1-4.el8.x86_64
cryptsetup-2.2.2-1.el8.x86_64
libgcrypt-1.8.3-4.el8.x86_64
cryptsetup-reencrypt-2.2.2-1.el8.x86_64
cryptsetup-libs-2.2.2-1.el8.x86_64
[root@storageqe-24 ~]#


#  lvs -a -o +devices
  LV              VG                Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices           
  home            rhel_storageqe-24 -wi-ao---- <5.38t                                                     /dev/sda3(8055)   
  root            rhel_storageqe-24 -wi-ao---- 50.00g                                                     /dev/sda3(1417734)
  swap            rhel_storageqe-24 -wi-ao---- 31.46g                                                     /dev/sda3(0)      
  POOL            snapper_thinp     twi-aot-D-  1.00g             100.00 23.54                            POOL_tdata(0)     
  [POOL_tdata]    snapper_thinp     Twi-ao----  1.00g                                                     /dev/sdc(1)       
  [POOL_tmeta]    snapper_thinp     ewi-ao----  4.00m                                                     /dev/sdc(257)     
  [lvol0_pmspare] snapper_thinp     ewi-------  4.00m                                                     /dev/sdc(0)       
  origin          snapper_thinp     Vwi-aot--- <1.98g POOL        50.59                                                     
  POOL            snapper_thinp1    twi-aot---  1.00g             100.00 24.22                            POOL_tdata(0)     
  [POOL_tdata]    snapper_thinp1    Twi-ao----  1.00g                                                     /dev/sde(1)       
  [POOL_tmeta]    snapper_thinp1    ewi-ao----  4.00m                                                     /dev/sde(257)     
  [lvol0_pmspare] snapper_thinp1    ewi-------  4.00m                                                     /dev/sde(0)       
  origin          snapper_thinp1    Vwi-aot---  1.00g POOL        100.00                                                    
  other1          snapper_thinp1    Vwi-a-t---  1.00g POOL        0.00                                                      
  other2          snapper_thinp1    Vwi-a-t---  1.00g POOL        0.00                                                      
  other3          snapper_thinp1    Vwi-a-t---  1.00g POOL        0.00                                                      
  other4          snapper_thinp1    Vwi-a-t---  1.00g POOL        0.00                                                      
  other5          snapper_thinp1    Vwi-a-t---  1.00g POOL        0.00                                                      
  lv01            vg01              -wi-a-----  1.00g                                                     /dev/sdb(0)       
  lv02            vg01              -wi-a-----  1.00g                                                     /dev/sdb(256)     
  lv03            vg01              -wi-a-----  1.00g                                                     /dev/sdb(512)     
  lv04            vg01              -wi-a-----  1.00g                                                     /dev/sdb(768)     
  lv05            vg01              -wi-a-----  1.00g                                                     /dev/sdb(1024)    
  lv06            vg01              -wi-a-----  1.00g                                                     /dev/sdb(1280)    


[root@storageqe-24 ~]# cryptsetup reencrypt --encrypt /dev/snapper_thinp1/origin --header /tmp/luks_detachedheader.20512 

WARNING!
========
Header file does not exist, do you want to create it?

Are you sure? (Type uppercase yes): YES
Enter passphrase for /tmp/luks_detachedheader.20512: 
Verify passphrase: 
Finished, time 00:11.169, 1024 MiB written, speed  91.7 MiB/s
[root@storageqe-24 ~]# cryptsetup luksOpen --disable-keyring /dev/snapper_thinp1/origin luks_origin1 --header /tmp/luks_detachedheader.20512
Enter passphrase for /dev/snapper_thinp1/origin: 
[root@storageqe-24 ~]# 
[root@storageqe-24 ~]# lsblk
NAME                          MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda                             8:0    0  5.5T  0 disk  
├─sda1                          8:1    0    1M  0 part  
├─sda2                          8:2    0    1G  0 part  /boot
└─sda3                          8:3    0  5.5T  0 part  
  ├─rhel_storageqe--24-root   253:0    0   50G  0 lvm   /
  ├─rhel_storageqe--24-swap   253:1    0 31.5G  0 lvm   [SWAP]
  └─rhel_storageqe--24-home   253:2    0  5.4T  0 lvm   /home
sdb                             8:16   0  5.5T  0 disk  
├─vg01-lv01                   253:3    0    1G  0 lvm   
├─vg01-lv02                   253:4    0    1G  0 lvm   
├─vg01-lv03                   253:5    0    1G  0 lvm   
├─vg01-lv04                   253:6    0    1G  0 lvm   
├─vg01-lv05                   253:7    0    1G  0 lvm   
└─vg01-lv06                   253:8    0    1G  0 lvm   
sdc                             8:32   0  5.5T  0 disk  
├─snapper_thinp-POOL_tmeta    253:9    0    4M  0 lvm   
│ └─snapper_thinp-POOL-tpool  253:11   0    1G  0 lvm   
│   ├─snapper_thinp-POOL      253:12   0    1G  0 lvm   
│   └─snapper_thinp-origin    253:13   0    2G  0 lvm   
│     └─luks_origin-overlay   253:16   0    2G  0 crypt 
│       └─luks_origin         253:14   0    2G  0 crypt 
└─snapper_thinp-POOL_tdata    253:10   0    1G  0 lvm   
  └─snapper_thinp-POOL-tpool  253:11   0    1G  0 lvm   
    ├─snapper_thinp-POOL      253:12   0    1G  0 lvm   
    └─snapper_thinp-origin    253:13   0    2G  0 lvm   
      └─luks_origin-overlay   253:16   0    2G  0 crypt 
        └─luks_origin         253:14   0    2G  0 crypt 
sdd                             8:48   0  5.5T  0 disk  
sde                             8:64   0  5.5T  0 disk  
├─snapper_thinp1-POOL_tmeta   253:17   0    4M  0 lvm   
│ └─snapper_thinp1-POOL-tpool 253:19   0    1G  0 lvm   
│   ├─snapper_thinp1-POOL     253:20   0    1G  0 lvm   
│   ├─snapper_thinp1-origin   253:21   0    1G  0 lvm   
│   │ └─luks_origin1          253:27   0    1G  0 crypt 
│   ├─snapper_thinp1-other1   253:22   0    1G  0 lvm   
│   ├─snapper_thinp1-other2   253:23   0    1G  0 lvm   
│   ├─snapper_thinp1-other3   253:24   0    1G  0 lvm   
│   ├─snapper_thinp1-other4   253:25   0    1G  0 lvm   
│   └─snapper_thinp1-other5   253:26   0    1G  0 lvm   
└─snapper_thinp1-POOL_tdata   253:18   0    1G  0 lvm   
  └─snapper_thinp1-POOL-tpool 253:19   0    1G  0 lvm   
    ├─snapper_thinp1-POOL     253:20   0    1G  0 lvm   
    ├─snapper_thinp1-origin   253:21   0    1G  0 lvm   
    │ └─luks_origin1          253:27   0    1G  0 crypt 
    ├─snapper_thinp1-other1   253:22   0    1G  0 lvm   
    ├─snapper_thinp1-other2   253:23   0    1G  0 lvm   
    ├─snapper_thinp1-other3   253:24   0    1G  0 lvm   
    ├─snapper_thinp1-other4   253:25   0    1G  0 lvm   
    └─snapper_thinp1-other5   253:26   0    1G  0 lvm   
sdf                             8:80   0  5.5T  0 disk  
sdg                             8:96   0  5.5T  0 disk  
sdh                             8:112  0  5.5T  0 disk  
sdi                             8:128  0  5.5T  0 disk  
sdj                             8:144  0  5.5T  0 disk  
sdk                             8:160  0  5.5T  0 disk  
└─sdk1                          8:161  0   10G  0 part  
sdl                             8:176  0  9.8G  0 disk  
luks_origin-hotzone-forward   253:15   0    2G  1 crypt 
└─luks_origin-overlay         253:16   0    2G  0 crypt 
  └─luks_origin               253:14   0    2G  0 crypt 
[root@storageqe-24 ~]# mount /dev/mapper/luks_origin1 /mnt/origin 
mount: /mnt/origin: wrong fs type, bad option, bad superblock on /dev/mapper/luks_origin1, missing codepage or helper program, or other error.
[root@storageqe-24 ~]# 



please have a look the error or something was wrong ?

Comment 4 Ondrej Kozina 2019-11-25 12:55:19 UTC
Hi,

sorry the bug description here is a bit vague. The problem was that we claimed in original RFE that after online decryption got finished the remaining useless dm-linear mapping would be automatically removed. And that did not work.

So to reproduce it you can run something like this (no need to add here such complicated storage stack):

1) cryptsetup luksFormat /dev/sdx --header my_new_luks2_header
2) cryptsetup open /dev/sdx --header my_new_luks2_header sdx_crypt
3) mkfs /dev/mapper/sdx_crypt
4) mount /dev/mapper/sdx_crypt /mnt/directory
5) cryptsetup reencrypt --decrypt /dev/sdx --header my_new_luks2_header
6) umount /mnt/directory

on old version:

The /dev/mapper/sdx_crypt would remain visible, but

on cryptsetup-2.2.2-1.el8 the /dev/mapper/sdx_crypt would disappear right after umount (or after last process using the device finished).

From your output above...Did you have working fs on /dev/snapper_thinp1/origin before you encrypted it with cryptsetup reencrypt --encrypt command?

Comment 5 guazhang@redhat.com 2019-11-26 07:06:23 UTC
Hello

thanks Ondrej for the details, I have tested with cryptsetup-2.2.2-1.el8.x86_64


[root@storageqe-24 ~]# cryptsetup luksFormat /dev/sdb1 --header my_new_luks2_header

WARNING!
========
Header file does not exist, do you want to create it?

Are you sure? (Type uppercase yes): YES
Enter passphrase for my_new_luks2_header: 
Verify passphrase: 
[root@storageqe-24 ~]# cryptsetup open /dev/sdb1 --header my_new_luks2_header sdx_crypt
Enter passphrase for /dev/sdb1: 
[root@storageqe-24 ~]# lsblk
NAME                        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda                           8:0    0  5.5T  0 disk  
├─sda1                        8:1    0    1M  0 part  
├─sda2                        8:2    0    1G  0 part  /boot
└─sda3                        8:3    0  5.5T  0 part  
  ├─rhel_storageqe--24-root 253:0    0   50G  0 lvm   /
  ├─rhel_storageqe--24-swap 253:1    0 31.5G  0 lvm   [SWAP]
  └─rhel_storageqe--24-home 253:2    0  5.4T  0 lvm   /home
sdb                           8:16   0  5.5T  0 disk  
└─sdb1                        8:17   0   10G  0 part  
  └─sdx_crypt               253:3    0   10G  0 crypt 
sdc                           8:32   0  5.5T  0 disk  
sdd                           8:48   0  5.5T  0 disk  
sde                           8:64   0  5.5T  0 disk  
sdf                           8:80   0  5.5T  0 disk  
sdg                           8:96   0  5.5T  0 disk  
sdh                           8:112  0  5.5T  0 disk  
sdi                           8:128  0  5.5T  0 disk  
sdj                           8:144  0  5.5T  0 disk  
sdk                           8:160  0  5.5T  0 disk  
sdl                           8:176  0  5.5T  0 disk  
[root@storageqe-24 ~]# mkfs.xfs -f /dev/mapper/sdx_crypt
meta-data=/dev/mapper/sdx_crypt  isize=512    agcount=4, agsize=655360 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1
data     =                       bsize=4096   blocks=2621440, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@storageqe-24 ~]#  mount /dev/mapper/sdx_crypt /mnt/bz/
[root@storageqe-24 ~]# lsblk
NAME                        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda                           8:0    0  5.5T  0 disk  
├─sda1                        8:1    0    1M  0 part  
├─sda2                        8:2    0    1G  0 part  /boot
└─sda3                        8:3    0  5.5T  0 part  
  ├─rhel_storageqe--24-root 253:0    0   50G  0 lvm   /
  ├─rhel_storageqe--24-swap 253:1    0 31.5G  0 lvm   [SWAP]
  └─rhel_storageqe--24-home 253:2    0  5.4T  0 lvm   /home
sdb                           8:16   0  5.5T  0 disk  
└─sdb1                        8:17   0   10G  0 part  
  └─sdx_crypt               253:3    0   10G  0 crypt /mnt/bz
sdc                           8:32   0  5.5T  0 disk  
sdd                           8:48   0  5.5T  0 disk  
sde                           8:64   0  5.5T  0 disk  
sdf                           8:80   0  5.5T  0 disk  
sdg                           8:96   0  5.5T  0 disk  
sdh                           8:112  0  5.5T  0 disk  
sdi                           8:128  0  5.5T  0 disk  
sdj                           8:144  0  5.5T  0 disk  
sdk                           8:160  0  5.5T  0 disk  
sdl                           8:176  0  5.5T  0 disk  
[root@storageqe-24 ~]# cryptsetup reencrypt --decrypt /dev/sdb1  --header my_new_luks2_header
Enter passphrase for /dev/sdb1: 
Auto-detected active dm device 'sdx_crypt' for data device /dev/sdb1.
Finished, time 02:27.421, 10240 MiB written, speed  69.5 MiB/s
[root@storageqe-24 ~]# lsblk
NAME                        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda                           8:0    0  5.5T  0 disk  
├─sda1                        8:1    0    1M  0 part  
├─sda2                        8:2    0    1G  0 part  /boot
└─sda3                        8:3    0  5.5T  0 part  
  ├─rhel_storageqe--24-root 253:0    0   50G  0 lvm   /
  ├─rhel_storageqe--24-swap 253:1    0 31.5G  0 lvm   [SWAP]
  └─rhel_storageqe--24-home 253:2    0  5.4T  0 lvm   /home
sdb                           8:16   0  5.5T  0 disk  
└─sdb1                        8:17   0   10G  0 part  
  └─sdx_crypt               253:3    0   10G  0 crypt /mnt/bz
sdc                           8:32   0  5.5T  0 disk  
sdd                           8:48   0  5.5T  0 disk  
sde                           8:64   0  5.5T  0 disk  
sdf                           8:80   0  5.5T  0 disk  
sdg                           8:96   0  5.5T  0 disk  
sdh                           8:112  0  5.5T  0 disk  
sdi                           8:128  0  5.5T  0 disk  
sdj                           8:144  0  5.5T  0 disk  
sdk                           8:160  0  5.5T  0 disk  
sdl                           8:176  0  5.5T  0 disk  
[root@storageqe-24 ~]# umount /mnt/bz/
[root@storageqe-24 ~]# lsblk
NAME                        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                           8:0    0  5.5T  0 disk 
├─sda1                        8:1    0    1M  0 part 
├─sda2                        8:2    0    1G  0 part /boot
└─sda3                        8:3    0  5.5T  0 part 
  ├─rhel_storageqe--24-root 253:0    0   50G  0 lvm  /
  ├─rhel_storageqe--24-swap 253:1    0 31.5G  0 lvm  [SWAP]
  └─rhel_storageqe--24-home 253:2    0  5.4T  0 lvm  /home
sdb                           8:16   0  5.5T  0 disk 
└─sdb1                        8:17   0   10G  0 part 
sdc                           8:32   0  5.5T  0 disk 
sdd                           8:48   0  5.5T  0 disk 
sde                           8:64   0  5.5T  0 disk 
sdf                           8:80   0  5.5T  0 disk 
sdg                           8:96   0  5.5T  0 disk 
sdh                           8:112  0  5.5T  0 disk 
sdi                           8:128  0  5.5T  0 disk 
sdj                           8:144  0  5.5T  0 disk 
sdk                           8:160  0  5.5T  0 disk 
sdl                           8:176  0  5.5T  0 disk 



so move to verified

Comment 7 errata-xmlrpc 2020-04-28 16:54:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1848