Bug 1405238

Summary: findmnt --target behaviour changed in 7.3, shows all mount-points in chroot
Product: Red Hat Enterprise Linux 7 Reporter: Ian Wienand <iwienand>
Component: util-linuxAssignee: Karel Zak <kzak>
Status: CLOSED ERRATA QA Contact: Radka Brychtova <rskvaril>
Severity: high Docs Contact:
Priority: high    
Version: 7.3CC: brad, gpaterno, lyarwood, rskvaril
Target Milestone: rcKeywords: Regression, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1414481 (view as bug list) Environment:
Last Closed: 2017-08-01 21:43:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1414481    

Description Ian Wienand 2016-12-15 23:40:48 UTC 
In a rather odd series of events, we found that images built with diskimage-builder and newly released centos7.3 images silently failed to boot.  Eventually I figured out that dracut had not included ext4 modules into the initramfs, so it couldn't mount anything.  It turns out dracut has a little loop that uses findmnt to check if it can run in "hostonly" mode [1].  In this mode, it probes the block devices to see what file-systems are there, and only includes those modules.

What we are doing is building an vm image in a chroot environment.  This means that "/" is not a mount-point, and with 7.2 dracut was finding this out and taking itself out of hostonly mode, and consequently building a generic image with all file-system modules included.  With util-linux in 7.3, that behaviour appears to have changed

Below is what happens with the 7.2 version which I downloaded and ran inside the building chroot

---
bash-4.2# rpm2cpio ./util-linux-2.23.2-26.el7.x86_64.rpm | cpio -di
bash-4.2# ./usr/bin/findmnt --target "/"
bash-4.2# echo $?
1
---

And the same thing with the 7.3 version

---
bash-4.2# rpm -qf /usr/bin/findmnt
util-linux-2.23.2-33.el7.x86_64
bash-4.2# /usr/bin/findmnt --target "/"
TARGET      SOURCE                         FSTYPE   OPTIONS
/tmp/ccache /dev/vdb1[/image-cache/ccache] ext4     rw,relatime,data=ordered
/proc       none                           proc     rw,relatime
/dev        udev                           devtmpfs rw,relatime,size=4082668k,nr_inodes=1020667,mode=755
/dev/pts    devpts                         devpts   rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
/sys        none                           sysfs    rw,relatime
/tmp/yum    /dev/vdb1[/image-cache/yum]    ext4     rw,relatime,data=ordered
bash-4.2# echo $?
0
---

This opposite return-code means dracut continues in hostonly mode, ends up *not* finding the root block-device properly and generating and initramfs with no file-system modules.

Note this is something to do with "--target", as running it without that gives the same result on both versions

---
bash-4.2# /usr/bin/findmnt "/"   # 2.23.2-33
bash-4.2# echo $?
1

bash-4.2# ./usr/bin/findmnt "/"  # 2.23.2-26
bash-4.2# echo $?
1
---
Comment 1 Ian Wienand 2016-12-15 23:54:47 UTC 
So I guess it must be the patch at [1]

Checking /proc/self/mountinfo within the chroot shows the same list as output by the command above.

---
bash-4.2# cat /proc/self/mountinfo 
45 43 253:17 /image-cache/ccache /tmp/ccache rw,relatime - ext4 /dev/vdb1 rw,data=ordered
46 43 0:3 / /proc rw,relatime - proc none rw
47 43 0:5 / /dev rw,relatime - devtmpfs udev rw,size=4082668k,nr_inodes=1020667,mode=755
48 47 0:12 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
49 43 0:15 / /sys rw,relatime - sysfs none rw
50 43 253:17 /image-cache/yum /tmp/yum rw,relatime - ext4 /dev/vdb1 rw,data=ordered
---

[1] https://git.centos.org/blob/rpms!util-linux.git/b394b9eae4a21e566f3bd62a559095e09f013f0d/SOURCES!0071-findmnt-don-t-rely-on-st_dev-for-target.patch
Comment 2 Ian Wienand 2016-12-15 23:59:31 UTC 
Sorry, from original comment the dracut loop that has inverted is

 http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh#n1054
Comment 3 Giuseppe Paterno' 2016-12-16 09:45:48 UTC 
Same here. 
It SERIOUSLY affects OpenStack nova compute when mounting cinder over NFS.
Over nova debug, it findmnt returns 0 even if target is not there.
 findmnt --target /var/lib/nova/mnt/a972c1ef86f3d8d0143da59338f98ace --source 192.168.10.120:/export/mount/
Comment 4 Karel Zak 2016-12-16 10:55:14 UTC 
Seems like a bug, thanks for report.
Comment 5 Karel Zak 2016-12-16 11:02:04 UTC 
(In reply to Ian Wienand from comment #2)
> Sorry, from original comment the dracut loop that has inverted is
> 
>  http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh#n1054

BTW, this upstream dracut code seems fragile:

    for i in /sys /proc /run /dev; do
        if ! findmnt --target "$i" &>/dev/null; then
            dwarning "Turning off host-only mode: '$i' is not mounted!"
            unset hostonly
        fi
    done

if the goal is to check mountpoint. man findmnt:

-T, --target path
Define the mount target.  If path is not a mountpoint file or directory, then findmnt checks the path elements in reverse order to get the mountpoint (this feature is supported only when searching in kernel files and unsupported for --fstab).  It's recommended to use the option --mountpoint when checks of path  elements  are  unwanted  and path is a strictly specified mountpoint.


It would be better to use findmnt --mountpoint "$i". This option is available since util-linux v2.27. 

[This comment is not relevant for RHEL7 of course.]
Comment 15 errata-xmlrpc 2017-08-01 21:43:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2186