In a rather odd series of events, we found that images built with diskimage-builder and newly released centos7.3 images silently failed to boot. Eventually I figured out that dracut had not included ext4 modules into the initramfs, so it couldn't mount anything. It turns out dracut has a little loop that uses findmnt to check if it can run in "hostonly" mode [1]. In this mode, it probes the block devices to see what file-systems are there, and only includes those modules. What we are doing is building an vm image in a chroot environment. This means that "/" is not a mount-point, and with 7.2 dracut was finding this out and taking itself out of hostonly mode, and consequently building a generic image with all file-system modules included. With util-linux in 7.3, that behaviour appears to have changed Below is what happens with the 7.2 version which I downloaded and ran inside the building chroot --- bash-4.2# rpm2cpio ./util-linux-2.23.2-26.el7.x86_64.rpm | cpio -di bash-4.2# ./usr/bin/findmnt --target "/" bash-4.2# echo $? 1 --- And the same thing with the 7.3 version --- bash-4.2# rpm -qf /usr/bin/findmnt util-linux-2.23.2-33.el7.x86_64 bash-4.2# /usr/bin/findmnt --target "/" TARGET SOURCE FSTYPE OPTIONS /tmp/ccache /dev/vdb1[/image-cache/ccache] ext4 rw,relatime,data=ordered /proc none proc rw,relatime /dev udev devtmpfs rw,relatime,size=4082668k,nr_inodes=1020667,mode=755 /dev/pts devpts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 /sys none sysfs rw,relatime /tmp/yum /dev/vdb1[/image-cache/yum] ext4 rw,relatime,data=ordered bash-4.2# echo $? 0 --- This opposite return-code means dracut continues in hostonly mode, ends up *not* finding the root block-device properly and generating and initramfs with no file-system modules. Note this is something to do with "--target", as running it without that gives the same result on both versions --- bash-4.2# /usr/bin/findmnt "/" # 2.23.2-33 bash-4.2# echo $? 1 bash-4.2# ./usr/bin/findmnt "/" # 2.23.2-26 bash-4.2# echo $? 1 ---
So I guess it must be the patch at [1] Checking /proc/self/mountinfo within the chroot shows the same list as output by the command above. --- bash-4.2# cat /proc/self/mountinfo 45 43 253:17 /image-cache/ccache /tmp/ccache rw,relatime - ext4 /dev/vdb1 rw,data=ordered 46 43 0:3 / /proc rw,relatime - proc none rw 47 43 0:5 / /dev rw,relatime - devtmpfs udev rw,size=4082668k,nr_inodes=1020667,mode=755 48 47 0:12 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000 49 43 0:15 / /sys rw,relatime - sysfs none rw 50 43 253:17 /image-cache/yum /tmp/yum rw,relatime - ext4 /dev/vdb1 rw,data=ordered --- [1] https://git.centos.org/blob/rpms!util-linux.git/b394b9eae4a21e566f3bd62a559095e09f013f0d/SOURCES!0071-findmnt-don-t-rely-on-st_dev-for-target.patch
Sorry, from original comment the dracut loop that has inverted is http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh#n1054
Same here. It SERIOUSLY affects OpenStack nova compute when mounting cinder over NFS. Over nova debug, it findmnt returns 0 even if target is not there. findmnt --target /var/lib/nova/mnt/a972c1ef86f3d8d0143da59338f98ace --source 192.168.10.120:/export/mount/
Seems like a bug, thanks for report.
(In reply to Ian Wienand from comment #2) > Sorry, from original comment the dracut loop that has inverted is > > http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh#n1054 BTW, this upstream dracut code seems fragile: for i in /sys /proc /run /dev; do if ! findmnt --target "$i" &>/dev/null; then dwarning "Turning off host-only mode: '$i' is not mounted!" unset hostonly fi done if the goal is to check mountpoint. man findmnt: -T, --target path Define the mount target. If path is not a mountpoint file or directory, then findmnt checks the path elements in reverse order to get the mountpoint (this feature is supported only when searching in kernel files and unsupported for --fstab). It's recommended to use the option --mountpoint when checks of path elements are unwanted and path is a strictly specified mountpoint. It would be better to use findmnt --mountpoint "$i". This option is available since util-linux v2.27. [This comment is not relevant for RHEL7 of course.]
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:2186