Hide Forgot
Description of problem: Apparently in Fedora 15, there is no longer an mtab file. The df command previously used the knowledge of bind mounts from the mtab to suppress the duplicate entries for the original filesystem and the bind mount. No df is printing multiple instances of the same filesystem and confusing the heck out of people. The df command should be modified to use the info in /proc/self/mountinfo so it can go back to printing only the one true root filesystem and not the additional copies with roots starting at subdirectories. See bug 701176 for much discussion of this issue (I obviously agree with the comments in there about df needing to be fixed, but couldn't find a bugzilla on df, so I created this one). Version-Release number of selected component (if applicable): coreutils-8.10-2.fc15.x86_64 How reproducible: 100% Steps to Reproduce: 1.Put a bind mount in fstab, reboot, and you'll see the root and and bind mount both listed as though they were the root mountpoint for the same disk. 2. 3. Actual results: duplicate entries shown by df Expected results: Just the root mountpoint for each disk shown Additional info:
Thanks for filing bugzilla for tracking the issue ... I'm quite sure there are more tools affected by this systemd change. There is recent discussion upstream about gnulib's mountlist module modification (http://lists.gnu.org/archive/html/bug-gnulib/2011-04/msg00048.html) - not really related to this, but probably could be handled together somehow...
I was just wondering if the simplest fix might be to make /etc/mtab a symlink to /proc/self/mtab and provide a /proc/self/mtab that has the exact format of the old /etc/mtab, but is a per namespace item. Really kludgy, but now everything that ever consulted mtab works like it did before.
df is not the only utility affected by the change of /etc/mtab to a link to /proc/self/mounts. mount(8) is also affected - mount -t ext4 shows entries for bind mounts made by sandbox, and there's no way to differentiate the bind mounts from the original non-bind ones: 532# mount -t ext4 /dev/mapper/vgF15-root on / type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/sdb1 on /boot type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/mapper/vgF15-var on /var type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/mapper/vgF15-opt on /opt type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/mapper/vgF15-build on /build type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/mapper/vgF15-usr on /usr type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/mapper/vgF15-local on /local type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/mapper/vgF15-home on /home type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/mapper/vgF15-var on /var/tmp type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) /dev/mapper/vgF15-home on /home type ext4 (rw,relatime,seclabel,barrier=1,data=ordered) 533# The following bind mounts appear: /tmp to /, /var/tmp to /var, and /home to /home. Because bind mounts cannot be differentiated by non-bind mounts, backup programs end up processing the same file systems multiple times, with the accompanying loss tape, time, and sanity (sysadmin's). My solution to this problem was to turn off sandbox. Why is sandbox run by default? Why are bind mount table entries indistinguishable from the original mounts? Why is the mount table so cluttered (this is progress?) :)
Michael, see discussion at bug #701176.
*** Bug 730138 has been marked as a duplicate of this bug. ***
*** Bug 746371 has been marked as a duplicate of this bug. ***
*** Bug 749356 has been marked as a duplicate of this bug. ***
> Because bind mounts cannot be differentiated by non-bind mounts and his ugly behavior should be fixed better sooner than later and should never have been introdiced in such a braindead way with F15 for making systemd-developers happy without thinking how this breaks df/mount and scripts relying on COREUTILS
Has any progress been made on this issue? I'm running F16 and seeing duplicate entries in df output.
This is still a problem in Fedora 16
Red Hathttps://bugzilla.redhat.com/show_bug.cgi?id=800312 You are not authorized to access Red Hatbug #800312 very well :-( this totally idiotic bug exists since a very long time wonerful life having the same drive 60 times because bind-mounts [root@arrakis:~]$ /bin/df | grep sdf1 | wc -l 60 ___________________ https://bugzilla.redhat.com/show_bug.cgi?id=709351 Ondrej Vasik <ovasik> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |800312
This "totally idiotic bug" is caused by "harmless" change of /etc/mtab to symlink requested by systemd guys... and solution still discussed upstream... If you have a patch (gnulib module) to resolve duplicity detection, it would be great. Otherwise, I can't promise anything about this bugzilla. And sorry about the private bugzilla, it is just tracking duplicate for RHEL...
------- Comment From maknayak.com 2012-04-24 10:06 EDT------- Here is the out put of "df -k" which displays info without any duplicate on F17 Alpha PPC64 on P7 Juno IOCL. [root@elm17f131 ~]# df -k Filesystem 1K-blocks Used Available Use% Mounted on rootfs 19399768 1986248 16440312 11% / devtmpfs 2002112 0 2002112 0% /dev tmpfs 2005952 0 2005952 0% /dev/shm tmpfs 2005952 8000 1997952 1% /run /dev/mapper/vg-lv_root 19399768 1986248 16440312 11% / tmpfs 2005952 0 2005952 0% /sys/fs/cgroup tmpfs 2005952 0 2005952 0% /media /dev/sda2 508745 124458 358687 26% /boot 9.47.82.129:/f17alpha/ 2405600 2405600 0 100% /mnt --- uname -a ---- Linux elm17f131.beaverton.ibm.com 3.3.1-5.fc17.ppc64 #1 SMP Wed Apr 11 06:19:45 MST 2012 ppc64 ppc64 ppc64 GNU/Linux Looks like F17 Alpha has fix for this issue. Thanks... Manas
(In reply to comment #13) > Looks like F17 Alpha has fix for this issue. I do not think so. As you can see there is at least the / duplicate and you do not see any other duplicates because there aren't any in the /proc/mounts. If you had sandbox or xguest on the system you would see more duplicates.
Just a follow up on the last two comments, (don't know about FC17 but) running FC16 on HP Prolient Amd64 w/RAID 1 on SATA driver, I get the same duplicates even when using df -k. As a sysadmin this is scary; I noticed it sometime after a fresh installs on our www and smtp servers, and at first I though I was either hacked, or my hardware mirror was causing some weirdness until I came across this bug. As you can see my df output is a mess. I know this is one of those bugs, but it's sooo in the admins face as I look at df often (especially on my file and db servers). This may be off topic, but I also notices that www shoes /dev/sda2 and smtp shows /dev/cciss/c0d0p2, did I frell up the install on www, I may have not used setup the RAID 1 I'll have to go back and look. [root@www ~]# uname -a Linux www.<redacted>.com 3.4.4-4.fc16.x86_64 #1 SMP Thu Jul 5 20:01:38 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux [root@www ~]# df -k Filesystem 1K-blocks Used Available Use% Mounted on rootfs 52324100 4910984 44791676 10% / devtmpfs 1014248 56 1014192 1% /dev tmpfs 1025264 0 1025264 0% /dev/shm tmpfs 1025264 564 1024700 1% /run /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% / tmpfs 1025264 0 1025264 0% /sys/fs/cgroup tmpfs 1025264 0 1025264 0% /media /dev/sda2 508644 73748 409296 16% /boot /dev/mapper/vg_www-lv_home 186662132 2990640 174326060 2% /home /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/var/named /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/etc/named /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/var/named/chroot/etc/named /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/etc/named.conf /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/var/named/chroot/etc/named.conf /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/etc/named.rfc1912.zones /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/var/named/chroot/etc/named.rfc1912.zones /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/etc/rndc.key /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/var/named/chroot/etc/rndc.key /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/usr/lib64/bind /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/var/named/chroot/usr/lib64/bind /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/etc/named.iscdlv.key /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/var/named/chroot/etc/named.iscdlv.key /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/etc/named.root.key /dev/mapper/vg_www-lv_root 52324100 4910984 44791676 10% /var/named/chroot/var/named/chroot/etc/named.root.key [root@www ~]# lvdisplay Found duplicate PV e28FHtdEdjXGW5mnBZ0EPNe2cKUI1upa: using /dev/sdb3 not /dev/sda3 --- Logical volume --- LV Name /dev/vg_www/lv_swap VG Name vg_www LV UUID N39u6s-HcEP-sliE-yqHa-YK9R-tMJ0-Od90BT LV Write Access read/write LV Status available # open 2 LV Size 3.94 GiB Current LE 126 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Name /dev/vg_www/lv_home VG Name vg_www LV UUID 0oQpey-l8af-Qs8m-zQhu-fK1W-j4Eu-SRSc2P LV Write Access read/write LV Status available # open 1 LV Size 178.25 GiB Current LE 5704 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2 --- Logical volume --- LV Name /dev/vg_www/lv_root VG Name vg_www LV UUID GCRnk7-qV0Y-kTXM-VmCK-OMaf-cV3H-I6eYwC LV Write Access read/write LV Status available # open 1 LV Size 50.00 GiB Current LE 1600 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1 [root@smtp ~]# df -m Filesystem 1M-blocks Used Available Use% Mounted on rootfs 51101 4122 44419 9% / devtmpfs 1936 0 1936 0% /dev tmpfs 1978 0 1978 0% /dev/shm /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% / tmpfs 1978 49 1930 3% /run tmpfs 1978 0 1978 0% /sys/fs/cgroup tmpfs 1978 0 1978 0% /media /dev/cciss/c0d0p2 497 97 376 21% /boot /dev/mapper/vg_smtp-lv_home 180477 2880 168562 2% /home /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% /var/named/chroot/etc/named /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% /var/named/chroot/var/named /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% /var/named/chroot/etc/named.conf /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% /var/named/chroot/etc/named.rfc1912.zones /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% /var/named/chroot/etc/rndc.key /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% /var/named/chroot/usr/lib64/bind /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% /var/named/chroot/etc/named.iscdlv.key /dev/mapper/vg_smtp-lv_root 51101 4122 44419 9% /var/named/chroot/etc/named.root.key [root@smtp ~]# lvdisplay --- Logical volume --- LV Name /dev/vg_smtp/lv_swap VG Name vg_smtp LV UUID afGZuG-4pt9-noIm-Veg0-DIZW-07HF-Z8dMnF LV Write Access read/write LV Status available # open 2 LV Size 5.88 GiB Current LE 188 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Name /dev/vg_smtp/lv_home VG Name vg_smtp LV UUID iKdlwk-gasq-uOme-ngAV-r0Bp-4P8N-BIl53l LV Write Access read/write LV Status available # open 1 LV Size 176.47 GiB Current LE 5647 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2 --- Logical volume --- LV Name /dev/vg_smtp/lv_root VG Name vg_smtp LV UUID bS5pfk-tL9j-73DW-O0ku-LPJM-gqX1-Yo3G3b LV Write Access read/write LV Status available # open 1 LV Size 50.00 GiB Current LE 1600 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1
the same bullshit on F17 and nobody acres i have here a server with 300 bind-mounts this is one of the category "braindead change" [root@srv-rhsoft:~]$ /bin/df -hT Dateisystem Typ Größe Benutzt Verf. Verw% Eingehängt auf rootfs rootfs 29G 7,5G 22G 26% / devtmpfs devtmpfs 7,8G 8,0K 7,8G 1% /dev tmpfs tmpfs 1,0G 4,0K 1,0G 1% /dev/shm tmpfs tmpfs 7,8G 4,7M 7,8G 1% /run /dev/md1 ext4 29G 7,5G 22G 26% / tmpfs tmpfs 7,8G 0 7,8G 0% /sys/fs/cgroup tmpfs tmpfs 7,8G 0 7,8G 0% /media tmpfs tmpfs 512M 0 512M 0% /var/www/sessiondata tmpfs tmpfs 200M 14M 187M 7% /var/cache/firefox /dev/md0 ext4 485M 71M 415M 15% /boot /dev/md2 ext4 3,6T 1,6T 2,0T 44% /mnt/data /dev/md2 ext4 3,6T 1,6T 2,0T 44% /home /dev/md2 ext4 3,6T 1,6T 2,0T 44% /tmp /dev/md2 ext4 3,6T 1,6T 2,0T 44% /var/tmp /dev/md2 ext4 3,6T 1,6T 2,0T 44% /Volumes/dune/www-servers /dev/md2 ext4 3,6T 1,6T 2,0T 44% /Volumes/dune/www-servers/phpincludes [root@srv-rhsoft:~]$ cat /etc/redhat-release Fedora release 17 (Beefy Miracle)
> This "totally idiotic bug" is caused by "harmless" change of > /etc/mtab to symlink requested by systemd guys... and > solution still discussed upstream... so WHY are these idiots permitted to introduce changes in the whole system and making the world turn around them without having fixes for the negative impacts BEFORE changes are introduced? /tmp on tmpfs is another example (off topic)
Created attachment 637288 [details] deduplicating df proof of concept (in Perl) This Perl script is a proof of concept of a de-duplicating df. The key points shown: - df must be taught to use /proc/self/mountinfo. There is not sufficient information in /proc/mounts. - mountinfo gives tells us the block device major:minor which we can use to recognize multiple mounts of the same device (bind mounts). - A good heuristic for deciding which of the duplicates to print is the length of the 4th field ("root") in mountinfo. Shorter is preferred. (The script requires perl-Filesys-Df to run.)
Hi, this has been discussed upstream here: http://lists.gnu.org/archive/html/coreutils/2012-08/msg00089.htm the patch was later amended here: http://lists.gnu.org/archive/html/coreutils/2012-09/msg00130.html and some request for feedback has been made here: http://lists.gnu.org/archive/html/coreutils/2012-10/msg00011.htm I'd be very thankful for some feedback on the patch's status Adding Pádraig Brady to CC
A bit modified solution accepted upstream - http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=10d35b438e731f6f0c1528d4855cdb9cf8b88349 , update will be done early next week.
coreutils-8.17-7.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/coreutils-8.17-7.fc18
Package coreutils-8.17-7.fc18: * should fix your issue, * was pushed to the Fedora 18 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing coreutils-8.17-7.fc18' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2012-20166/coreutils-8.17-7.fc18 then log in and leave karma (feedback).
Any chance on a backport to currently supported releases (FC16/FC17?)
> Any chance on a backport to currently supported releases hopefully! this unacceptable behavior on machines with some hundret bind-mount for sftp-chroot was reported for F15 and F18 does nobody interest in production this time for many reasons
Yes, I do plan to do F16/F17 update...
coreutils-8.15-9.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/coreutils-8.15-9.fc17
coreutils-8.12-9.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/coreutils-8.12-9.fc16
Woo! Just tested it on F16, and gave it an upvote! Thanks for getting this in!
coreutils-8.15-9.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
df now fails to show NFS mounts some of the time this seems to be part of the last update this is a CRITICAL failure df -h list of mounted systems not showing $HOME/Documents mounted from other machine
Steve: can you please be more specific? Which Fedora? Does df -a work?
df -a does work, system is fedora 17 (32 bit) with all updates --------------------------------------------- $ uname -r 3.6.10-2.fc17.i686.PAE $ rpm -qa | grep coreutils coreutils-8.15-9.fc17.i686 --------------------------------------------- # pwd /home/thorfinn # ls Documents/ | wc 108 117 1703 # umount /home/thorfinn/Documents/ umount.nfs4: /home/thorfinn/Documents: device is busy # sync # umount /home/thorfinn/Documents/ # ls Documents/ | wc 0 0 0 # /bin/mount -o rsize=8192,wsize=8192,soft hekla:/home/thorfinn/Documents/ /home/thorfinn/Documents/ # ls Documents/ | wc 108 117 1703 # df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 2.0G 4.0K 2.0G 1% /dev tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 2.0G 1020K 2.0G 1% /run /dev/sda3 450G 50G 378G 12% / tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup tmpfs 2.0G 0 2.0G 0% /media /dev/sda1 985M 265M 670M 29% /boot /dev/md0 2.7T 1.5T 1.2T 56% /home laxdala:/media/seagate/media/ 631G 524G 75G 88% /media/laxdala hekla:/home/thorfinn/.thunderbird 917G 445G 427G 52% /media/borg --------------------------------------------- # df -a Filesystem 1K-blocks Used Available Use% Mounted on rootfs 471650640 51845116 395847048 12% / proc 0 0 0 - /proc sysfs 0 0 0 - /sys devtmpfs 2055096 4 2055092 1% /dev devpts 0 0 0 - /dev/pts tmpfs 2063904 0 2063904 0% /dev/shm tmpfs 2063904 1020 2062884 1% /run /dev/sda3 471650640 51845116 395847048 12% / securityfs 0 0 0 - /sys/kernel/security selinuxfs 0 0 0 - /sys/fs/selinux tmpfs 2063904 0 2063904 0% /sys/fs/cgroup cgroup 0 0 0 - /sys/fs/cgroup/systemd cgroup 0 0 0 - /sys/fs/cgroup/cpuset cgroup 0 0 0 - /sys/fs/cgroup/cpu,cpuacct cgroup 0 0 0 - /sys/fs/cgroup/memory cgroup 0 0 0 - /sys/fs/cgroup/devices cgroup 0 0 0 - /sys/fs/cgroup/freezer cgroup 0 0 0 - /sys/fs/cgroup/net_cls cgroup 0 0 0 - /sys/fs/cgroup/blkio cgroup 0 0 0 - /sys/fs/cgroup/perf_event systemd-1 0 0 0 - /proc/sys/fs/binfmt_misc tmpfs 2063904 0 2063904 0% /media configfs 0 0 0 - /sys/kernel/config mqueue 0 0 0 - /dev/mqueue debugfs 0 0 0 - /sys/kernel/debug hugetlbfs 0 0 0 - /dev/hugepages binfmt_misc 0 0 0 - /proc/sys/fs/binfmt_misc sunrpc 0 0 0 - /var/lib/nfs/rpc_pipefs sunrpc 0 0 0 - /proc/fs/nfsd /dev/sda1 1007896 271200 685496 29% /boot /dev/md0 2884297944 1516943552 1220840428 56% /home fusectl 0 0 0 - /sys/fs/fuse/connections gvfs-fuse-daemon 0 0 0 - /run/user/thorfinn/gvfs laxdala:/media/seagate/media/ 661180784 549214112 77838800 88% /media/laxdala hekla:/home/thorfinn/.thunderbird 961428808 465893936 446697032 52% /media/borg hekla:/home/thorfinn/Videos 961428808 465893936 446697032 52% /media/hekla hekla:/home/thorfinn/Documents 961428808 465893936 446697032 52% /home/thorfinn/Documents
Ok thanks for info, that confirms the symptoms of the others. With /etc/mtab symlink to /proc/mounts, df suppresses duplicates in first come first serve manner - therefore may produce unexpected results. As you can see /media/borg (which is shown) has almost certainly the same dev_no as /home/thorfinn/Documents - and is before the /home/thorfinn/Documents in /proc/mounts - see https://bugzilla.redhat.com/show_bug.cgi?id=887763 which reports this behaviour. I don't see other option than some kind of sorting/heuristic done by df/kernel to produce better results. Other than that, we may only revert the suppressing duplicates, which is even worse option (as you can get all devices by df -a).
The correct algorithm does not need to be a heuristic, it just needs to use the information from the /proc/self/mountinfo file mentioned in comment #1 of this bug. That file contains all the info needed to correctly identify and suppress "bind mounts". The numbers at the front of the lines provide a tree structure which can identify the "root" filesystem versus the bind mounts. That way you can suppress all the non-root bind mounts, leaving the expected set of mounts for the df command to actually print.
Well, we still use the /etc/mtab symlink to /proc/mounts - using it is imho better for portability. However, maybe the /etc/mtab symlink could point to /proc/self/mountinfo... Karel, what do you think as the owner of this file?
(In reply to comment #36) > Well, we still use the /etc/mtab symlink to /proc/mounts - using it is imho > better for portability. However, maybe the /etc/mtab symlink could point to > /proc/self/mountinfo... Karel, what do you think as the owner of this file? No way, the file format is completely different and the format is de-facto unix standard.
coreutils-8.12-9.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.
coreutils-8.17-7.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.