Bug 887763

Summary: The output of the command "df" (without any param) omits the line for the root filesystem
Product: [Fedora] Fedora Reporter: Joachim Backes <joachim.backes>
Component: coreutilsAssignee: Ondrej Vasik <ovasik>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: abuse, admiller, bugs.michael, cra, erik, gholms, h.reindl, imsmthnelse, kdudka, kzak, mail, mmarzantowicz, orion, ovasik, pbrady, p, pingou, samuel-rhbugs, twaugh
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-08-09 14:23:51 EDT Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:

Description Joachim Backes 2012-12-17 04:28:00 EST
Description of problem:

On my fully updated (coreutils-8.17-7.fc18.x86_64)
F18 box, the command "df" without any param does not output the size of
the root filesystem:

> df

Filesystem     1K-blocks     Used Available Use% Mounted on
devtmpfs         1652700        0   1652700   0% /dev
tmpfs            1670312       92   1670220   1% /dev/shm
tmpfs            1670312     3524   1666788   1% /run
tmpfs            1670312        0   1670312   0% /sys/fs/cgroup
rpc_pipefs     105732188 10614892  89746368  11% /var/lib/nfs/rpc_pipefs
tmpfs            1670312       52   1670260   1% /tmp
/dev/sdb1       96124904 49765628  41476324  55% /VBOX_BACKUP
/dev/sdb3       96124936 33037220  58204760  37% /SYSTEM_BACKUP
/dev/sdb2       96124936 11326920  79915060  13% /DISK2_BACKUP
/dev/sda2       43256140   418748  40640104   2% /wine
/dev/sda5       48062440  3947828  41673136   9% /disk2
/dev/sda6       96124904 24978264  66263688  28% /vbox

Version-Release number of selected component (if applicable):
coreutils-8.17-7.fc18.x86_64

How reproducible:
always!

Steps to Reproduce:
1.df
2.
3.
  
Actual results:
No line for the root FS

Expected results:
Line for the root FS is included

Additional info:

1. coreutils.x86_64 0:8.17-6.fc18 does not show this effect!
2. /etc/mtab is correct (it has a line for the root FS)
Comment 1 Joachim Backes 2012-12-17 04:43:06 EST
Output of the mount command: (see the /dev/sda8 line)

proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=1652700k,nr_inodes=413175,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)

/dev/sda8 on / type ext4 (rw,relatime,seclabel,data=ordered)

systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
configfs on /sys/kernel/config type configfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
tmpfs on /tmp type tmpfs (rw,relatime,seclabel)proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=1652700k,nr_inodes=413175,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
/dev/sda8 on / type ext4 (rw,relatime,seclabel,data=ordered)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
configfs on /sys/kernel/config type configfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
tmpfs on /tmp type tmpfs (rw,relatime,seclabel)
/dev/sdb3 on /SYSTEM_BACKUP type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda6 on /vbox type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sdb1 on /VBOX_BACKUP type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda2 on /wine type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda5 on /disk2 type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sdb2 on /DISK2_BACKUP type ext4 (rw,relatime,seclabel,data=ordered)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
/dev/sdb3 on /SYSTEM_BACKUP type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda6 on /vbox type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sdb1 on /VBOX_BACKUP type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda2 on /wine type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda5 on /disk2 type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sdb2 on /DISK2_BACKUP type ext4 (rw,relatime,seclabel,data=ordered)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
Comment 2 Michael Schwendt 2012-12-17 06:00:18 EST
As observed on test list, the filesystem attributes in the following line from the broken "df" output match Joachim's root filesystem:

rpc_pipefs     105732188 10614892  89746368  11% /var/lib/nfs/rpc_pipefs

Compare with:
# df /  
/dev/sda8      105732188 10614892  89746368  11% /  

An entry for /var/lib/nfs/rpc_pipefs is not in the mount output.
Comment 3 Mateusz Marzantowicz 2012-12-17 08:07:36 EST
I confirm that this bug affects my system too:

$ rpm -q coreutils
coreutils-8.17-7.fc18.i686

$ df
Filesystem              1K-blocks     Used Available Use% Mounted on
devtmpfs                  1012788        0   1012788   0% /dev
tmpfs                     1028084       72   1028012   1% /dev/shm
tmpfs                     1028084     2392   1025692   1% /run
tmpfs                     1028084        0   1028084   0% /sys/fs/cgroup
rpc_pipefs               10079084  5080644   4486440  54% /var/lib/nfs/rpc_pipefs
tmpfs                     1028084        8   1028076   1% /tmp
/dev/sda2                  495844   139277    330967  30% /boot
/dev/mapper/fedora-home 116222396 87046880  23271748  79% /home

$ df -a
Filesystem              1K-blocks     Used Available Use% Mounted on
rootfs                   10079084  5080644   4486440  54% /
proc                            0        0         0    - /proc
sysfs                           0        0         0    - /sys
devtmpfs                  1012788        0   1012788   0% /dev
securityfs                      0        0         0    - /sys/kernel/security
selinuxfs                       0        0         0    - /sys/fs/selinux
tmpfs                     1028084       72   1028012   1% /dev/shm
devpts                          0        0         0    - /dev/pts
tmpfs                     1028084     2392   1025692   1% /run
tmpfs                     1028084        0   1028084   0% /sys/fs/cgroup
cgroup                          0        0         0    - /sys/fs/cgroup/systemd
cgroup                          0        0         0    - /sys/fs/cgroup/cpuset
cgroup                          0        0         0    - /sys/fs/cgroup/cpu,cpuacct
cgroup                          0        0         0    - /sys/fs/cgroup/memory
cgroup                          0        0         0    - /sys/fs/cgroup/devices
cgroup                          0        0         0    - /sys/fs/cgroup/freezer
cgroup                          0        0         0    - /sys/fs/cgroup/net_cls
cgroup                          0        0         0    - /sys/fs/cgroup/blkio
cgroup                          0        0         0    - /sys/fs/cgroup/perf_event
rpc_pipefs               10079084  5080644   4486440  54% /var/lib/nfs/rpc_pipefs
/dev/mapper/fedora-root  10079084  5080644   4486440  54% /
systemd-1                       0        0         0    - /proc/sys/fs/binfmt_misc
mqueue                          0        0         0    - /dev/mqueue
hugetlbfs                       0        0         0    - /dev/hugepages
debugfs                         0        0         0    - /sys/kernel/debug
tmpfs                     1028084        8   1028076   1% /tmp
configfs                        0        0         0    - /sys/kernel/config
/dev/sda2                  495844   139277    330967  30% /boot
/dev/mapper/fedora-home 116222396 87046880  23271748  79% /home
fusectl                         0        0         0    - /sys/fs/fuse/connections
gvfsd-fuse                      0        0         0    - /run/user/1000/gvfs
binfmt_misc                     0        0         0    - /proc/sys/fs/binfmt_misc
Comment 4 Ondrej Vasik 2012-12-17 08:14:19 EST
Thanks for report, probably something got wrong in the backport for F18. I'm just curious - does it affect F17/F16 as well? There are pending updates for these released Fedoras... and I suspect these are affected too by this update.
Comment 5 Mateusz Marzantowicz 2012-12-17 08:36:08 EST
On Fedora 17 (64 bit):

$ rpm -q coreutils
coreutils-8.15-9.fc17.x86_64

df shows root file system as expected.
Comment 6 Kamil Dudka 2012-12-17 09:47:55 EST
(In reply to comment #2)
> As observed on test list, the filesystem attributes in the following line
> from the broken "df" output match Joachim's root filesystem:
> 
> rpc_pipefs     105732188 10614892  89746368  11% /var/lib/nfs/rpc_pipefs
> 
> Compare with:
> # df /  
> /dev/sda8      105732188 10614892  89746368  11% /  

Good point.  /var/lib/nfs/rpc_pipefs comes before /.  Hence, if they have the same device number, the latter will be skipped.

Joachim, could you please attach the output of the following command?

stat --printf "%d\t%n\n" / /var/lib/nfs/rpc_pipefs
Comment 7 Mateusz Marzantowicz 2012-12-17 12:34:41 EST
$ stat --printf "%d\t%n\n" / /var/lib/nfs/rpc_pipefs
64769	/
64769	/var/lib/nfs/rpc_pipefs
Comment 8 Joachim Backes 2012-12-17 13:14:16 EST
(In reply to comment #6)
> (In reply to comment #2)
> > As observed on test list, the filesystem attributes in the following line
> > from the broken "df" output match Joachim's root filesystem:
> > 
> > rpc_pipefs     105732188 10614892  89746368  11% /var/lib/nfs/rpc_pipefs
> > 
> > Compare with:
> > # df /  
> > /dev/sda8      105732188 10614892  89746368  11% /  
> 
> Good point.  /var/lib/nfs/rpc_pipefs comes before /.  Hence, if they have
> the same device number, the latter will be skipped.
> 
> Joachim, could you please attach the output of the following command?
> 
> stat --printf "%d\t%n\n" / /var/lib/nfs/rpc_pipefs


stat --printf "%d\t%n\n" / /var/lib/nfs/rpc_pipefs
2056	/
2056	/var/lib/nfs/rpc_pipefs
Comment 9 Ondrej Vasik 2012-12-17 15:33:40 EST
So, the deduplication works and root filesystem is shown - but first come first serve causes the issue... and as /proc/mounts can be random, df now produces random output. This is really not perfect - as it may affect other things as well. Some kind of predictibility should probably be added to df deduplication logic.
Comment 10 Pádraig Brady 2012-12-17 18:30:27 EST
So on a Fedora 15 system here, these have different devices:

$ stat --printf "%d\t%n\n" / /var/lib/nfs/rpc_pipefs
2066	/
36	/var/lib/nfs/rpc_pipefs

$ df /var/lib/nfs/rpc_pipefs
Filesystem           1K-blocks      Used Available Use% Mounted on
sunrpc                       0         0         0   -  /var/lib/nfs/rpc_pipefs

Is this something specific to sunrpc on Fedora 18,
or is it something more general like bind mounts
not being handled or something.
Comment 11 Kamil Dudka 2012-12-17 19:05:13 EST
(In reply to comment #10)
> Is this something specific to sunrpc on Fedora 18,
> or is it something more general

It must be something more general.  I executed the above command on my f18 and f19 VMs and the device numbers differed in both cases.  It might have been caused by the fact that both the VMs did not actually use NFS.
Comment 12 Bernhard Voelker 2013-01-22 19:50:30 EST
I think it could be the following:
/var/lib/nfs/rpc_pipefs has the same device number as / because
it may either have been "shadowed" by another mount, or the file
system is really no longer mounted but it has not been deleted
from /etc/mtab. (On a system with /etc/mtab being a symlink to
/proc/self/mounts, the latter is not a problem even with
'umount -n').
Comment 13 Harald Reindl 2013-02-05 02:45:08 EST
and "util-linux-2.22.2-3.fc18.x86_64" kills df / mount completly

[root@rh:~]$ cat /downloads/util-linux.txt 
/usr/bin/df: cannot read table of mounted file systems: No such file or directory

Feb 05 08:31:18 Updated: libuuid-2.22.2-3.fc18.x86_64
Feb 05 08:31:19 Updated: libblkid-2.22.2-3.fc18.x86_64
Feb 05 08:31:19 Updated: libmount-2.22.2-3.fc18.x86_64
Feb 05 08:31:24 Updated: util-linux-2.22.2-3.fc18.x86_64
____________________________________

after downgrade you need a full reboot to get any output from "Mount"
Comment 14 Harald Reindl 2013-02-05 04:50:32 EST
and another nice side effect

Feb  5 10:47:57 srv-rhsoft dovecot: master: Dovecot v2.1.14 starting up (core dumps disabled)
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: / is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/systemd is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/cpuset is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/memory is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/devices is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/freezer is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/net_cls is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/blkio is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/perf_event is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /dev/mqueue is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/kernel/debug is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /dev/hugepages is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/kernel/security is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /proc/sys/fs/binfmt_misc is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Feb  5 10:47:57 srv-rhsoft dovecot: master: Warning: /sys/fs/cgroup/cpu,cpuacct is no longer mounted. See http://wiki2.dovecot.org/Mountpoints
Comment 15 Karel Zak 2013-02-05 05:23:13 EST
Harald, 

I don't see anything strange after update to util-linux 2.22.2-3. All works as expected. (Note that the update fixes hexdump, liblkid cache and cal(1) -- nothing related to mount(8) or df(1))


Please, try 

 $ ls -la /etc/mtab
 $ strace df
  
to see more details.
Comment 16 Harald Reindl 2013-02-05 05:30:09 EST
it seems that currently after ANY downgrade / upgrade of util-linux "mount" and "df" are stopping to work until reboot the machine which reminds me to windows
Comment 17 Harald Reindl 2013-02-05 06:00:28 EST
proven on a testmachine, after reboot "mount" and "df" are possible again
but with this util-linux version rootfs is also no longer visible as it was before - this is all wired and implies that coreutils AND util-linux a) play together and b) have currently real problems

[root@testserver:~]$ df
Dateisystem    Typ      Größe Benutzt Verf. Verw% Eingehängt auf
/dev/sdb1      ext4       12G    4,3G  7,6G   37% /
/dev/sda1      ext4      189M     42M  148M   23% /boot
/dev/sdd1      ext4      5,0G    220M  4,8G    5% /home
/dev/sdc1      ext4       30G    9,7G   20G   33% /Volumes/dune

Aktualisiert:
  fedora-logos.noarch 0:17.0.3-3.fc18       gl-manpages.noarch 0:1.1-5.20130122.fc18 kde-runtime.x86_64 0:4.9.5-2.fc18     
  kde-runtime-drkonqi.x86_64 0:4.9.5-2.fc18 kde-runtime-flags.noarch 0:4.9.5-2.fc18  kde-runtime-libs.x86_64 0:4.9.5-2.fc18
  libblkid.x86_64 0:2.22.2-3.fc18           libmount.x86_64 0:2.22.2-3.fc18          libuser.x86_64 0:0.58-2.fc18          
  libuuid.x86_64 0:2.22.2-3.fc18            perl-Log-Dispatch.noarch 0:2.35-1.fc18   util-linux.x86_64 0:2.22.2-3.fc18     
________________________________________________________

[root@testserver:~]$ df
df: Lesen der Tabelle eingehängter Dateisysteme nicht möglich: Datei oder Verzeichnis nicht gefunden
________________________________________________________

after reboot

[root@testserver:~]$ df
Dateisystem    Typ   Größe Benutzt Verf. Verw% Eingehängt auf
/dev/sda1      ext4   189M     42M  148M   23% /boot
/dev/sdd1      ext4   5,0G    220M  4,8G    5% /home
/dev/sdc1      ext4    30G    9,7G   20G   33% /Volumes/dune
Comment 18 Karel Zak 2013-02-05 06:48:15 EST
I really don't see relation between util-linux and df(1) output. The /etc/mtab is symlink and /proc/mounts and /proc/self/mountinfo are generated by kernel.

> [root@testserver:~]$ df
> df: Lesen der Tabelle eingehängter Dateisysteme nicht möglich: Datei oder
> Verzeichnis nicht gefunden

this is reason why I asked for strace output.
Comment 19 Harald Reindl 2013-02-05 06:54:16 EST
> I really don't see relation between util-linux and df(1) 
> output. The /etc/mtab is symlink and /proc/mounts and 
> /proc/self/mountinfo are generated by kernel

well, the kernel knows about the mounts but the coreutils/util-linux crap 
does not as you can see below very clear


[root@rh:~]$ /bin/mount | grep /dev/md1
[root@rh:~]$ cat /proc/mounts | grep /dev/md1
/dev/md1 / ext4 rw,noatime,nodiratime,commit=45,stripe=256,inode_readahead_blks=256 0 0
Comment 20 Harald Reindl 2013-02-05 07:04:07 EST
> The /etc/mtab is symlink

and the crap of util-linux is killing the "stat /etc/mtab" link at update

[root@testserver:~]$ stat /etc/mtab
stat: cannot stat '/etc/mtab': No such file or directory

[root@testserver:~]$ df
df: cannot read table of mounted file systems: No such file or directory

[root@testserver:~]$ ln -s /proc/mounts /etc/mtab

[root@testserver:~]$ df
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sdb1      ext4       12G  4.3G  7.6G  37% /
/dev/sda1      ext4      189M   42M  148M  23% /boot
/dev/sdd1      ext4      5.0G  220M  4.8G   5% /home
/dev/sdc1      ext4       30G  9.7G   20G  33% /Volumes/dune

and on THIS machine with util-linux-2.22.1-2.4.fc18.x86_64 the rootfs appears again as you can see above, on my both physical machines not, that is all really really frustrating and i can not see any improvment do the /etc/mtab behavior for decades which was replaced starting with F15
______________________________________

[root@testserver:~]$ LANG=c; strace /bin/df -hT
execve("/bin/df", ["/bin/df", "-hT"], [/* 26 vars */]) = 0
brk(0)                                  = 0x1aa1000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7feafc857000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=104753, ...}) = 0
mmap(NULL, 104753, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7feafc83d000
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\33\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=2067976, ...}) = 0
mmap(NULL, 3896312, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7feafc280000
mprotect(0x7feafc42d000, 2097152, PROT_NONE) = 0
mmap(0x7feafc62d000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1ad000) = 0x7feafc62d000
mmap(0x7feafc633000, 17400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7feafc633000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7feafc83c000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7feafc83a000
arch_prctl(ARCH_SET_FS, 0x7feafc83a740) = 0
mprotect(0x7feafc62d000, 16384, PROT_READ) = 0
mprotect(0x615000, 4096, PROT_READ)     = 0
mprotect(0x7feafc858000, 4096, PROT_READ) = 0
munmap(0x7feafc83d000, 104753)          = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=104781616, ...}) = 0
mmap(NULL, 104781616, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7feaf5e92000
close(3)                                = 0
brk(0)                                  = 0x1aa1000
brk(0x1ac2000)                          = 0x1ac2000
brk(0)                                  = 0x1ac2000
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2444, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7feafc856000
read(3, "# Locale name alias data base.\n#"..., 4096) = 2444
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x7feafc856000, 4096)            = 0
open("/usr/lib/locale/c/LC_IDENTIFICATION", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/etc/mtab", O_RDONLY|O_CLOEXEC)   = -1 ENOENT (No such file or directory)
write(2, "/bin/df: ", 9/bin/df: )                = 9
write(2, "cannot read table of mounted fil"..., 41cannot read table of mounted file systems) = 41
write(2, ": No such file or directory", 27: No such file or directory) = 27
write(2, "\n", 1
)                       = 1
close(1)                                = 0
close(2)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++
Comment 21 Bernhard Voelker 2013-02-05 08:18:26 EST
> open("/etc/mtab", O_RDONLY|O_CLOEXEC)   = -1 ENOENT

df is using gnulib's read_file_system_list() which in turn
uses setmntent("/etc/mtab", "r") followed by getmntent()
calls.

If your system lacks /etc/mtab (either as file or as link
to /proc/mounts or /proc/self/mounts), then I'd *guess*
it's not df's fault. ;-)

BTW: you didn't tell which coreutils version you have. There
has been greater work on src/df.c in the Git repo in the last
6 months.
Comment 22 Harald Reindl 2013-02-05 08:32:09 EST
> If your system lacks /etc/mtab then I'd *guess* it's not df's fault. ;-)

see this for util-linux:
https://bugzilla.redhat.com/show_bug.cgi?id=907749

but IF /etc/mtab is there (which is after a reboot) it is still wired that recently RANDOMLY the rootfs is not listed in df/mount-output

> BTW: you didn't tell which coreutils version you have

coreutils-8.17-8.fc18.x86_64
if you read my name you can assume that i tested the latest koji-builds or at least updates-tsting in case of strange things happening before report anything
Comment 23 Bernhard Voelker 2013-02-05 08:50:11 EST
BTW: df will suppress the rootfs entry in future anyway (unless -a).
I'm no Fedora guy, but I thought Ondrej already included such
a patch in F18:
http://pkgs.fedoraproject.org/cgit/coreutils.git/commit/?h=f18&id=1b40d50d
Comment 24 Harald Reindl 2013-02-05 09:03:55 EST
> BTW: df will suppress the rootfs entry in future anyway (unless -a)

what a bullshit idea

-a list also bind-mounts which was e REAL problem for years after F15
so in the future we will no longer see the rootfs in logwatch or
the some hundret bind-mounts are coming back to the list?

why do developers not leave the world in peace with changing working things and step back to the times where /etc/mtab did work in 99.9% of all cases exactly as expected as also mlocate and so on?
Comment 25 Bernhard Voelker 2013-02-05 09:31:29 EST
I bet we're talking about different things here.

*This* bug is about df not showing the "root file system",
meaning the "/" entry. This was in fact a bug in df and was
a side effect of hiding bind mounts.

On the other hand, I was talking about the "rootfs"
pseudo file system which is used during early boot.

Feel free to read the 2 related upstream commits:
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=bb116d35
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=f25519d6
and the discussion on the upstream mailing list:
http://lists.gnu.org/archive/html/coreutils/2013-01/msg00093.html

Have a nice day,
Berny
Comment 26 Harald Reindl 2013-02-05 10:04:45 EST
> *This* bug is about df not showing the "root file system",
> meaning the "/" entry

when i say rootfs i ALWAYS mean /
Comment 27 Karel Zak 2013-02-05 10:28:36 EST
coreutils guys, wouldn't be possible that without mtab ln(1) does not work? util-linux spec file contains:

  %post
  rm -f /etc/mtab
  ln -s /proc/mounts /etc/mtab

.. I still don't see a reason why Harald lost mtab after update.
Comment 28 Karel Zak 2013-02-05 10:30:06 EST
*** Bug 907749 has been marked as a duplicate of this bug. ***
Comment 29 Harald Reindl 2013-02-05 11:35:50 EST
yes, looking at the SPEC-file i see also no reason or difference between the "ln -s /proc/mounts /etc/mtab" which was in fact the same i did manually to make df/mount happy again after realize what goes really wrong

on the other hand "rm -f" and "ln -s" on each and every update 
in %post is crappy and may lead to race conditions, in fact
/etc/mtab as symlink was introduced in F15, the real fix would
be that all the shiny tools use /proc/mounts directly or at least
the symlink should be included in the files-list instead this
%post snippet

%post is really really a bad place for such things and way
too often misused in the fedora-world for quick and dirty
workarounds which sadly staing forever until they make
visible problems
Comment 30 Joachim Backes 2013-02-05 11:48:42 EST
(In reply to comment #27)
> coreutils guys, wouldn't be possible that without mtab ln(1) does not work?
> util-linux spec file contains:
> 
>   %post
>   rm -f /etc/mtab
>   ln -s /proc/mounts /etc/mtab
> 
> .. I still don't see a reason why Harald lost mtab after update.

I had the same problem! But after reboot, /etc/mtab appeared again!

But not as link to /proc/mounts, it is a simple file:

ls -l /etc/mtab
-rw-r--r--. 1 root root 676 Feb  5 17:34 /etc/mtab
Comment 31 Bernhard Voelker 2013-02-06 02:48:18 EST
(In reply to comment #27)
> coreutils guys, wouldn't be possible that without mtab ln(1) does not work?

no, ln is totally unlrelated to /etc/mtab.

> .. I still don't see a reason why Harald lost mtab after update.

The only reason I can imagine is that /proc was not mounted (somehow).
In that case, df also hits ENOENT even if the correct symlink is in place.
Comment 32 Harald Reindl 2013-02-06 04:03:27 EST
WTF how do you imagine /proc not mounted and "df" working before update / downgrade util-linux but broken after? how often and on how many machines should i reproduce the game below until someone realizes that it is absolutely crap to mangle /etc/mtab in rpm-post-scripts?

on the users list someone had also the problem that /etc/mtab was instead a symlinka real file, i have seen even this and i can reprdouce this behavior on 2 virtual machines as also on two physical ones

[root@testserver:~]$ df
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sdb1      ext4       12G  4.3G  7.6G  37% /
/dev/sda1      ext4      189M   42M  148M  23% /boot
/dev/sdd1      ext4      5.0G  220M  4.8G   5% /home
/dev/sdc1      ext4       30G  9.7G   20G  33% /Volumes/dune

[root@testserver:~]$ stat /etc/mtab 
  File: '/etc/mtab' -> '/proc/mounts'
  Size: 12              Blocks: 0          IO Block: 4096   symbolic link
Device: 811h/2065d      Inode: 41046       Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root

Removed:
  libblkid.x86_64 0:2.22.2-3.fc18           libmount.x86_64 0:2.22.2-3.fc18         libuuid.x86_64 0:2.22.2-3.fc18        
  util-linux.x86_64 0:2.22.2-3.fc18        

Installed:
  libblkid.x86_64 0:2.22.1-2.4.fc18         libmount.x86_64 0:2.22.1-2.4.fc18       libuuid.x86_64 0:2.22.1-2.4.fc18      
  util-linux.x86_64 0:2.22.1-2.4.fc18   

[root@testserver:~]$ df
df: cannot read table of mounted file systems: No such file or directory

[root@testserver:~]$ stat /etc/mtab 
stat: cannot stat '/etc/mtab': No such file or directory

[root@testserver:~]$ ln -s /proc/mounts /etc/mtab

[root@testserver:~]$ df
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sdb1      ext4       12G  4.3G  7.6G  37% /
/dev/sda1      ext4      189M   42M  148M  23% /boot
/dev/sdd1      ext4      5.0G  220M  4.8G   5% /home
/dev/sdc1      ext4       30G  9.7G   20G  33% /Volumes/dune

Updated:
  libblkid.x86_64 0:2.22.2-3.fc18           libmount.x86_64 0:2.22.2-3.fc18         libuuid.x86_64 0:2.22.2-3.fc18        
  util-linux.x86_64 0:2.22.2-3.fc18    

[root@testserver:~]$ df
df: cannot read table of mounted file systems: No such file or directory

[root@testserver:~]$ stat /etc/mtab 
stat: cannot stat '/etc/mtab': No such file or directory

[root@testserver:~]$ ln -s /proc/mounts /etc/mtab

[root@testserver:~]$ df
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sdb1      ext4       12G  4.3G  7.6G  37% /
/dev/sda1      ext4      189M   42M  148M  23% /boot
/dev/sdd1      ext4      5.0G  220M  4.8G   5% /home
/dev/sdc1      ext4       30G  9.7G   20G  33% /Volumes/dune
[root@testserver:~]$
Comment 33 Bernhard Voelker 2013-02-06 04:28:41 EST
(In reply to comment #32)
> i can reprdouce this behavior on 2 virtual machines as also on two physical ones

Thanks for confirming that ln basically works without /etc/mtab being there - that was Karel's question.

I don't know the reason for such a post script either.
Comment 34 Fedora Update System 2013-02-06 05:33:46 EST
util-linux-2.22.2-4.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/util-linux-2.22.2-4.fc18
Comment 35 Harald Reindl 2013-02-06 05:41:27 EST
the same as before - update "util-linux" and /etc/mtab is gone


Aktualisiert:
  libblkid.x86_64 0:2.22.2-4.fc18           libmount.x86_64 0:2.22.2-4.fc18         libuuid.x86_64 0:2.22.2-4.fc18        
  util-linux.x86_64 0:2.22.2-4.fc18        

Komplett!
[root@rh:/downloads]$ df
/usr/bin/df: Lesen der Tabelle eingehängter Dateisysteme nicht möglich: Datei oder Verzeichnis nicht gefunden
Comment 36 Karel Zak 2013-02-06 05:42:44 EST
I have added  if [ ! -L /etc/mtab ]; then ... to the %post to avoid unnecessary rm & ln.
Comment 37 Harald Reindl 2013-02-06 06:15:58 EST
but after update to util-linux.x86_64 0:2.22.2-4.fc18 /etc/mtab is still removed and df / mount are only working after "ln -s /proc/mounts /etc/mtab"
Comment 38 Karel Zak 2013-02-06 06:29:05 EST
I see, strange. It would be nice to found a way how to debug the problem. 

Try to update without yum and by "rpm -vv -U ...". Not sure.
Comment 39 Joachim Backes 2013-02-06 07:27:47 EST
(In reply to comment #37)
> but after update to util-linux.x86_64 0:2.22.2-4.fc18 /etc/mtab is still
> removed and df / mount are only working after "ln -s /proc/mounts /etc/mtab"

Same effect on my box :-(
Comment 40 Harald Reindl 2013-02-06 07:49:59 EST
what about a package without ANY %post-and-whatever-scriptlets?

util-linux is not the type of packge where i mangle outside of yum
Comment 41 Harald Reindl 2013-02-08 02:54:19 EST
oh and this complete /etc/mtab crap is broken

god knows what replaced the symlink with a physical file this time
but besides the fact that each update/downgrade of util-linux kills 
it complete it looks like this may be the reason that / is missing 
from "df" or at least one reason

[root@testserver:~]$ LANG=c; stat /etc/mtab 
  File: '/etc/mtab'
  Size: 813             Blocks: 8          IO Block: 4096   regular file
Device: 811h/2065d      Inode: 41059       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2013-02-08 08:37:49.370939104 +0100
Modify: 2013-02-08 08:37:49.370939104 +0100
Change: 2013-02-08 08:37:49.370939104 +0100
 Birth: -

[root@testserver:~]$ df
Filesystem     Type   Size  Used Avail Use% Mounted on
/dev/sda1      ext4   189M   42M  148M  23% /boot
/dev/sdc1      ext4    30G  9.6G   21G  33% /Volumes/dune
/dev/sdd1      ext4   5.0G  220M  4.8G   5% /home

[root@testserver:~]$ unlink /etc/mtab 

[root@testserver:~]$ ln -s /proc/mounts /etc/mtab
[root@testserver:~]$ df
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sdb1      ext4       12G  4.1G  7.8G  35% /
/dev/sda1      ext4      189M   42M  148M  23% /boot
/dev/sdc1      ext4       30G  9.6G   21G  33% /Volumes/dune
/dev/sdd1      ext4      5.0G  220M  4.8G   5% /home
Comment 42 Karel Zak 2013-02-08 03:58:09 EST
I have tried:

yum -y -d 10 --rpmverbosity=debug -v update --enablerepo=updates-testing util-linux libuuid libblkid libmount  &> ~/log

to debug the update and all seems correct.
Comment 43 Harald Reindl 2013-02-08 04:00:04 EST
this is all really really wired since i am not the only one with this problems, both, the missing symlink as also facing /etc/mtab as physical file
Comment 44 Karel Zak 2013-02-08 08:32:56 EST
(In reply to comment #43)
> the missing symlink as also facing /etc/mtab as physical file

This is expected behaviour, without the symlink the libmount enables the classic mtab mode and maintains the file.

The question is why there is no the symlink after the update.
Comment 45 Michael Schwendt 2013-02-08 09:08:30 EST
I wonder if Harald (or anyone who can reproduce the problem so easily) can attach the output of "rpm -Uvvh ..." and perhaps build the package with a modified scriptlet in an attempt at debugging whether/why "ln" fails without an error return code?

Just curious (and despite that I don't like the scriptlet anyway), if instead of

  ...
  rm -f /etc/mtab
  ln -s /proc/mounts /etc/mtab
  ...

one did

  ...
  ln -s /proc/mounts /etc/mtab.NEW
  mv -f /etc/mtab.NEW /etc/mtab
  ...

what would happen?
Comment 46 Harald Reindl 2013-02-08 09:25:58 EST
i have to drink about that this evening :-)

well, snapshot of the VM while have a newer and a older version from koji with all the affected packages should do the job without much pain - whatever here happens makes me scary
___________________

the replacement of /etc/mtab with a file today is clear
Feb 08 07:10:56 Updated: util-linux-2.22.2-4.fc18.x86_64

i did not really look at the package-list and so rebootet for other reasons my testbed without manually restore the symlink and without look at "df" since it was in the middle of the night for my biorhythm :-)
Comment 47 Karel Zak 2013-02-08 09:33:26 EST
(In reply to comment #45)
> I wonder if Harald (or anyone who can reproduce the problem so easily) can
> attach the output of "rpm -Uvvh ..." and perhaps build the package with a
> modified scriptlet in an attempt at debugging whether/why "ln" fails without
> an error return code?

The command from comment #42 returns all necessary information and on my system it's pretty clear that version 2.22.2-4 does not call ls(1) at all if there is the symlink:

D: %post(util-linux-2.22.2-4.fc18.x86_64): execv(/bin/sh) pid 13477
+ '[' -d /var/log ']'
+ touch /var/log/lastlog
+ chown root:root /var/log/lastlog
+ chmod 0644 /var/log/lastlog
+ '[' -x /usr/sbin/selinuxenabled ']'
+ /usr/sbin/selinuxenabled
+ '[' '!' -L /etc/mtab ']'
D: %post(util-linux-2.22.2-4.fc18.x86_64): waitpid(13477) rc 13477 status 0

Harald, can you share the VM image (gziped or so) with us so we can reproduce the problem?
Comment 48 Harald Reindl 2013-02-08 09:45:19 EST
> Harald, can you share the VM image (gziped or so) with us 
> so we can reproduce the problem?

not this one, it contains a copy of too many credentials and kyes as also security-live-configurations because it's my "here any needed package is so long built and rebuilt until i can update the company main-repo-and-buildserver" machine to prepare yum-dist-upgrades for the whole infrastructure

BUT since this happens too on both of my physical machines i would wonder if my minimal-goldenmaster would not behave the same way, i will give it a shot ASAP and pack the VMware image of it is reproduceable temporary for download
Comment 49 Harald Reindl 2013-02-08 11:03:19 EST
shit - the minimal-setup i could offer to upload can not reproduce it :-(

aaaah this would have been a small and clean VM with 1.3 GB uncompressed upgraded from fedora 9 until Fedora 18 with yum and my testserver where i can reproduce this was cloned years ago from it

may it be that mounts-points are palying in this game?
________________________________________________________________-

[root@testserver:~]$ cat /etc/fstab 
UUID=6f7807bd-a508-44c7-9e72-651f82b6ad6c  /boot                                 ext4   defaults                                                                                         0 1
UUID=9b4bf81a-5b1e-4922-b0d1-e6b65e9b61f9  /                                     ext4   defaults,data=writeback,commit=20,barrier=0,delalloc,inode_readahead_blks=64,noatime,nodiratime  0 1
UUID=9cc2a2d7-3e27-4f1a-a797-e74a328c32e8  /home                                 ext4   defaults,data=writeback,commit=20,barrier=0,delalloc,inode_readahead_blks=64,noatime,nodiratime  0 2
UUID=2195961e-4644-4401-ae30-61884ea1a26b  /Volumes/dune                         ext4   defaults,data=writeback,commit=20,barrier=0,delalloc,inode_readahead_blks=64,noatime,nodiratime  0 2

tmpfs                                      /var/www/sessiondata                  tmpfs  defaults,noatime,nodiratime,size=10m
/mnt/256Mb.swap                            swap                                  swap   defaults

/Volumes/dune/www-servers/flexiorder       /usr/local/sftp-homes/flow/flexiorder none   bind
/Volumes/dune/www-servers/fwx              /usr/local/sftp-homes/flow/fwx        none   bind
/Volumes/dune/www-servers/cms              /usr/local/sftp-homes/flow/cms        none   bind
/Volumes/dune/builduser                    /home/builduser                       none   bind
/Volumes/dune/.tmp                         /tmp                                  none   bind
/Volumes/dune/.tmp                         /var/tmp                              none   bind

________________________________________________________________-

[root@testserver:~]$ mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,size=1017212k,nr_inodes=254303,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
/dev/sdb1 on / type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=36,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
tmpfs on /var/www/sessiondata type tmpfs (rw,noatime,nodiratime,size=10240k)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/sda1 on /boot type ext4 (rw,relatime)
/dev/sdc1 on /Volumes/dune type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
/dev/sdc1 on /var/tmp type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
/dev/sdc1 on /tmp type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
/dev/sdc1 on /usr/local/sftp-homes/flow/cms type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
/dev/sdc1 on /usr/local/sftp-homes/flow/fwx type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
/dev/sdc1 on /usr/local/sftp-homes/flow/flexiorder type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
/dev/sdd1 on /home type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
/dev/sdc1 on /home/builduser type ext4 (rw,noatime,nodiratime,nobarrier,commit=20,data=writeback,inode_readahead_blks=64)
________________________________________________________________-
Comment 50 Charles R. Anderson 2013-02-08 14:55:42 EST
Just a guess(In reply to comment #45)
> I wonder if Harald (or anyone who can reproduce the problem so easily) can
> attach the output of "rpm -Uvvh ..." and perhaps build the package with a
> modified scriptlet in an attempt at debugging whether/why "ln" fails without
> an error return code?
> 
> Just curious (and despite that I don't like the scriptlet anyway), if
> instead of
> 
>   ...
>   rm -f /etc/mtab
>   ln -s /proc/mounts /etc/mtab
>   ...
> 
> one did
> 
>   ...
>   ln -s /proc/mounts /etc/mtab.NEW
>   mv -f /etc/mtab.NEW /etc/mtab
>   ...
> 
> what would happen?

I can guess that maybe there is a race condition in between the original "rm" and "ln" steps, such that if something else on the system happens to invoke libmount during this time causes the creation of the mtab file if it doesn't already exist (according to comment #44).

If that is the case, then this should eliminate the race condition:

ln -snf /proc/mounts /etc/mtab
Comment 51 Michael Schwendt 2013-02-08 16:30:02 EST
"ln -snf" is not atomic, as it first unlink()s the target then creates the symlink.

> a race condition in between the original "rm" and "ln" steps

Then ln would complain, because something else has created the target already.
Comment 52 Orion Poplawski 2013-02-08 19:02:07 EST
The missing /etc/mtab symlink I believe was a problem with rpm-4.10.2-2.  If util-linux was upgraded with that installed, the link went away.  4.10.2-1 and 4.10.3-1 seem okay
Comment 53 Harald Reindl 2013-02-09 14:38:09 EST
*shrug* 

that should be investigated and maybe catched with any sort of autotests because we are speaking here about that absolute core-packages of the distribution
Comment 54 Ondrej Vasik 2013-02-14 16:00:24 EST
*** Bug 910737 has been marked as a duplicate of this bug. ***
Comment 55 Fedora Update System 2013-02-19 10:47:51 EST
util-linux-2.22.2-5.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/util-linux-2.22.2-5.fc18
Comment 56 Bas Mevissen 2013-02-27 05:59:38 EST
Running util-linux-2.22.2-6.fc18.x86_64, for some reason my /etc/mtab was a real file with incomplete information and not a link to /proc/mounts

df and mount now seem to show the correct information. At least, it is much more than before.

System is a clean install of Fedora 18 (after being "fedup" with a failed upgrade...). So I would suggest to add some forced fixup for /etc/mtab in a future update to correct the problem for everyone. Maybe add a warning that is displayed during an util-linux update.
Comment 57 Shawn Starr 2013-04-09 04:42:11 EDT
This isn't an issue anymore for me with: util-linux-2.22.2-6.fc18.x86_64

$ df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 3.9G     0  3.9G   0% /dev
tmpfs                    3.9G  704K  3.9G   1% /dev/shm
tmpfs                    3.9G  3.3M  3.9G   1% /run
tmpfs                    3.9G     0  3.9G   0% /sys/fs/cgroup
======= /dev/mapper/fedora-root   74G   52G   19G  74% /
tmpfs                    3.9G   72K  3.9G   1% /tmp
/dev/sda2                477M   97M  356M  22% /boot
/dev/mapper/fedora-home   24G   18G  4.9G  79% /home

Note: I reinstalled and did a Fresh Fedora 18 w/ updates and see no issues.