Bug 154383

Summary: df takes a very long time to display GFS mounted filesystems
Product: [Retired] Red Hat Cluster Suite Reporter: Martijn Brizee <m.brizee>
Component: gfsAssignee: Ken Preslan <kpreslan>
Status: CLOSED NOTABUG QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 3CC: riel
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2005-07-18 20:10:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Martijn Brizee 2005-04-11 09:39:37 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

Description of problem:
I have 2 systems connected to the same SCSI diskarray. The array has an internal RAID controller. There are 2 partitions configured on the array. An RAID 0 set of about 2 TB and an RAID 5 set of about 1,5 TB. RAID 0 is sda; RAID 5 is sdb. Both arrays are imported with pool. Pool0 is sda and Pool1 is sdb.

When I do a df after mounting the GFS filesystems it takes a long time for df to to show the statistics for the GFS partitions.

A strace of the 'hanging' df gives:

[root@zeus root]# strace -p 21641
Process 21641 attached - interrupt to quit
statfs("/mnt/raid0",

When df finally ends the strace looks like this:
[root@zeus root]# strace -p 21641
Process 21641 attached - interrupt to quit
statfs("/mnt/raid0", {f_type=0x1161970, f_bsize=4096, f_blocks=487675972, f_bfree=487637206, f_bavail=487637206, f_files=487638413, f_ffree=487637206, f_fsid={0, 0}, f_namelen=255, f_frsize=0}) = 0
stat("/mnt/raid0", {st_mode=S_IFDIR|0755, st_size=3864, ...}) = 0
open("/proc/mounts", O_RDONLY)          = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a9566e000
read(3, "rootfs / rootfs rw 0 0\n/dev/root"..., 1024) = 453
stat("/mnt/raid0", {st_mode=S_IFDIR|0755, st_size=3864, ...}) = 0
close(3)                                = 0
munmap(0x2a9566e000, 4096)              = 0
write(1, "/dev/pool/pool1      1950703888 "..., 69) = 69
statfs("/mnt/raid5", {f_type=0x1161970, f_bsize=4096, f_blocks=390062136, f_bfree=389805476, f_bavail=389805476, f_files=389805482, f_ffree=389805476, f_fsid={0, 0}, f_namelen=255, f_frsize=0}) = 0
stat("/mnt/raid5", {st_mode=S_IFDIR|0755, st_size=3864, ...}) = 0
open("/proc/mounts", O_RDONLY)          = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a9566e000
read(3, "rootfs / rootfs rw 0 0\n/dev/root"..., 1024) = 453
stat("/mnt/raid5", {st_mode=S_IFDIR|0755, st_size=3864, ...}) = 0
close(3)                                = 0
munmap(0x2a9566e000, 4096)              = 0
write(1, "/dev/pool/pool0      1560248544 "..., 69) = 69
munmap(0x2a9566d000, 4096)              = 0
exit_group(0)                           = ?
Process 21641 detached

Typeing df again after the first df has ended shows results on a normal speed however.

Umounting -> mounting the partitions reproduces the problem above.



Version-Release number of selected component (if applicable):
coreutils-4.5.3-26, GFS-6.0.2-17

How reproducible:
Always

Steps to Reproduce:
1. configure GFS and pool
2. mount GFS partition
3. do df in /
  

Actual Results:  df took long to display results

Expected Results:  df should display results immediately

Additional info:

Comment 1 Rik van Riel 2005-04-11 11:32:22 UTC
Reassigning to GFS maintainer.

Comment 2 Ken Preslan 2005-04-12 09:58:20 UTC
A local filesystem keeps all its statfs() info in one centralized place.  A df
just has to read the information from that one place to report it -- it's nice
and fast.  If GFS did this, it would be come a scalability bottleneck -- every
allocation or deallocation would have to change that one set of counters.  GFS
gets around this by distributing the statfs information across the filesystem. 
Each Resource Group (allocation container unit) has its own set of counters. 
When you do a statfs, GFS has to read in each RG and add together all the
individual counters to get the whole FS' statistics.  To make this faster, GFS
keeps a cache of the per-RG statistics, so df doesn't always need to read off
the disk.  The first df after a mount initializes this cache.

So, what you saw is expected.  The first df is slow as it initializes the cache.
 The second one is faster as it uses the cached data.

If it really bugs you:  When you make the filesystem, you can use the -r option
(e.g. "-r 1024" or "-r 1536) to gfs_mkfs to make the RGs bigger.  You'll have
fewer RGs, so reading in the headers to initialize the cache will be faster.