Bug 154383 - df takes a very long time to display GFS mounted filesystems
df takes a very long time to display GFS mounted filesystems
Status: CLOSED NOTABUG
Product: Red Hat Cluster Suite
Classification: Red Hat
Component: gfs (Show other bugs)
3
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Ken Preslan
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-04-11 05:39 EDT by Martijn Brizee
Modified: 2010-01-11 22:04 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-07-18 16:10:12 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Martijn Brizee 2005-04-11 05:39:37 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

Description of problem:
I have 2 systems connected to the same SCSI diskarray. The array has an internal RAID controller. There are 2 partitions configured on the array. An RAID 0 set of about 2 TB and an RAID 5 set of about 1,5 TB. RAID 0 is sda; RAID 5 is sdb. Both arrays are imported with pool. Pool0 is sda and Pool1 is sdb.

When I do a df after mounting the GFS filesystems it takes a long time for df to to show the statistics for the GFS partitions.

A strace of the 'hanging' df gives:

[root@zeus root]# strace -p 21641
Process 21641 attached - interrupt to quit
statfs("/mnt/raid0",

When df finally ends the strace looks like this:
[root@zeus root]# strace -p 21641
Process 21641 attached - interrupt to quit
statfs("/mnt/raid0", {f_type=0x1161970, f_bsize=4096, f_blocks=487675972, f_bfree=487637206, f_bavail=487637206, f_files=487638413, f_ffree=487637206, f_fsid={0, 0}, f_namelen=255, f_frsize=0}) = 0
stat("/mnt/raid0", {st_mode=S_IFDIR|0755, st_size=3864, ...}) = 0
open("/proc/mounts", O_RDONLY)          = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a9566e000
read(3, "rootfs / rootfs rw 0 0\n/dev/root"..., 1024) = 453
stat("/mnt/raid0", {st_mode=S_IFDIR|0755, st_size=3864, ...}) = 0
close(3)                                = 0
munmap(0x2a9566e000, 4096)              = 0
write(1, "/dev/pool/pool1      1950703888 "..., 69) = 69
statfs("/mnt/raid5", {f_type=0x1161970, f_bsize=4096, f_blocks=390062136, f_bfree=389805476, f_bavail=389805476, f_files=389805482, f_ffree=389805476, f_fsid={0, 0}, f_namelen=255, f_frsize=0}) = 0
stat("/mnt/raid5", {st_mode=S_IFDIR|0755, st_size=3864, ...}) = 0
open("/proc/mounts", O_RDONLY)          = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a9566e000
read(3, "rootfs / rootfs rw 0 0\n/dev/root"..., 1024) = 453
stat("/mnt/raid5", {st_mode=S_IFDIR|0755, st_size=3864, ...}) = 0
close(3)                                = 0
munmap(0x2a9566e000, 4096)              = 0
write(1, "/dev/pool/pool0      1560248544 "..., 69) = 69
munmap(0x2a9566d000, 4096)              = 0
exit_group(0)                           = ?
Process 21641 detached

Typeing df again after the first df has ended shows results on a normal speed however.

Umounting -> mounting the partitions reproduces the problem above.



Version-Release number of selected component (if applicable):
coreutils-4.5.3-26, GFS-6.0.2-17

How reproducible:
Always

Steps to Reproduce:
1. configure GFS and pool
2. mount GFS partition
3. do df in /
  

Actual Results:  df took long to display results

Expected Results:  df should display results immediately

Additional info:
Comment 1 Rik van Riel 2005-04-11 07:32:22 EDT
Reassigning to GFS maintainer.
Comment 2 Ken Preslan 2005-04-12 05:58:20 EDT
A local filesystem keeps all its statfs() info in one centralized place.  A df
just has to read the information from that one place to report it -- it's nice
and fast.  If GFS did this, it would be come a scalability bottleneck -- every
allocation or deallocation would have to change that one set of counters.  GFS
gets around this by distributing the statfs information across the filesystem. 
Each Resource Group (allocation container unit) has its own set of counters. 
When you do a statfs, GFS has to read in each RG and add together all the
individual counters to get the whole FS' statistics.  To make this faster, GFS
keeps a cache of the per-RG statistics, so df doesn't always need to read off
the disk.  The first df after a mount initializes this cache.

So, what you saw is expected.  The first df is slow as it initializes the cache.
 The second one is faster as it uses the cached data.

If it really bugs you:  When you make the filesystem, you can use the -r option
(e.g. "-r 1024" or "-r 1536) to gfs_mkfs to make the RGs bigger.  You'll have
fewer RGs, so reading in the headers to initialize the cache will be faster.

Note You need to log in before you can comment on or make changes to this bug.