RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 836200 - [xfs/xfstests 219] Not all quotas are reported by repquota on XFS
Summary: [xfs/xfstests 219] Not all quotas are reported by repquota on XFS
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: quota
Version: 7.0
Hardware: All
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Petr Pisar
QA Contact: qe-baseos-daemons
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-06-28 10:38 UTC by Boris Ranto
Modified: 2016-06-20 00:38 UTC (History)
5 users (show)

Fixed In Version: quota-4.00-4.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 837341 (view as bug list)
Environment:
Last Closed: 2012-07-03 14:53:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Boris Ranto 2012-06-28 10:38:56 UTC
Description of problem:
Running xfstests test case no. 219, the test fails with 'Bad number of inodes used (type=g)'. This happens because the repquota does not report any group quotas (no #$gid line in the output of repquota, it contains #0 line only).

Version-Release number of selected component (if applicable):
kernel-3.3.0-0.14.el7.x86_64
xfsprogs-3.1.8-1.el7.x86_64
quota-4.00-3.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Run the xfstests test case no. 219
  
Actual results:
The test failure because the repquota does not report the group quota.

Expected results:
The test passes.

Additional info:
This does not happen with xfs on rhel6 nor with ext4 on rhel7.

Comment 1 Eric Sandeen 2012-06-28 17:59:07 UTC
Newer quota repquota isn't reporting it - I guess this is likely to be a quota issue?  quota 3.17 is ok, 4.0 is not.  Output diff:

[root@inode xfstests-dev]# /bin/mount -t xfs -o grpquota,context=system_u:object_r:nfs_t:s0 /dev/sdb6 /mnt/scratch

3.7:

[root@inode xfstests-dev]# repquota -g -n /mnt/scratch/
*** Report for group quotas on device /dev/sdb6
Block grace time: 7days; Inode grace time: 7days
                        Block limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
#0        --       0       0       0              3     0     0       
#2        --     144       0       0              3     0     0       


4.0:

[root@inode xfstests-dev]# repquota.new -g -n /mnt/scratch/
*** Report for group quotas on device /dev/sdb6
Block grace time: 7days; Inode grace time: 7days
                        Block limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
#0        --       0       0       0              3     0     0

Comment 2 Eric Sandeen 2012-06-28 20:17:54 UTC
Looks like this is needed:

From: Jan Kara <jack>
Date: Fri, 8 Jun 2012 09:11:20 +0000 (+0200)
Subject: repquota: Fix reporting for XFS
X-Git-Url: http://linuxquota.git.sourceforge.net/git/gitweb.cgi?p=linuxquota%2Flinuxquota;a=commitdiff_plain;h=6ba6546dd167297cb9ed69d0257ee245b0faea47

repquota: Fix reporting for XFS

Conversion to generic quota scanning introduced a bug for XFS where we
stopped scanning after quotactl reported first error. quotactl for XFS
however reports ENOENT when it has nothing to report for a particular user
/ group and we shouldn't stop scanning after that. We tried to test for this
but the test was wrong. Fix it.

Signed-off-by: Jan Kara <jack>

Comment 3 Dave Chinner 2012-07-03 04:19:30 UTC
(In reply to comment #2)
> Looks like this is needed:
> 
> From: Jan Kara <jack>
> Date: Fri, 8 Jun 2012 09:11:20 +0000 (+0200)
> Subject: repquota: Fix reporting for XFS
> X-Git-Url:
> http://linuxquota.git.sourceforge.net/git/gitweb.
> cgi?p=linuxquota%2Flinuxquota;a=commitdiff_plain;
> h=6ba6546dd167297cb9ed69d0257ee245b0faea47
> 
> repquota: Fix reporting for XFS
> 
> Conversion to generic quota scanning introduced a bug for XFS where we
> stopped scanning after quotactl reported first error. quotactl for XFS
> however reports ENOENT when it has nothing to report for a particular user
> / group and we shouldn't stop scanning after that. We tried to test for this
> but the test was wrong. Fix it.
> 
> Signed-off-by: Jan Kara <jack>

Yeah, I reported this problem to Jan when I upgraded a test box about three weeks ago. The above commit fixes the problem, so upgrading the quota tools package should fix this.

Comment 4 Petr Pisar 2012-07-03 14:41:42 UTC
RHEL-7 is affected only.

Comment 6 Jason Tibbitts 2016-02-09 02:08:05 UTC
Could someone verify that this bug didn't creep back into RHEL7?

On my Centos 7.2 machine with quota-4.01-11.el7.x86_64, repquota -a only shows root even though I have a couple of hundred quotas set for each filesystem.  xfs_quota -x -c report -u -b -h -t -a -L1000 -U99999 reports what I would expect, but is a good bit less convenient to use because it doesn't work with any of my existing tooling.

Comment 7 Jason Tibbitts 2016-02-09 02:13:34 UTC
And just for added info, I built the current rawhide package (4.03-2) for EL7 and installed it, and it has the same behavior (repquota only shows quotas for for root).  It appears to work fine if the filesystem is ext4, just not xfs.

Comment 8 Eric Sandeen 2016-02-09 03:18:56 UTC
Did it work before, and something changed?  Or has it never worked for you?

Most importantly: Are you using something like sssd + LDAP?

What you report sounds like the issue where quota is unable to use getpwent to enumerate users due to the configuration of your user database; we have plans to fix that, and a bug is filed: Bug 1164652 - xfs_quota doesn't report LDAP users if sssd is used. - although unfortunately it is a private bug.

-Eric

Comment 9 Jason Tibbitts 2016-02-09 03:45:55 UTC
Well, this is the first time I've tried using xfs in my usual file server configuration.  Previously I've used ext4, so I can't say much for the timeline.

I am using sssd and LDAP, yes.  I guess it's possible, but I'd be pretty surprised of the behavior of repquota with sssd/LDAP changed depending on the underlying filesystem.

To test, I created a new ext4 volume.  I then used rsync to copy from the xfs volume to the ext4 volume.  I would expect the output of repquota on each filesystem to be quite similar.  Yet:

$ repquota -C /local/test /export/grad-08

*** Report for user quotas on device /dev/mapper/nas-test
Block grace time: 7days; Inode grace time: 7days
                        Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --      20       0       0              2     0     0       
ali       --   11404       0       0            808     0     0       
amajzun   --    5104       0       0            549     0     0       
brpahari  -- 2290624       0       0           8874     0     0       
[...]


*** Report for user quotas on device /dev/mapper/nas-grad--08
Block grace time: 7days; Inode grace time: 7days
                        Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --       4       0       0              3     0     0


/local/test is the ext4 volume; /nas/grad-08 is the xfs volume.  They have identical contents.  Also note that the -C option is required for repquote to look up uids.  But even if I don't pass it, I get numeric uids for the test volume but still no useful output for the second.  Will play with strace a bit and see if it tells me anything useful.

Comment 10 Eric Sandeen 2016-02-09 03:50:03 UTC
(In reply to Jason Tibbitts from comment #9)

> I am using sssd and LDAP, yes.  I guess it's possible, but I'd be pretty
> surprised of the behavior of repquota with sssd/LDAP changed depending on
> the underlying filesystem.

It does.  ext4's quota file is visible to userspace, and the quota tools can directly parse the file to find out what IDs have active quotas.

xfs's quota inode is hidden, and today all we can do is iterate over all users, asking the kernel for quota information one ID at a time.

I have patches to allow the kernel to do the "smart parsing" to speed this up, and get rid of the getpwent() loop that's used today, so that we don't have to enumerate all users on the system.

Honestly; I know what causes this behavior, and it's getting fixed upstream, and then will be fixed in RHEL7.

Comment 11 Eric Sandeen 2016-02-09 03:52:43 UTC
The reason "xfs_quota -x -c report -u -b -h -t -a -L1000 -U99999" works is because it doesn't do user enumeration, but just queries every UID between the limits.  There are also patches upstream to allow this to look up names to go with the IDs.

https://git.kernel.org/cgit/fs/xfs/xfsprogs-dev.git/commit/?h=for-next&id=85dcd9a09855d8423863806c70a9e11887f054ad

Comment 12 Jason Tibbitts 2016-02-09 03:57:28 UTC
On either filesystem, repquota seems to call quotactl on every mounted filesystem for some reason, so I'll ignore that.  On ext4, here's the meat:

quotactl(Q_GETINFO|USRQUOTA, "/dev/mapper/nas-test", 0, {bgrace=604800, igrace=604800, flags=0, valid=IIF_BGRACE|IIF_IGRACE|IIF_FLAGS}) = 0
quotactl(Q_GETINFO|GRPQUOTA, "/dev/mapper/nas-test", 0, 0x7ffdf39fd8a0) = -1 ESRCH (No such process)
lstat("/local", {st_mode=S_IFDIR|0755, st_size=17, ...}) = 0
lstat("/local/test", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
statfs("/local/test", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=12868728, f_bfree=9716621, f_bavail=9702525, f_files=3276800, f_ffree=3210371, f_fsid={-1062176525, -1456201798}, f_namelen=255, f_frsize=4096}) = 0
stat("/dev/mapper/nas-test", {st_mode=S_IFBLK|0660, st_rdev=makedev(253, 19), ...}) = 0
stat("/local/test", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0x7fdf4c985000, 4096)            = 0
stat("/local/test", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/local", {st_mode=S_IFDIR|0755, st_size=17, ...}) = 0
lstat("/local/test", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/dev/mapper/nas-test", {st_mode=S_IFBLK|0660, st_rdev=makedev(253, 19), ...}) = 0
quotactl(Q_GETFMT|USRQUOTA, "/dev/mapper/nas-test", 0, {0x4 /* QFMT_VFS_??? */}) = 0
open("/local/test/aquota.user", O_RDONLY) = 3
lseek(3, 0, SEEK_SET)                   = 0
read(3, "\21\37\300\331\1\0\0\0", 8)    = 8
close(3)                                = 0
quotactl(Q_SYNC|USRQUOTA, "/dev/mapper/nas-test", 0, NULL) = 0
open("/local/test/aquota.user", O_RDONLY) = 3
flock(3, LOCK_SH)                       = 0
quotactl(Q_GETINFO|USRQUOTA, "/dev/mapper/nas-test", 0, {bgrace=604800, igrace=604800, flags=0, valid=IIF_BGRACE|IIF_IGRACE|IIF_FLAGS}) = 0
lseek(3, 0, SEEK_SET)                   = 0
read(3, "\21\37\300\331\1\0\0\0", 8)    = 8
lseek(3, 8, SEEK_SET)                   = 8
read(3, "\200:\t\0\200:\t\0\0\0\0\0\t\0\0\0\0\0\0\0\10\0\0\0", 24) = 24

At this point it starts opening locale files and prints its report and exits.


On xfs, it has significantly different behavior:

stat("/export/grad-08", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/export", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/export/grad-08", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/dev/mapper/nas-grad--08", {st_mode=S_IFBLK|0660, st_rdev=makedev(253, 11), ...}) = 0
quotactl(Q_XGETQSTAT|USRQUOTA, "/dev/mapper/nas-grad--08", 0, {version=1, ...}) = 0

Then it opens the locale files and prints the report header.  Then it opens /etc/passwd, then calls quotactl a bunch of times with ENOTENT the result for most of them except for root.

So, yeah, I guess it does have completely different behavior depending on the underlying filesystem, which is... odd.  I'll have to dig into the source, but I guess repquota doesn't know how to ask xfs for a count per user without knowing the users in advance.  Which is... unfortunate.

Comment 13 Jason Tibbitts 2016-02-09 04:02:03 UTC
OK, so, yeah, I guess I'm kind of stuck.  I kind of liked the idea of xfs because it has a proper backup utility, but I guess I'll need to evaluate which functionality I need more.

Anyway, even though the symptoms seem to be exactly the same as the actual topic of this bug, the underlying cause is completely different.  Sorry for the spam.

Comment 14 Eric Sandeen 2016-02-09 13:47:55 UTC
No worries.

Like I said, this should be fixed in the next release of RHEL7.

In the meantime, if your user database is not huge, you could turn enumeration back on in sssd as a short term workaround on xfs.

Comment 15 Eric Sandeen 2016-02-09 13:49:44 UTC
For reference, http://oss.sgi.com/archives/xfs/2016-01/msg00573.html

Comment 19 Jason Tibbitts 2016-02-09 20:12:15 UTC
OK, cool.  What I've decided to do is bypass the command line utilities completely and just use python and the ctypes module to make the quotactl syscalls directly.  I'm surprised there aren't existing utilities for generating quota information that's actually intended to be parseable, though things have gotten a little better more recently.

Comment 20 Eric Sandeen 2016-02-09 20:13:32 UTC
The "-f file" options on some of the xfs_quota command might help you there.

Comment 21 Petr Pisar 2016-02-10 11:09:58 UTC
repquota became --output option in 4.02 version. It allows to export quota  information in XML and CSV. However, RHEL 7 has older version without this feature.


Note You need to log in before you can comment on or make changes to this bug.