Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1265811 - oo-accept-node reports a quota failures when a loop device is used.
oo-accept-node reports a quota failures when a loop device is used.
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Containers (Show other bugs)
2.0.0
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Miciah Dashiel Butler Masters
chaoyang
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-23 16:11 EDT by Eric Rich
Modified: 2016-03-22 12:51 EDT (History)
7 users (show)

See Also:
Fixed In Version: openshift-origin-node-util-1.38.6.2-1.el6op
Doc Type: Bug Fix
Doc Text:
Cause: In order to check that filesystem quotas were enabled, the oo-accept-node tool used the output of the df command to determine the device and mountpoint of the gear base directory. oo-accept-node then specified this mountpoint to the repquota command and expected to see the device in repquota's output. However, if the gear base directory is a loop mount, df prints the loop image filename whereas repquota prints the loop device. Thus oo-accept-node was looking for the wrong string in the repquota output. Consequence: If the gear base directory was a loop mount, oo-accept-node printed a spurious error indicating that quotas were disabled for the filesystem even if they were enabled. Fix: oo-accept-node now uses the mount command to determine whether the filesystem is a loop mount and, if it is, oo-accept-node now parses the filesystem options to determine the loop device for the filesystem. In addition, because using a loop mount degrades performance, oo-accept-node will print a warning if the gear base directory is a loop mount. Result: oo-accept-node now will print an error only if quotas really are disabled on the filesystem, and oo-accept-node will print a warning if the filesystem is a loop mount.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-22 12:51:19 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:0489 normal SHIPPED_LIVE Important: Red Hat OpenShift Enterprise 2.2.9 security, bug fix, and enhancement update 2016-03-22 16:49:04 EDT

  None (edit)
Description Eric Rich 2015-09-23 16:11:43 EDT
Description of problem:

When running [0] you fail to properly get the loop device path, and as such you get the following string (output):  

    [root@node2 openshift]# /sbin/quotaon -u -p /opt/openshift/test.fs
    quotaon: Specified path /opt/openshift/node.ext4 is not directory nor device.
    quotaon: No correct mountpoint specified.

The information used to populate, this output comes from [1] and more or less grabs '/opt/openshift/test.fs' for the oo_mount,  However this is not the mount device. The actual device is /dev/loop0. In short grabbing the output shown above results in a "string" miss match and a false positive reporting, from oo-accept-node. 

You can see from other commands (using the right device) that the correct string is reported. 

    [root@node2 openshift]# /sbin/quotaon -u -p /var/lib/openshift/
    user quota on /var/lib/openshift (/dev/loop0) is on

Again the issue here seems to be with using the command (below) to get the device (for loop back devices). 

    [root@node2 openshift]# /bin/df -P /var/lib/openshift/ | /usr/bin/tail -1
    /opt/openshift/test.fs      999112  2836    943848       1% /var/lib/openshift

Version-Release number of selected component (if applicable): 2.0 -> 2.2.6
How reproducible: 100% 

Steps to Reproduce:

# truncate -s 1G /opt/openshift/test.fs
# mkfs -t ext4 /opt/openshift/test.fs
# mount -oloop,usrquota /opt/openshift/test.fs /var/lib/openshift
# oo-accept-node

Actual results:
    FAIL: quotas are not enabled on /var/lib/openshift (/opt/openshift/node.ext4)

Expected results:
    oo-accept-node should pass   

Additional info:

[0] https://github.com/openshift/origin-server/blob/master/node-util/sbin/oo-accept-node#L549-L552
[1] https://github.com/openshift/origin-server/blob/master/node-util/sbin/oo-accept-node#L540-L552
Comment 4 Miciah Dashiel Butler Masters 2016-02-08 17:58:11 EST
PR: https://github.com/openshift/origin-server/pull/6350
Comment 5 openshift-github-bot 2016-02-09 12:19:33 EST
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/afe315643abeb4463ca57e043e1a8037f99b111a
oo-accept-node: check_quotas: cope with loop mount

Modify oo-accept-node's quota check to check whether the mount is using
a loop device mount and, if so, check the quota on the loop device.  In
addition, print a warning that using a loop mount may impact performance
negatively.

This commit fixes bug 1265811.

https://bugzilla.redhat.com/show_bug.cgi?id=1265811
Comment 8 Rory Thrasher 2016-02-22 17:13:28 EST
QE,

Can we verify that oo-accept-node will now pass based on the original reproduction steps?

# truncate -s 1G /opt/openshift/test.fs
# mkfs -t ext4 /opt/openshift/test.fs
# mount -oloop,usrquota /opt/openshift/test.fs /var/lib/openshift
# oo-accept-node

Puddle: http://etherpad.corp.redhat.com/puddle-2-2-2016-02-19


Thank you
Comment 9 Anping Li 2016-02-29 04:48:50 EST
Reported as 'quotas are not enabled' by oo-accept-node. but step 2)&4) should be quota is turn on.

1) [root@node1 ~]# mount
/opt/openshift/test.fs on /var/lib/openshift type ext4 (rw,usrquota,loop=/dev/loop0)


2) [root@node1 ~]# /sbin/quotaon -u -p /var/lib/openshift/
user quota on /var/lib/openshift (/dev/loop0) is on

3) [root@node1 ~]# oo-accept-node
WARNING: /var/lib/openshift is a loop mount (loop device: /dev/loop0)
).  Using a loop mount may reduce performance.
FAIL: quotas are not enabled on /var/lib/openshift (/dev/loop0)
)
1 ERRORS


4) [root@node1 ~]# /sbin/quotaon -u -p /var/lib/openshift/
user quota on /var/lib/openshift (/dev/loop0) is on

5) [root@node1 ~]# rpm -qf /usr/sbin/oo-accept-node
openshift-origin-node-util-1.38.6.1-1.el6op.noarch
Comment 10 Anping Li 2016-02-29 06:05:44 EST
If the mount show loop0 as the suffix as 2), we will hit this issue.

1) /opt/openshift/test.fs on /var/lib/openshift type ext4 (rw,loop=/dev/loop0,usrquota)
2) /opt/openshift/test.fs on /var/lib/openshift type ext4 (rw,usrquota,loop=/dev/loop0)

I add one line in /etc/fstab, and run "mount -o remount /var/lib/openshift", and got the message as 2).

# tail -1 /etc/fstab 
/opt/openshift/test.fs /var/lib/openshift   ext4    defaults,usrquota     1 0
Comment 11 openshift-github-bot 2016-03-01 12:01:49 EST
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/896aba36d358cda1bd7b1a5d229704d995c72d4d
oo-accept-node: check-quotas: fix mnt opts parsing

Improve the parsing of mount options from the `mount` command's output by
properly splitting on newlines and then removing the parentheses around the
mount options.

Before this commit, the pattern match `/loop=([^,]+)/` on the mount options
was capturing the right parenthesis and newline from the `mount` command's
output if the loop option was the last mount option.

This commit is related to bug 1265811.

https://bugzilla.redhat.com/show_bug.cgi?id=1265811
Comment 12 Rory Thrasher 2016-03-03 17:39:02 EST
QE,

This should be fixed now.  Can we verify with the latest puddle? Thank you.

Puddle: http://etherpad.corp.redhat.com/puddle-2-2-2016-02-19
Comment 13 Johnny Liu 2016-03-03 22:35:29 EST
Verified this bug with, and PASS.


# truncate -s 1G /opt/openshift/test.fs
# mkfs -t ext4 /opt/openshift/test.fs
# mount -oloop,usrquota /opt/openshift/test.fs /var/lib/openshift
# mount |grep openshift
/opt/openshift/test.fs on /var/lib/openshift type ext4 (rw,loop=/dev/loop1,usrquota)
# quotaon /var/lib/openshift/
# /sbin/quotaon -u -p /var/lib/openshift/
user quota on /var/lib/openshift (/dev/loop1) is on
# oo-accept-node
WARNING: /var/lib/openshift is a loop mount (loop device: /dev/loop1).  Using a loop mount may reduce performance.
PASS


# truncate -s 1G /opt/openshift/test.fs
# mkfs -t ext4 /opt/openshift/test.fs
# mount -oloop,usrquota /opt/openshift/test.fs /var/lib/openshift
# mount |grep openshift
/opt/openshift/test.fs on /var/lib/openshift type ext4 (rw,loop=/dev/loop1,usrquota)
Add one line to /etc/fstab
/opt/openshift/test.fs /var/lib/openshift   ext4    defaults,usrquota     1 0
# mount -o remount /var/lib/openshift
# mount |grep openshift
/opt/openshift/test.fs on /var/lib/openshift type ext4 (rw,usrquota,loop=/dev/loop1)
# quotaon /var/lib/openshift/
# /sbin/quotaon -u -p /var/lib/openshift/
user quota on /var/lib/openshift (/dev/loop1) is on
# oo-accept-node 
WARNING: /var/lib/openshift is a loop mount (loop device: /dev/loop1).  Using a loop mount may reduce performance.
PASS
Comment 15 errata-xmlrpc 2016-03-22 12:51:19 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-0489.html

Note You need to log in before you can comment on or make changes to this bug.