Bug 1612058
| Summary: | Arbiter brick can contain less files than expected when we specify average file size explicitly | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Valerii Ponomarov <vponomar> |
| Component: | heketi | Assignee: | Raghavendra Talur <rtalur> |
| Status: | CLOSED ERRATA | QA Contact: | Valerii Ponomarov <vponomar> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | cns-3.10 | CC: | akrishna, hchiramm, jmulligan, kramdoss, madam, pprakash, rhs-bugs, rtalur, sankarshan, storage-qa-internal, vinug |
| Target Milestone: | --- | ||
| Target Release: | CNS 3.10 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Heketi passed incorrect options to the XFS format command when creating bricks. This meant that inode tables on arbiter bricks were being sized as though they were for data bricks, and therefore reserved a large number of inodes for data that would never be written to the arbiter brick. Heketi now passes the correct options and the inode tables of arbiter bricks are sized to ensure they can hold metadata for the rest of the volume or volume set.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-09-12 09:23:49 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1568862 | ||
|
Description
Valerii Ponomarov
2018-08-03 11:30:06 UTC
I am working on this. Analysis: when a 13G volume with average size of 500 KB is created: we get an arbiter calculation of: bricksize: 13631488, filesize: 500, arbiterbricksize: 27262 mkfs command: mkfs.xfs -i size=512 -n size=8192 /dev/mapper/vg_8b7da1a300b44823878dfab9c4112224-brick_bbc42f67c168b1b8d1e6f9b176088491 Hence, with each file taking up 1K in inode block, we should be able to create 27262 files. However, the stat info is File: ‘/var/lib/heketi/mounts/vg_8b7da1a300b44823878dfab9c4112224/brick_bbc42f67c168b1b8d1e6f9b176088491’ Size: 19 Blocks: 0 IO Block: 4096 directory Device: fc04h/64516d Inode: 14336 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Context: system_u:object_r:unlabeled_t:s0 Access: 1970-01-01 05:30:00.000000000 +0530 Modify: 2018-08-08 23:03:07.302437334 +0530 Change: 2018-08-08 23:03:07.302437334 +0530 Birth: - We can see only 14336 inodes available. Formatting the space partition with xfs inode option of maxpct=100 gives us about 38K inodes. 38K being greater than 27262 satisfies our requirement. We get more inodes than the calculation in heketi because in heketi we also accounted for blocks needed by those 27262 files' extended attributes. To test the same, I create a arbiter volume and created a file. Then I copied over the file to xfs partition with maxpct=100, 27262 times. This test worked. Other tests performed: I added a extended attribute to all files using setfattr -n user.rtalur1 -v a10byteval I added another set of ea to all files using setfattr -n user.rtalur2 -v a10byteval At this point, we still have about 75% of the partition free. I think this is a satisfactory test. Hence, I will create a patch which changes xfs partition options for arbiter brick. (In reply to Raghavendra Talur from comment #3) > Analysis: > > when a 13G volume with average size of 500 KB is created: > we get an arbiter calculation of: bricksize: 13631488, filesize: 500, > arbiterbricksize: 27262 > mkfs command: mkfs.xfs -i size=512 -n size=8192 > /dev/mapper/vg_8b7da1a300b44823878dfab9c4112224- > brick_bbc42f67c168b1b8d1e6f9b176088491 > > Hence, with each file taking up 1K in inode block, we should be able to > create 27262 files. > However, the stat info is > > File: > ‘/var/lib/heketi/mounts/vg_8b7da1a300b44823878dfab9c4112224/ > brick_bbc42f67c168b1b8d1e6f9b176088491’ > Size: 19 Blocks: 0 IO Block: 4096 directory > Device: fc04h/64516d Inode: 14336 Links: 3 > Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) > Context: system_u:object_r:unlabeled_t:s0 > Access: 1970-01-01 05:30:00.000000000 +0530 > Modify: 2018-08-08 23:03:07.302437334 +0530 > Change: 2018-08-08 23:03:07.302437334 +0530 > Birth: - > > > We can see only 14336 inodes available. I should have pasted the df output, not stat. Below is df output showing about 14K inodes. xfs 14K 26 14K 1% 22M 1.6M 20M 8% - > > > Formatting the space partition with xfs inode option of maxpct=100 gives us > about 38K inodes. 38K being greater than 27262 satisfies our requirement. > > We get more inodes than the calculation in heketi because in heketi we also > accounted for blocks needed by those 27262 files' extended attributes. To > test the same, I create a arbiter volume and created a file. Then I copied > over the file to xfs partition with maxpct=100, 27262 times. This test > worked. > > Other tests performed: > I added a extended attribute to all files using setfattr -n user.rtalur1 -v > a10byteval > I added another set of ea to all files using setfattr -n user.rtalur2 -v > a10byteval > > At this point, we still have about 75% of the partition free. I think this > is a satisfactory test. > > Hence, I will create a patch which changes xfs partition options for arbiter > brick. Patch posted at https://github.com/heketi/heketi/pull/1306 Verified it using "rhgs3/rhgs-volmanager-rhel7:3.4.0-3". Where we can create amount of files on arbiter bricks equal to "volume size / avg file size". Updated doc text in the Doc Text field. Please review for technical accuracy. Updated doc text in the Doc Text field. Please review for technical accuracy. Doc Text looks OK Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:2686 |