Bug 1484460 - [RHEL7] [GSS]high latency observed in entrylk operation that cause application experiencing high latency
[RHEL7] [GSS]high latency observed in entrylk operation that cause applicatio...
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: replicate (Show other bugs)
3.2
Unspecified Unspecified
high Severity high
: ---
: RHGS 3.2.0 Async
Assigned To: Ravishankar N
nchilaka
: ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-08-23 12:06 EDT by Atin Mukherjee
Modified: 2017-09-14 04:38 EDT (History)
18 users (show)

See Also:
Fixed In Version: glusterfs-3.8.4-18.7
Doc Type: Bug Fix
Doc Text:
When the replication process creates an entry as part of self-heal, it uses the iatt buffer to send the create operation. Previously, entries were created regardless of iatt buffer validity. This resulted in an empty file with GFID xattr of all zeroes being created when an invalid iatt buffer was sent. Checks are now included to ensure that iatt buffer content is valid before creating the entry in this situation.
Story Points: ---
Clone Of: 1474178
: 1484650 (view as bug list)
Environment:
Last Closed: 2017-09-14 04:38:53 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Comment 11 Bipin Kunal 2017-09-12 09:23:01 EDT
Description of problem:

Application is seeing very high load while accessing gluster volume.The load is mainly based on the response time from gluster for the application's requests.The moment load crosses a threshold level (70-75% in their calculation), the application is not able to serve the incoming requests faster as it has to wait longer for gluster's response.

Volume Name: dv_repo
Type: Distributed-Replicate
Volume ID: 5662a158-6768-45bf-8dda-263c80d5a674
Status: Started
Number of Bricks: 3 x 3 = 9
Transport-type: tcp
Bricks:
Brick1: Server1:/export/dv_repo
Brick2: Server2:/export/dv_repo
Brick3: Server3:/export/dv_repo
Brick4: Server4:/export/dv_repo
Brick5: Server5:/export/dv_repo
Brick6: Server6:/export/dv_repo
Brick7: Server7:/export/dv_repo
Brick8: Server8:/export/dv_repo
Brick9: Server9/export/dv_repo
Options Reconfigured:
cluster.self-heal-daemon: disable
changelog.changelog: on
geo-replication.ignore-pid-check: on
geo-replication.indexing: on
performance.readdir-ahead: on
storage.owner-uid: 8080



Profile Info 
---------------
gluster volume profile dv_repo info incremental
Brick: Server3:/export/dv_repo
----------------------------------------
Interval 4 Stats:
   Block Size:                128b+                 256b+                 512b+
 No. of Reads:                    0                     2                     6
No. of Writes:                    1                     1                     5

   Block Size:               1024b+                2048b+                4096b+
 No. of Reads:                    5                     6                    23
No. of Writes:                   10                  1001                   418

   Block Size:               8192b+               16384b+               32768b+
 No. of Reads:                   28                    76                   120
No. of Writes:                 2593                  1976                   643

   Block Size:              65536b+              131072b+
 No. of Reads:                  312                  7105
No. of Writes:                 5320                     0
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us           1293      FORGET
      0.00       0.00 us       0.00 us       0.00 us            669     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00      51.44 us      38.00 us      94.00 us              9     OPENDIR
      0.00     120.62 us      55.00 us     361.00 us             16    SETXATTR
      0.00      43.00 us      18.00 us      84.00 us             52        STAT
      0.00     658.50 us     421.00 us    1036.00 us              4       MKDIR
      0.00      62.90 us      22.00 us     248.00 us             48     INODELK
      0.00     215.34 us      34.00 us     798.00 us             41    GETXATTR
      0.01      24.72 us       9.00 us     113.00 us            669       FLUSH
      0.02      42.17 us      17.00 us     208.00 us           1292      STATFS
      0.07      48.19 us      14.00 us     653.00 us           4519       FSTAT
      0.12     504.29 us     166.00 us    1729.00 us            669      CREATE
      0.14     526.19 us     136.00 us    2471.00 us            762      UNLINK
      0.14     165.72 us      25.00 us   19349.00 us           2532     XATTROP
      0.18      34.13 us      10.00 us     735.00 us          15164    FINODELK
      0.24      90.53 us      21.00 us    1046.00 us           7690        READ
      0.32      93.14 us      20.00 us   31061.00 us          10012    FXATTROP
      0.55     134.86 us      34.00 us   18012.00 us          11968       WRITE
      0.87    3181.46 us     104.00 us   20044.00 us            798       FSYNC
     21.51     150.90 us      33.00 us   34797.00 us         417326      LOOKUP
     75.84    1687.11 us       8.00 us 41182533.00 us         131608     ENTRYLK

    Duration: 229 seconds
   Data Read: 970995797 bytes
Data Written: 751416802 bytes

Brick: Server2:/export/dv_repo
----------------------------------------
Interval 4 Stats:
   Block Size:                128b+                 512b+                1024b+
 No. of Reads:                    0                     0                     0
No. of Writes:                    1                     3                     8

   Block Size:               2048b+                4096b+                8192b+
 No. of Reads:                    2                     2                     8
No. of Writes:                  526                   402                  2559

   Block Size:              16384b+               32768b+               65536b+
 No. of Reads:                   16                    27                    55
No. of Writes:                 1898                   363                  1661

   Block Size:             131072b+
 No. of Reads:                 1815
No. of Writes:                    0
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us            425      FORGET
      0.00       0.00 us       0.00 us       0.00 us            237     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00      60.50 us      30.00 us     103.00 us              4    GETXATTR
      0.00      44.83 us      33.00 us      58.00 us              6        STAT
      0.00      62.11 us      40.00 us      91.00 us              9     OPENDIR
      0.00     125.25 us      46.00 us     176.00 us             12    SETXATTR
      0.00     163.72 us      21.00 us     370.00 us             18    READDIRP
      0.00      98.73 us      50.00 us     295.00 us             33     XATTROP
      0.00      28.43 us      10.00 us      88.00 us            237       FLUSH
      0.01      38.64 us      14.00 us     155.00 us            474    FINODELK
      0.01     204.55 us      43.00 us     552.00 us            128       MKNOD
      0.02     206.38 us      45.00 us     439.00 us            252      CREATE
      0.02      41.05 us      14.00 us     214.00 us           1292      STATFS
      0.02      49.75 us      14.00 us     348.00 us           1128       FSTAT
      0.03      87.43 us      24.00 us     700.00 us            918    FXATTROP
      0.05     461.27 us      63.00 us    1959.00 us            252      UNLINK
      0.08      91.03 us      22.00 us     385.00 us           1931        READ
      0.08    6398.33 us      21.00 us  189665.00 us             30     INODELK
      0.29      92.59 us      34.00 us     587.00 us           7421       WRITE
      1.13      98.11 us      20.00 us    3159.00 us          26806       MKDIR
     16.03      86.20 us      22.00 us    4126.00 us         433826      LOOKUP
     82.22    2909.57 us       8.00 us 38563527.00 us          65936     ENTRYLK

    Duration: 229 seconds
   Data Read: 245544394 bytes
Data Written: 302268158 bytes

Brick: Server8:/export/dv_repo
----------------------------------------
Interval 4 Stats:
   Block Size:                128b+                 256b+                 512b+
 No. of Reads:                    0                     0                     0
No. of Writes:                    2                     3                     7

   Block Size:               1024b+                2048b+                4096b+
 No. of Reads:                    3                     5                    10
No. of Writes:                   21                  1539                  1923

   Block Size:               8192b+               16384b+               32768b+
 No. of Reads:                   16                    18                    46
No. of Writes:                11861                  8706                   140

   Block Size:              65536b+              131072b+
 No. of Reads:                  192                  3220
No. of Writes:                   27                     0
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us            760      FORGET
      0.00       0.00 us       0.00 us       0.00 us            328     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00     180.50 us     175.00 us     186.00 us              2        OPEN
      0.00      39.18 us      18.00 us      59.00 us             22        STAT
      0.00      98.33 us      31.00 us     132.00 us              9     OPENDIR
      0.00     258.25 us     220.00 us     310.00 us              4       MKDIR
      0.01     114.38 us      60.00 us     183.00 us             13    SETXATTR
      0.01      62.46 us      27.00 us     125.00 us             26     INODELK
      0.01     195.88 us      25.00 us     694.00 us             16    READDIRP
      0.03     131.35 us      53.00 us     388.00 us             62     XATTROP
      0.03     169.16 us      45.00 us     382.00 us             51    GETXATTR
      0.04      28.41 us      10.00 us      84.00 us            328       FLUSH
      0.13      47.58 us      19.00 us     192.00 us            668    FINODELK
      0.26      49.78 us      16.00 us     136.00 us           1292      STATFS
      0.31     117.72 us      34.00 us     260.00 us            666    FXATTROP
      0.37     284.81 us     150.00 us   12933.00 us            328      CREATE
      0.47      58.31 us      15.00 us    1513.00 us           2046       FSTAT
      0.65     475.47 us     148.00 us    9187.00 us            343      UNLINK
      0.94      50.19 us      11.00 us     368.00 us           4710     ENTRYLK
      1.59     113.93 us      23.00 us    1890.00 us           3513        READ
      7.14      74.13 us      30.00 us     550.00 us          24228       WRITE
     88.00     165.02 us      33.00 us   32511.00 us         134156      LOOKUP

    Duration: 229 seconds
   Data Read: 443973556 bytes
Data Written: 371097376 bytes

Brick: Server6:/export/dv_repo
----------------------------------------
Interval 4 Stats:
   Block Size:                  8b+                  16b+                  32b+
 No. of Reads:                    0                     0                     0
No. of Writes:                    1                     2                     2

   Block Size:                 64b+                 128b+                 256b+
 No. of Reads:                    0                     0                     0
No. of Writes:                    2                     1                    13

   Block Size:                512b+                1024b+                2048b+
 No. of Reads:                    0                     1                     7
No. of Writes:                   17                    39                  3514

   Block Size:               4096b+                8192b+               16384b+
 No. of Reads:                    8                    24                    40
No. of Writes:                 5045                 26970                 18290

   Block Size:              32768b+               65536b+              131072b+
 No. of Reads:                  109                   211                  5474
No. of Writes:                  198                    37                     0

 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us           1290      FORGET
      0.00       0.00 us       0.00 us       0.00 us            708     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00      84.00 us      70.00 us      98.00 us              2        OPEN
      0.00      61.75 us      27.00 us      85.00 us              4    GETXATTR
      0.00      69.89 us      43.00 us     101.00 us              9     OPENDIR
      0.01      95.54 us      51.00 us     200.00 us             13    SETXATTR
      0.01      44.52 us      14.00 us      87.00 us             31        STAT
      0.01      48.50 us      23.00 us      99.00 us             32     INODELK
      0.01     768.00 us     638.00 us     901.00 us              4       MKDIR
      0.08      25.62 us       9.00 us     104.00 us            708       FLUSH
      0.26      46.12 us      16.00 us     156.00 us           1292      STATFS
      0.26      42.50 us      17.00 us     136.00 us           1412    FINODELK
      0.54      44.83 us      12.00 us     181.00 us           2802     ENTRYLK
      0.63     101.65 us      24.00 us     339.00 us           1428    FXATTROP
      0.76      51.68 us      12.00 us     275.00 us           3379       FSTAT
      1.42     473.12 us     128.00 us    2420.00 us            691      UNLINK
      1.48     485.07 us     172.00 us    1078.00 us            706      CREATE
      2.55     100.28 us      21.00 us    2085.00 us           5876        READ
     16.61      70.79 us      27.00 us    2839.00 us          54131       WRITE
     75.38     134.80 us      34.00 us   17308.00 us         129034      LOOKUP

    Duration: 229 seconds
   Data Read: 746041663 bytes
Data Written: 807564460 bytes

Brick: Server5/export/dv_repo
----------------------------------------
Interval 4 Stats:
   Block Size:                  8b+                  16b+                  32b+
 No. of Reads:                    0                     0                     0
No. of Writes:                    1                     2                     2

   Block Size:                 64b+                 128b+                 256b+
 No. of Reads:                    1                     0                     1
No. of Writes:                    2                     1                    13

   Block Size:                512b+                1024b+                2048b+
 No. of Reads:                    2                     4                     4
No. of Writes:                   17                    39                  3514

   Block Size:               4096b+                8192b+               16384b+
 No. of Reads:                    8                    32                    68
No. of Writes:                 5045                 26970                 18290

   Block Size:              32768b+               65536b+              131072b+
 No. of Reads:                   86                   229                  5951
No. of Writes:                  198                    37                     0

 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us           1290      FORGET
      0.00       0.00 us       0.00 us       0.00 us            708     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00     112.50 us     109.00 us     116.00 us              2        OPEN
      0.00      58.25 us      46.00 us      67.00 us              4    GETXATTR
      0.00      87.22 us      41.00 us     130.00 us              9     OPENDIR
      0.00     264.75 us     205.00 us     337.00 us              4       MKDIR
      0.00      44.43 us      14.00 us     115.00 us             28        STAT
      0.01     111.38 us      65.00 us     174.00 us             13    SETXATTR
      0.01      54.50 us      25.00 us     139.00 us             32     INODELK
      0.02     199.23 us      26.00 us    1125.00 us             22    READDIRP
      0.07      26.31 us       9.00 us     122.00 us            708       FLUSH
      0.24      47.20 us      16.00 us     138.00 us           1292      STATFS
      0.25      45.51 us      18.00 us     206.00 us           1412    FINODELK
      0.52      47.01 us      16.00 us     320.00 us           2802     ENTRYLK
      0.61     109.87 us      26.00 us     229.00 us           1428    FXATTROP
      0.68     246.35 us     132.00 us    3819.00 us            706      CREATE
      0.77      53.48 us      11.00 us     198.00 us           3672       FSTAT
      1.33     491.62 us     137.00 us    5182.00 us            691      UNLINK
      2.58     103.18 us      20.00 us    2865.00 us           6388        READ
     15.45      72.84 us      26.00 us    1442.00 us          54131       WRITE
     77.45     153.20 us      34.00 us  104853.00 us         129033      LOOKUP

    Duration: 229 seconds
   Data Read: 809567845 bytes
Data Written: 807564460 bytes

Brick: Server9:/export/dv_repo
----------------------------------------
Interval 1 Stats:
   Block Size:                128b+                 256b+                 512b+
 No. of Reads:                    0                     1                     1
No. of Writes:                    2                     3                     7

   Block Size:               1024b+                2048b+                4096b+
 No. of Reads:                    0                     2                     8
No. of Writes:                   21                  1540                  1923

   Block Size:               8192b+               16384b+               32768b+
 No. of Reads:                    7                    26                    35
No. of Writes:                11861                  8707                   140

   Block Size:              65536b+              131072b+
 No. of Reads:                  132                  2331
No. of Writes:                   27                     0
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us            728      FORGET
      0.00       0.00 us       0.00 us       0.00 us            328     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00     124.00 us     115.00 us     133.00 us              2        OPEN
      0.00      69.75 us      51.00 us      93.00 us              4    GETXATTR
      0.00      88.00 us      47.00 us     145.00 us              9     OPENDIR
      0.00      45.57 us      21.00 us      90.00 us             21        STAT
      0.00      91.82 us      50.00 us     162.00 us             11     XATTROP
      0.01     106.15 us      52.00 us     136.00 us             13    SETXATTR
      0.01      58.54 us      26.00 us     124.00 us             26     INODELK
      0.03      26.11 us       9.00 us     332.00 us            328       FLUSH
      0.05    2926.75 us     378.00 us    5261.00 us              4       MKDIR
      0.13      48.64 us      14.00 us     277.00 us            670    FINODELK
      0.25      48.96 us      19.00 us    1462.00 us           1292      STATFS
      0.32    1585.08 us     207.00 us    4095.00 us             51       MKNOD
      0.33     123.49 us      55.00 us     839.00 us            666    FXATTROP
      0.34      55.83 us      15.00 us     329.00 us           1511       FSTAT
      0.77     565.10 us     123.00 us    9516.00 us            343      UNLINK
      0.96      51.25 us      10.00 us    3426.00 us           4710     ENTRYLK
      1.11     109.68 us      24.00 us    1135.00 us           2554        READ
      2.01    1541.15 us     165.00 us    4827.00 us            328      CREATE
      7.11      73.78 us      29.00 us    4718.00 us          24231       WRITE
     86.57     162.33 us      32.00 us   22330.00 us         134159      LOOKUP

    Duration: 229 seconds
   Data Read: 321019450 bytes
Data Written: 371119591 bytes

Brick: Server1:/export/dv_repo
----------------------------------------
Interval 4 Stats:
   Block Size:                128b+                 256b+                 512b+
 No. of Reads:                    1                     4                    10
No. of Writes:                    1                     1                     5

   Block Size:               1024b+                2048b+                4096b+
 No. of Reads:                    3                    16                    25
No. of Writes:                   10                  1001                   418

   Block Size:               8192b+               16384b+               32768b+
 No. of Reads:                   52                   117                   193
No. of Writes:                 2593                  1976                   643

   Block Size:              65536b+              131072b+
 No. of Reads:                  531                 11145
No. of Writes:                 5320                     0
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us           1233      FORGET
      0.00       0.00 us       0.00 us       0.00 us            669     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00     354.00 us     290.00 us     527.00 us              4       MKDIR
      0.00     266.60 us      16.00 us    7646.00 us            105        STAT
      0.00    3322.67 us      62.00 us   11074.00 us              9     OPENDIR
      0.00     214.21 us       9.00 us   27224.00 us            669       FLUSH
      0.00   74751.00 us      41.00 us  445195.00 us              6    READDIRP
      0.00     482.80 us      22.00 us   74947.00 us           1292      STATFS
      0.00    1240.61 us     198.00 us   29810.00 us            669      CREATE
      0.00    1342.66 us     168.00 us   41128.00 us            762      UNLINK
      0.00     292.44 us      12.00 us   44091.00 us           6578       FSTAT
      0.00     200.80 us      10.00 us   42864.00 us          18898    FINODELK
      0.00     362.70 us      22.00 us   36913.00 us          12102        READ
      0.00    6783.40 us     110.00 us   61766.00 us            798       FSYNC
      0.00     625.49 us      46.00 us   42693.00 us          11968       WRITE
      0.00     965.13 us      27.00 us   64134.00 us          10012    FXATTROP
      0.01 4200339.38 us      77.00 us 41078178.00 us             16    SETXATTR
      0.02 1998934.33 us      28.00 us 41365012.00 us             85    GETXATTR
      0.07 14500594.56 us      30.00 us 235894105.00 us             48     INODELK
      0.07  274987.36 us      39.00 us 41596014.00 us           2532     XATTROP
     46.33 1079715.51 us      52.00 us 41770605.00 us         417268      LOOKUP
     53.50 3980262.68 us      11.00 us 472355943.00 us         130686     ENTRYLK

    Duration: 229 seconds
   Data Read: 1527648224 bytes
Data Written: 751416802 bytes

Brick: Server4:/export/dv_repo
----------------------------------------
Interval 4 Stats:
   Block Size:                  8b+                  16b+                  32b+
 No. of Reads:                    0                     0                     0
No. of Writes:                    1                     2                     2

   Block Size:                 64b+                 128b+                 256b+
 No. of Reads:                    0                     1                     0
No. of Writes:                    2                     1                    13

   Block Size:                512b+                1024b+                2048b+
 No. of Reads:                    5                     4                     6
No. of Writes:                   17                    39                  3514

   Block Size:               4096b+                8192b+               16384b+
 No. of Reads:                   11                    17                    54
No. of Writes:                 5045                 26970                 18290

   Block Size:              32768b+               65536b+              131072b+
 No. of Reads:                   97                   281                  6181
No. of Writes:                  198                    37                     0

 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us           1295      FORGET
      0.00       0.00 us       0.00 us       0.00 us            708     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00     120.50 us      88.00 us     153.00 us              2        OPEN
      0.00      90.89 us      60.00 us     141.00 us              9     OPENDIR
      0.00     482.00 us      53.00 us     911.00 us              2    READDIRP
      0.00     286.25 us     237.00 us     334.00 us              4       MKDIR
      0.00      96.15 us      53.00 us     172.00 us             13    SETXATTR
      0.01      45.62 us      18.00 us     101.00 us             29        STAT
      0.01      55.53 us      26.00 us     111.00 us             32     INODELK
      0.06      22.24 us       8.00 us      83.00 us            708       FLUSH
      0.08     178.77 us      30.00 us     409.00 us            115    GETXATTR
      0.22      39.85 us      15.00 us     178.00 us           1412    FINODELK
      0.24      46.76 us      16.00 us     139.00 us           1292      STATFS
      0.46      41.61 us      15.00 us     171.00 us           2802     ENTRYLK
      0.57     101.40 us      26.00 us     217.00 us           1428    FXATTROP
      0.70     252.43 us     160.00 us     513.00 us            706      CREATE
      0.79      52.28 us      12.00 us     200.00 us           3843       FSTAT
      1.27     467.65 us     136.00 us    5732.00 us            691      UNLINK
      2.71     103.23 us      21.00 us    9713.00 us           6660        READ
     13.90      65.20 us      28.00 us    3065.00 us          54131       WRITE
     78.96     155.36 us      36.00 us   87299.00 us         129041      LOOKUP

    Duration: 229 seconds
   Data Read: 845204064 bytes
Data Written: 807564460 bytes

Brick: Server7:/export/dv_repo
----------------------------------------
Interval 4 Stats:
   Block Size:                128b+                 256b+                 512b+
 No. of Reads:                    0                     2                     0
No. of Writes:                    2                     3                     7

   Block Size:               1024b+                2048b+                4096b+
 No. of Reads:                    5                     4                     4
No. of Writes:                   21                  1539                  1923

   Block Size:               8192b+               16384b+               32768b+
 No. of Reads:                   17                    35                    66
No. of Writes:                11863                  8708                   140

   Block Size:              65536b+              131072b+
 No. of Reads:                  199                  3942
No. of Writes:                   25                     0
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us            774      FORGET
      0.00       0.00 us       0.00 us       0.00 us            327     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              9  RELEASEDIR
      0.00      99.50 us      62.00 us     137.00 us              2        OPEN
      0.00     236.00 us      29.00 us     443.00 us              2    READDIRP
      0.00      42.18 us      21.00 us      71.00 us             22        STAT
      0.00     281.00 us     255.00 us     311.00 us              4       MKDIR
      0.01     100.62 us      67.00 us     180.00 us             13    SETXATTR
      0.01      60.27 us      29.00 us     126.00 us             26     INODELK
      0.01     207.33 us      35.00 us    1171.00 us              9     OPENDIR
      0.02     107.02 us      65.00 us     269.00 us             51     XATTROP
      0.04      26.83 us       8.00 us     105.00 us            327       FLUSH
      0.05     174.41 us      35.00 us     447.00 us             75    GETXATTR
      0.12      44.34 us      18.00 us     123.00 us            652    FINODELK
      0.25      47.97 us      16.00 us     229.00 us           1292      STATFS
      0.32     117.19 us      39.00 us     270.00 us            658    FXATTROP
      0.36     269.22 us     166.00 us    1256.00 us            327      CREATE
      0.53      45.34 us      11.00 us     480.00 us           2836     ENTRYLK
      0.55      54.27 us      12.00 us     783.00 us           2482       FSTAT
      0.77     553.87 us     149.00 us   12336.00 us            339      UNLINK
      1.90     108.45 us      22.00 us    1621.00 us           4278        READ
      7.03      70.74 us      28.00 us    1578.00 us          24231       WRITE
     88.02     159.88 us      30.00 us   57080.00 us         134187      LOOKUP

    Duration: 229 seconds
   Data Read: 541364650 bytes
Data Written: 370965933 bytes

storage.owner-gid: 8080
diagnostics.latency-measurement: on
diagnostics.count-fop-hits: on
performance.low-prio-threads: 64
performance.normal-prio-threads: 64
performance.high-prio-threads: 64
performance.io-thread-count: 64
server.outstanding-rpc-limit: 128
network.inode-lru-limit: 32000
server.event-threads: 8
cluster.entry-self-heal: off
cluster.metadata-self-heal: off
cluster.data-self-heal: off
Comment 15 errata-xmlrpc 2017-09-14 04:38:53 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2730

Note You need to log in before you can comment on or make changes to this bug.