Bug 762711 (GLUSTER-979)

Summary: [3.0.5rc3] Dbench fails on write-behind
Product: [Community] GlusterFS Reporter: Anush Shetty <anush>
Component: write-behindAssignee: Raghavendra G <raghavendra>
Status: CLOSED NOTABUG QA Contact:
Severity: medium Docs Contact:
Priority: low    
Version: 3.0.4CC: gluster-bugs, vijay
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Anush Shetty 2010-06-03 06:22:00 UTC 
Dbench is a simple posix + write-behind setup. The backend for this was on another dht+afr gluster mountpoint 

Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs
5 of 10 processes prepared for launch   0 sec
10 of 10 processes prepared for launch   0 sec
releasing clients
   9        13     0.46 MB/sec  warmup   1 sec  latency 998.247 ms
  10        28     4.26 MB/sec  warmup   2 sec  latency 178.639 ms
  10        42     5.35 MB/sec  warmup   3 sec  latency 340.743 ms
  10        58     6.03 MB/sec  warmup   4 sec  latency 302.611 ms
  10        81     7.24 MB/sec  warmup   5 sec  latency 133.790 ms
  10       104     7.93 MB/sec  warmup   6 sec  latency 136.084 ms
  10       125     7.99 MB/sec  warmup   7 sec  latency 178.062 ms
  10       148     7.88 MB/sec  warmup   8 sec  latency 184.170 ms
  10       169     7.70 MB/sec  warmup   9 sec  latency 217.623 ms
  10       190     7.56 MB/sec  warmup  10 sec  latency 261.899 ms
  10       213     7.52 MB/sec  warmup  11 sec  latency 236.207 ms
  10       237     7.54 MB/sec  warmup  12 sec  latency 174.411 ms
  10       257     7.46 MB/sec  warmup  13 sec  latency 160.841 ms
  10       282     7.49 MB/sec  warmup  14 sec  latency 165.022 ms
  10       306     7.46 MB/sec  warmup  15 sec  latency 152.025 ms
  10       330     7.44 MB/sec  warmup  16 sec  latency 157.038 ms
  10       353     7.43 MB/sec  warmup  17 sec  latency 177.832 ms
  10       374     7.39 MB/sec  warmup  18 sec  latency 176.836 ms
  10       393     7.32 MB/sec  warmup  19 sec  latency 169.982 ms
  10       415     7.33 MB/sec  warmup  20 sec  latency 154.468 ms
  10       439     7.35 MB/sec  warmup  21 sec  latency 167.550 ms
  10       459     7.31 MB/sec  warmup  22 sec  latency 192.347 ms
  10       480     7.28 MB/sec  warmup  23 sec  latency 172.585 ms
  10       500     7.27 MB/sec  warmup  24 sec  latency 166.961 ms
  10       519     7.17 MB/sec  warmup  25 sec  latency 237.123 ms
  10       537     7.16 MB/sec  warmup  26 sec  latency 163.297 ms
  10       551     7.07 MB/sec  warmup  27 sec  latency 182.830 ms
  10       574     6.99 MB/sec  warmup  28 sec  latency 267.170 ms
  10       594     6.93 MB/sec  warmup  29 sec  latency 305.811 ms
  10       612     6.90 MB/sec  warmup  30 sec  latency 342.489 ms
  10       629     6.77 MB/sec  warmup  31 sec  latency 312.499 ms
  10       649     6.59 MB/sec  warmup  32 sec  latency 458.916 ms
  10       660     6.39 MB/sec  warmup  33 sec  latency 1066.125 ms
  10       672     6.21 MB/sec  warmup  34 sec  latency 401.421 ms
  10       693     6.06 MB/sec  warmup  35 sec  latency 443.421 ms
  10       710     5.90 MB/sec  warmup  36 sec  latency 399.808 ms
[710] rename ./clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT ./clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP failed (File exists) - expected NT_STATUS_OK
ERROR: child 4 failed at line 710
Child failed with status 1

Client log:

/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (Invalid argument)
[2010-06-03 11:27:13] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 48305: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (Invalid argument)
[2010-06-03 11:27:43] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 62935: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:28:13] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 76943: /touch/clients/client2/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client2/~dmtmp/PWRPNT/PPTC112.TMP => -1 (Invalid argument)
[2010-06-03 11:28:44] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 91716: /touch/clients/client7/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client7/~dmtmp/PWRPNT/PPTC112.TMP => -1 (Invalid argument)
[2010-06-03 11:28:44] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 91730: /touch/clients/client0/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client0/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:29:15] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 105865: /touch/clients/client0/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client0/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:29:47] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 120158: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:30:20] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 134742: /touch/clients/client5/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client5/~dmtmp/PWRPNT/PPTC112.TMP => -1 (Invalid argument)
[2010-06-03 11:30:20] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 134761: /touch/clients/client0/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client0/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:30:51] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 148763: /touch/clients/client7/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client7/~dmtmp/PWRPNT/PPTC112.TMP => -1 (Invalid argument)
[2010-06-03 11:31:22] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 163001: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:31:55] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 177219: /touch/clients/client7/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client7/~dmtmp/PWRPNT/PPTC112.TMP => -1 (Invalid argument)
[2010-06-03 11:32:32] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 191210: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (Invalid argument)
[2010-06-03 11:33:06] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 204395: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:33:43] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 218361: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:34:24] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 233104: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:34:25] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 233163: /touch/clients/client0/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client0/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:34:56] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 246406: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:35:37] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 261586: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:35:37] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 261607: /touch/clients/client7/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client7/~dmtmp/PWRPNT/PPTC112.TMP => -1 (Invalid argument)
[2010-06-03 11:36:13] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 275624: /touch/clients/client0/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client0/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:36:46] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 289956: /touch/clients/client0/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client0/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:37:21] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 304019: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:37:21] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 304066: /touch/clients/client7/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client7/~dmtmp/PWRPNT/PPTC112.TMP => -1 (Invalid argument)
[2010-06-03 11:37:51] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 316777: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:38:25] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 331455: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:38:57] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 345653: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:39:29] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 360006: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:39:59] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 373421: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:40:32] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 387662: /touch/clients/client0/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client0/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:41:06] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 402288: /touch/clients/client7/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client7/~dmtmp/PWRPNT/PPTC112.TMP => -1 (Invalid argument)
[2010-06-03 11:41:39] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 416946: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:42:09] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 430243: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:42:41] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 444290: /touch/clients/client9/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client9/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:43:12] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 457715: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:43:46] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 472012: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
[2010-06-03 11:44:22] W [fuse-bridge.c:1533:fuse_rename_cbk] glusterfs-fuse: 486938: /touch/clients/client4/~dmtmp/PWRPNT/NEWPCB.PPT -> /touch/clients/client4/~dmtmp/PWRPNT/PPTB1E4.TMP => -1 (File exists)
Comment 1 Raghavendra G 2010-06-17 00:11:44 UTC 
were there any performance translators loaded on the mount point which is used as backend?
Comment 2 Vijay Bellur 2010-10-27 06:17:52 UTC 
Not recommended configuration.