Description of problem:
writes operation on files fails on nfs mount when there are parallel writes from other mounts on the same volume and quota limit-usage is set 

Version-Release number of selected component (if applicable):
3.3.0qa26

How reproducible:
often

Steps to Reproduce:
1.create a distribute-replicate volume (2 x 3, 200GB total available space).
Start the volume
2.Create 2 gluster mounts and one nfs mount
3.Start dd on one of the gluster mount and nfs mount
4.start ping_pong on a file on other gluster mount
5.enable quota on the volume
6.set the quota limit-usage to some value (The quota limit-usage was set to 150GB.)
7.The dd on the nfs mount fails.

Actual results:
[03/10/12 - 17:32:47 root@APP-CLIENT1 nfsc1]# dd if=/dev/zero of=nfsf1 bs=1M count=51200
dd: writing `nfsf1': Invalid argument
4152+0 records in
4151+0 records out
4352638976 bytes (4.4 GB) copied, 387.761 s, 11.2 MB/s


Expected results:
dd should continue to run. 

Additional info:
Brick Log:-
------------
[2012-03-10 17:41:50.374193] W [marker-quota.c:2042:mq_inspect_directory_xattr] 0-store1-marker: cannot add a new contribution node
[2012-03-10 17:41:50.374308] W [marker-quota.c:2042:mq_inspect_directory_xattr] 0-store1-marker: cannot add a new contribution node
[2012-03-10 17:41:50.374719] D [io-threads.c:275:iot_schedule] 0-store1-io-threads: FINODELK scheduled as normal fop
[2012-03-10 17:41:50.374965] E [marker-quota-helper.c:230:mq_dict_set_contribution] (-->/usr/local/lib/glusterfs/3.3.0qa26/xlator/debug/io-stats.so(io_stats_lookup+0x28c) [0x7f2eb8caea02] (-->/usr/local/lib/glusterfs/3.3.0qa26/xlator/features/marker.so(marker_lookup+0x142) [0x7f2eb8eccc45] (-->/usr/local/lib/glusterfs/3.3.0qa26/xlator/features/marker.so(mq_req_xattr+0x123) [0x7f2eb8ed7cdc]))) 0-marker: invalid argument: loc->parent


NFS Server Log:- (Check the log during the time frame: [2012-03-10 17:39:55.205570])
--------------
[2012-03-10 17:40:57.079499] D [nfs3-helpers.c:1765:nfs3_log_rw_call] 0-nfs-nfsv3: XID: 369f3732, WRITE: args: FH: hashcount 1, exportid ed0838af-6b69-4a3b-8357-fc7a71cad4af, gfid 39786995-e612-429e-9939-fc1fe1a055c7, offset: 4233867264,  count: 65536, UNSTABLE
[2012-03-10 17:40:57.080238] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-1 returned error (No such file or directory)
[2012-03-10 17:40:57.081214] D [nfs3-helpers.c:1765:nfs3_log_rw_call] 0-nfs-nfsv3: XID: 359f3732, WRITE: args: FH: hashcount 1, exportid ed0838af-6b69-4a3b-8357-fc7a71cad4af, gfid 39786995-e612-429e-9939-fc1fe1a055c7, offset: 4233801728,  count: 65536, UNSTABLE
[2012-03-10 17:40:57.082019] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-1 returned error (No such file or directory)
[2012-03-10 17:40:57.082337] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-1 returned error (No such file or directory)
[2012-03-10 17:40:57.082556] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-1 returned error (No such file or directory)
[2012-03-10 17:40:57.082706] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-1 returned error (No such file or directory)
[2012-03-10 17:40:57.082837] D [nfs3-helpers.c:1765:nfs3_log_rw_call] 0-nfs-nfsv3: XID: 349f3732, WRITE: args: FH: hashcount 1, exportid ed0838af-6b69-4a3b-8357-fc7a71cad4af, gfid 39786995-e612-429e-9939-fc1fe1a055c7, offset: 4233736192,  count: 65536, UNSTABLE
[2012-03-10 17:40:57.083509] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-1 returned error (No such file or directory)
[2012-03-10 17:40:57.103183] D [nfs3-helpers.c:1765:nfs3_log_rw_call] 0-nfs-nfsv3: XID: 339f3732, WRITE: args: FH: hashcount 1, exportid ed0838af-6b69-4a3b-8357-fc7a71cad4af, gfid 39786995-e612-429e-9939-fc1fe1a055c7, offset: 4233670656,  count: 65536, UNSTABLE
[2012-03-10 17:40:57.103801] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-1 returned error (No such file or directory)
[2012-03-10 17:40:57.103974] D [nfs3-helpers.c:1765:nfs3_log_rw_call] 0-nfs-nfsv3: XID: 329f3732, WRITE: args: FH: hashcount 1, exportid ed0838af-6b69-4a3b-8357-fc7a71cad4af, gfid 39786995-e612-429e-9939-fc1fe1a055c7, offset: 4233605120,  count: 65536, UNSTABLE

[2012-03-10 17:40:57.105841] D [afr-self-heal-common.c:148:afr_sh_print_pending_matrix] 0-store1-replicate-0: pending_matrix: [ 3 3 3 ]
[2012-03-10 17:40:57.105868] D [afr-self-heal-common.c:148:afr_sh_print_pending_matrix] 0-store1-replicate-0: pending_matrix: [ 3 3 3 ]
[2012-03-10 17:40:57.105884] D [afr-self-heal-common.c:148:afr_sh_print_pending_matrix] 0-store1-replicate-0: pending_matrix: [ 3 3 3 ]
[2012-03-10 17:40:57.105903] D [afr-self-heal-common.c:753:afr_mark_sources] 0-store1-replicate-0: Number of sources: 3
[2012-03-10 17:40:57.105921] D [afr-self-heal-data.c:799:afr_lookup_select_read_child_by_txn_type] 0-store1-replicate-0: returning read_child: 2
[2012-03-10 17:40:57.105937] D [afr-common.c:1268:afr_lookup_select_read_child] 0-store1-replicate-0: Source selected as 2 for <gfid:00000000-0000-0000-0000-000000000000>
[2012-03-10 17:40:57.105960] D [afr-common.c:1075:afr_lookup_build_response_params] 0-store1-replicate-0: Building lookup response from 2
[2012-03-10 17:40:57.105988] D [afr-common.c:1137:afr_lookup_set_self_heal_params_by_xattr] 0-store1-replicate-0: data self-heal is pending for <gfid:00000000-0000-0000-0000-000000000000>.
[2012-03-10 17:40:57.106008] D [afr-common.c:1137:afr_lookup_set_self_heal_params_by_xattr] 0-store1-replicate-0: data self-heal is pending for <gfid:00000000-0000-0000-0000-000000000000>.
[2012-03-10 17:40:57.106027] D [afr-common.c:1137:afr_lookup_set_self_heal_params_by_xattr] 0-store1-replicate-0: data self-heal is pending for <gfid:00000000-0000-0000-0000-000000000000>.
[2012-03-10 17:40:57.106046] I [afr-common.c:1313:afr_launch_self_heal] 0-store1-replicate-0: background  data self-heal triggered. path: <gfid:00000000-0000-0000-0000-000000000000>, reason: lookup detected pending operations
[2012-03-10 17:40:57.106104] D [afr-self-heal-metadata.c:78:afr_sh_metadata_done] 0-store1-replicate-0: proceeding to data check on <gfid:00000000-0000-0000-0000-000000000000>
[2012-03-10 17:40:57.123809] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-1 returned error (No such file or directory)
[2012-03-10 17:40:57.195300] D [dht-common.c:269:dht_discover_cbk] 0-store1-dht: lookup of <gfid:00000000-0000-0000-0000-000000000000> on store1-replicate-0 returned error (Input/output error)
[2012-03-10 17:40:57.195335] I [dht-layout.c:600:dht_layout_normalize] 0-store1-dht: found anomalies in <gfid:00000000-0000-0000-0000-000000000000>. holes=1 overlaps=0
[2012-03-10 17:40:57.195355] D [dht-layout.c:614:dht_layout_normalize] 0-store1-dht: path=<gfid:00000000-0000-0000-0000-000000000000> err=No such file or directory on subvol=store1-replicate-1
[2012-03-10 17:40:57.195372] D [dht-layout.c:614:dht_layout_normalize] 0-store1-dht: path=<gfid:00000000-0000-0000-0000-000000000000> err=Input/output error on subvol=store1-replicate-0
[2012-03-10 17:40:57.195387] D [dht-common.c:197:dht_discover_complete] 0-store1-dht: normalizing failed on <gfid:00000000-0000-0000-0000-000000000000>
[2012-03-10 17:40:57.195456] E [nfs3-helpers.c:3768:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:00000000-0000-0000-0000-000000000000>: Invalid argument
[2012-03-10 17:40:57.195543] E [nfs3.c:2150:nfs3_write_resume] 0-nfs-nfsv3: Unable to resolve FH: store1 : 00000000-0000-0000-0000-000000000000
[2012-03-10 17:40:57.195573] W [nfs3-helpers.c:3392:nfs3_log_common_res] 0-nfs-nfsv3: XID: 2e9f3732, WRITE: NFS: 22(Invalid argument for operation), POSIX: 14(Bad address)