rm -rf parallely from 2 clients gives ENOMEM. But from single client it continues. Configuration is single client-server model without any clustering xlators.

[2011-08-17 14:56:57.733089] D [inodelk.c:293:__inode_unlock_lock] 0-vol-locks:  Matching lock found for unlock
[2011-08-17 14:56:57.733114] D [marker-quota.c:1184:quota_inodelk_cbk] 0-vol-marker: inodelk released on /
[2011-08-17 14:56:57.740990] D [marker-quota.c:1981:inspect_file_xattr] 0-vol-marker: size=4096 contri=4096
[2011-08-17 14:56:57.741202] D [marker-quota.c:1981:inspect_file_xattr] 0-vol-marker: size=4096 contri=4096
[2011-08-17 14:56:57.741462] E [posix.c:652:posix_stat] 0-vol-posix: lstat on /okpa/linux-3.0.1/fs/freevxfs/Makefile failed: No such file or directory
[2011-08-17 14:56:57.741492] I [server3_1-fops.c:964:server_unlink_cbk] 0-vol-server: 515808: UNLINK /okpa/linux-3.0.1/fs/freevxfs/Makefile (798179) ==> -1 (Cannot allocate memo
ry)

It is because if this piece of code.

int32_t
marker_unlink_stat_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
                        int32_t op_ret, int32_t op_errno, struct iatt *buf)
{
	marker_local_t *local = NULL;

	if (op_ret < 0) {
	        goto err;
	}

	local = frame->local;
	if (local == NULL) {
                goto err;
	}

	local->ia_nlink = buf->ia_nlink;

        STACK_WIND (frame, marker_unlink_cbk, FIRST_CHILD(this),
	            FIRST_CHILD(this)->fops->unlink, &local->loc);
	return 0;
err:
        STACK_UNWIND_STRICT (unlink, frame, -1, ENOMEM, NULL, NULL);

	return 0;
}

Situation is: 

Marker sends stat first before sending the unlink. By the time stat reaches posix if the file is deleted by another client, then stat fails with op_errno set to ENOENT. But in the corresponding call back function of marker (i.e. the above funtion marker_unlink_stat_cbk), we check for (op_ret < 0) and if so we unwind by sending ENOMEM as the error to the above xlators which results in the application(rm -rf in this case) also getting ENOMEM.