this is the questionable HotRod clear() replication test case scenario, It works but I suspect this is not the way to test it because hotrod hra1 and hra2 send the request to the same node.

    	String[] testHosts = getServers();
        Properties props1 = new Properties();
        Properties props2 = new Properties();
        props1.setProperty("infinispan.client.hotrod.server_list", testHosts[0] +":11222");
        props2.setProperty("infinispan.client.hotrod.server_list", testHosts[1] +":11222");
        RemoteCacheManager hra1 = new RemoteCacheManager(props1);
        RemoteCacheManager hra2 = new RemoteCacheManager(props2);
        Cache c1 = hra1.getCache("aaa");
        Cache c2 = hra2.getCache("aaa");
        c1.put("a", "a");
        c1.put("b", "b");
        c1.put("c", "c");
        assertEquals("a", c2.get("a"));
        assertEquals("b", c2.get("b"));
        assertEquals("c", c2.get("c"));
        c1.clear();
        assertNull(c2.get("a"));
        assertNull(c2.get("b"));
        assertNull(c2.get("c"));

Hot Rod and REST result in cluster wide clears, Memcached should be the same.

Michal, not sure what your problem is with that test that you mention in the last comment. Whether c1 or c2 calls to get/put go to same node depend on diff factors, whether HR servers are replicated or distributed. The clear will get to one of the nodes and clear all the cache everywhere. I agree that you might not be sure where c1.clear() lands. You could just add extra checks for c1.get() to make sure it returns null. 

Link: Added: This issue depends ISPN-821

The comment is not very important, it just showed how I tried to learn whether hotrod clear() replicates.
I just didn't know whether this test is good enough to prove whether clear() will clear all nodes.
it would prove this only in case c1.get returns null and c2.get returns not null
but in this case c2.returns null

and it might be because c1 and c2 are getting from the same node, not because clear() replicates, 
it's possible that the testcase would pass even if clear() didn't replicate.

The problem with flush_all replicating is that you then can't use the strategy suggested of flush_all plus a delay in order to avoid clearing all memcached servers at the same time:

"The intent of flush_all with a delay, was that in a setting where you
have a pool of memcached servers, and you need to flush all content,
you have the option of not resetting all memcached servers at the
same time (which could e.g. cause a spike in database load with all
clients suddenly needing to recreate content that would otherwise
have been found in the memcached daemon).

The delay option allows you to have them reset in e.g. 10 second
intervals (by passing 0 to the first, 10 to the second, 20 to the
third, etc. etc.)."

I think that's precisely why I made it local and skip cache store. I think I'll leave it as it is.

If anything, I could make flush_all without params be cluster wide, and flush_all with delay be local only. However, I think this might cause more confusion.

I see, that's a good argument for leaving it as it is. OK then,

anyway seems worth mentioning in the docs...

Good point, I'll add a note to the Memcached wiki explaining how flush_all call is local.

Thanks for the thorough work Michal!

Updated http://community.jboss.org/docs/DOC-14644