Bug 764139 (GLUSTER-2407)

Having seen this come up multiple times in email/IRC, I just added some text to http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Configuring_Distributed_Replicated_Volumes to highlight the importance of specifying bricks in an order that ensures proper data protection.  For example:

  volume create fubar replica 2 srv1:/brk1 srv1:/brk2 srv2:/brk1 srv2:/brk2

This will result in the following structure:

  fubar (distribute)
    fubar-replicate-1 (replicate)
      srv1:/brk1
      srv1:/brk2
    fubar-replicate-2 (replicate)
      srv2:/brk1
      srv2:/brk2

This is obviously sub-optimal, as a failure of srv1 will leave both replica members for fubar-replicate-1 unavailable.  I'm sure I don't need to tell you guys that the following is preferable:

  volume create fubar replica 2 srv1:/brk1 srv2:/brk1 srv1:/brk2 srv2:/brk2

The enhancement would be to check for co-location of replica members on a single server, and at the very least print a warning that such a configuration does not ensure proper data protection.  We might even propose a "fixed" alternative using the same bricks in a better order (as above).  I don't think it's worth trying to deal with multi-homed hosts, or rack-level colocations, or other less obvious data-layout issues, but this particular user error seems so common and obvious that we should anticipate and mitigate it.