Description of problem:
When creating a distributed gluster filesystem and setting subvols-per-directory to a number lower than the number of bricks (we set it to 1 in our specific case), for each subdirectory within the volume, the trusted.glusterfs.dht attribute is only created on the bricks that are chosen to receive files, and is absent entirely on the other bricks. Consequently, every time a subdirectory is listed on the client side, The following lines appear (#bricks - subvols-per-directory) times in the mount log in /var/log/glusterfs :
[2013-10-23 23:08:28.297937] I [dht-layout.c:718:dht_layout_dir_mismatch] 0-glusterfs-volume-dht: /some/subdir - disk layout missing
[2013-10-23 23:08:28.297964] I [dht-common.c:623:dht_revalidate_cbk] 0-glusterfs-volume-dht: mismatching layouts for /some/subdir

We are aware that these are harmless informational messages, but their sheer volume is causing issues. Specifically, we're maintaining a locate database of this gluster filesystem, and every day, the database is updated and the size of the log grows by more than 10GB. This is a problem because it makes it difficult to find other information in the log and because /var/log is mounted on a rather small partition.

The problem can be worked around by running the following oneliner on all the server nodes:
find /glusterbricks/brick* -type d -exec sh -c "getfattr -n trusted.glusterfs.dht -e hex '{}' 2>&1 >/dev/null | grep -q 'No such attribute$' && setfattr -n trusted.glusterfs.dht -v 0x00000001000000000000000000000000 '{}'" \;
However, this is not a very satisfactory workaround because:
- it is somewhat time-consuming,
- it needs to be repeated every time a significant number of directories is added to the filesystem,
- we dislike messing with Gluster's internals.

Version-Release number of selected component (if applicable):
glusterfs release 3.4.0

How reproducible:
Always, so far.

Steps to Reproduce:
1. Create distributed volume with multiple bricks
2. gluster volume set glusterfs-volume subvols-per-directory 1
3. gluster volume rebalance glusterfs-volume start force
4. Mount volume on a client and populate it with directories and files
5. Wait some time, then list one of these directories (on client)
6. Observe log in mount log in /var/log/glusterfs (on client)

Actual results:
Log is full of the following messages
[2013-10-23 23:08:28.297937] I [dht-layout.c:718:dht_layout_dir_mismatch] 0-glusterfs-volume-dht: /some/subdir - disk layout missing
[2013-10-23 23:08:28.297964] I [dht-common.c:623:dht_revalidate_cbk] 0-glusterfs-volume-dht: mismatching layouts for /some/subdir

Expected results:
Log doesn't grow so fast.

Additional info:
Problem can be worked around by running the following oneliner on server:
find /bricks/brick* -type d -exec sh -c "getfattr -n trusted.glusterfs.dht -e hex '{}' 2>&1 >/dev/null | grep -q 'No such attribute$' && setfattr -n trusted.glusterfs.dht -v 0x00000001000000000000000000000000 '{}'" \;
but workaround needs to be re-applied every time directories are added.

It would be nice if a developer could tell us whether this workaround will have side effects, ie. whether there's a good reason trusted.glusterfs.dht is missing. If so, we feel the messages in the log should be suppressed.