Description of problem: While creating volumes, if the peers aren't part of the cluster, the volume create fails but it sets the extended attributes on the local brick. Because when we retry to create the volume after adding the peers to a cluster, the volume creation errors out saying that local brick is already part of a volume.



Version-Release number of selected component (if applicable): glusterfs-3.4.0.21rhs-1.el6rhs.x86_64


How reproducible: Consistently 


Steps to Reproduce:
1. Try creating a volume with adding peers to the cluster 
2. Add peers to the cluster
3. Now retry creating the volume

Actual results:

Volume creation errors out.

Expected results:

When peers are not in cluster, extended attributes shouldn't be set.

Additional info:

# gluster volume create glance-vol replica 2 rhshdp01.lab.eng.blr.redhat.com:/glance1/s1 rhshdp02.lab.eng.blr.redhat.com:/glance1/s2 rhshdp03.lab.eng.blr.redhat.com:/glance1/s3 rhshdp04.lab.eng.blr.redhat.com:/glance1/s4

volume create: glance-vol: failed: Host rhshdp02.lab.eng.blr.redhat.com is not in 'Peer in Cluster' state

[root@rhshdp01 ~]# gluster peer probe rhshdp02.lab.eng.blr.redhat.com
peer probe: success.
[root@rhshdp01 ~]# gluster peer probe rhshdp03.lab.eng.blr.redhat.com
peer probe: success.
[root@rhshdp01 ~]# gluster peer probe rhshdp04.lab.eng.blr.redhat.com
peer probe: success.

[root@rhshdp01 ~]# gluster volume create glance-vol replica 2 rhshdp01.lab.eng.blr.redhat.com:/glance1/s1 rhshdp02.lab.eng.blr.redhat.com:/glance1/s2 rhshdp03.lab.eng.blr.redhat.com:/glance1/s3 rhshdp04.lab.eng.blr.redhat.com:/glance1/s4

volume create: glance-vol: failed: /glance1/s1 is already part of a volume

[root@rhshdp01 ~]# gluster volume info
No volumes present

[root@rhshdp01 ~]# gluster peer status
Number of Peers: 3

Hostname: rhshdp02.lab.eng.blr.redhat.com
Uuid: b728d5fd-66c1-4dc8-8a33-7ee915275a19
State: Peer in Cluster (Connected)

Hostname: rhshdp03.lab.eng.blr.redhat.com
Uuid: 278b9ce9-ee45-4a89-83d0-194e00e98189
State: Peer in Cluster (Connected)

Hostname: rhshdp04.lab.eng.blr.redhat.com
Uuid: 9a0a3c6f-5724-4c81-ba49-4cc0d31cf817
State: Peer in Cluster (Connected)

[root@rhshdp01 ~]# getfattr -d -m . -e hex /glance1/s1/
getfattr: Removing leading '/' from absolute path names
# file: glance1/s1/
trusted.glusterfs.volume-id=0x4e6f1356626e4768bb01a9191b7da2ac

[root@rhshdp01 ~]#