Bug 1154019
| Summary: | DHT: Rebalance- Rebalance doesn't start if we have any mounts of the volume | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | shylesh <shmohan> |
| Component: | distribute | Assignee: | Nithya Balachandran <nbalacha> |
| Status: | CLOSED ERRATA | QA Contact: | Amit Chaurasia <achauras> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 2.1 | CC: | achauras, mzywusko, nbalacha, rcyriac, surs, vagarwal |
| Target Milestone: | --- | Keywords: | ZStream |
| Target Release: | RHGS 2.1.5 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | glusterfs-3.4.0.70rhs-1 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2014-11-13 12:23:38 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1147095 | ||
|
Description
shylesh
2014-10-17 09:30:44 UTC
This happens only with fuse mount not NFS This bug also affects quota. Since quota does a temporary mount whenever enabled, so user will not be able to do "rebalance start" however user can use "start force" to trigger rebalance. Verified this bug on the latest build of *.70.* as shown below :
[root@dht-rhs-20 ~]# rpm -qa | grep -i gluster
gluster-swift-container-1.10.0-2.el6rhs.noarch
glusterfs-fuse-3.4.0.70rhs-1.el6rhs.x86_64
gluster-swift-1.10.0-2.el6rhs.noarch
glusterfs-libs-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-geo-replication-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-api-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-rdma-3.4.0.70rhs-1.el6rhs.x86_64
gluster-swift-proxy-1.10.0-2.el6rhs.noarch
gluster-swift-account-1.10.0-2.el6rhs.noarch
gluster-swift-plugin-1.10.0-5.el6rhs.noarch
vdsm-gluster-4.13.0-24.el6rhs.noarch
gluster-swift-object-1.10.0-2.el6rhs.noarch
samba-glusterfs-3.6.9-167.10.el6rhs.x86_64
glusterfs-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-server-3.4.0.70rhs-1.el6rhs.x86_64
[root@dht-rhs-20 ~]#
Steps followed to verify:
1. Created a volume, mounted it both with Fuse and NFS.
2. Added bricks to it.
3. Started rebalance after adding the brick.
4. Rebalance started without any error message.
5. Further, enabled the quota on the volume and specified a limit to a folder and initiated rebalance again.
6. Rebalance finished successfully without any errors.
The screenshots of the verification steps:
---------------------------------------------------------------------
[root@dht-rhs-20 ~]# rpm -qa | grep -i gluster
gluster-swift-container-1.10.0-2.el6rhs.noarch
glusterfs-fuse-3.4.0.70rhs-1.el6rhs.x86_64
gluster-swift-1.10.0-2.el6rhs.noarch
glusterfs-libs-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-geo-replication-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-api-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-rdma-3.4.0.70rhs-1.el6rhs.x86_64
gluster-swift-proxy-1.10.0-2.el6rhs.noarch
gluster-swift-account-1.10.0-2.el6rhs.noarch
gluster-swift-plugin-1.10.0-5.el6rhs.noarch
vdsm-gluster-4.13.0-24.el6rhs.noarch
gluster-swift-object-1.10.0-2.el6rhs.noarch
samba-glusterfs-3.6.9-167.10.el6rhs.x86_64
glusterfs-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-server-3.4.0.70rhs-1.el6rhs.x86_64
[root@dht-rhs-20 ~]# mount
/dev/vda2 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/vda1 on /boot type ext4 (rw)
/dev/vdb1 on /data/newbrick1 type xfs (rw)
/dev/vdb2 on /data/newbrick2 type xfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
localhost:gv0 on /var/run/gluster/gv0 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)
10.70.47.101:/gv0 on /mnt/gvmnt type nfs (rw,addr=10.70.47.101)
[root@dht-rhs-20 ~]# gluster volume info
Volume Name: gv0
Type: Distribute
Volume ID: e9a0d5b5-c043-47e6-9803-20c78d3b79a0
Status: Started
Number of Bricks: 11
Transport-type: tcp
Bricks:
Brick1: 10.70.47.99:/data/newbrick1/gv0
Brick2: 10.70.47.101:/data/newbrick1/gv0
Brick3: 10.70.47.99:/data/newbrick2/br1
Brick4: 10.70.47.99:/data/newbrick2/br2
Brick5: 10.70.47.99:/data/newbrick2/br3
Brick6: 10.70.47.101:/data/newbrick2/br1
Brick7: 10.70.47.101:/data/newbrick2/br2
Brick8: 10.70.47.101:/data/newbrick2/br3
Brick9: 10.70.47.99:/data/newbrick2/br4
Brick10: 10.70.47.101:/data/newbrick2/br4
Brick11: 10.70.47.99:/data/newbrick2/brick-100
Options Reconfigured:
features.quota: on
[root@dht-rhs-20 ~]# gluster volume add-brick gv0 10.70.47.101:/data/newbrick2/brick-100 force
volume add-brick: success
[root@dht-rhs-20 ~]#
[root@dht-rhs-20 ~]# gluster volume info
Volume Name: gv0
Type: Distribute
Volume ID: e9a0d5b5-c043-47e6-9803-20c78d3b79a0
Status: Started
Number of Bricks: 12
Transport-type: tcp
Bricks:
Brick1: 10.70.47.99:/data/newbrick1/gv0
Brick2: 10.70.47.101:/data/newbrick1/gv0
Brick3: 10.70.47.99:/data/newbrick2/br1
Brick4: 10.70.47.99:/data/newbrick2/br2
Brick5: 10.70.47.99:/data/newbrick2/br3
Brick6: 10.70.47.101:/data/newbrick2/br1
Brick7: 10.70.47.101:/data/newbrick2/br2
Brick8: 10.70.47.101:/data/newbrick2/br3
Brick9: 10.70.47.99:/data/newbrick2/br4
Brick10: 10.70.47.101:/data/newbrick2/br4
Brick11: 10.70.47.99:/data/newbrick2/brick-100
Brick12: 10.70.47.101:/data/newbrick2/brick-100
Options Reconfigured:
features.quota: on
[root@dht-rhs-20 ~]# gluster volume rebalance gv0 start
volume rebalance: gv0: success: Starting rebalance on volume gv0 has been successful.
ID: a701172a-0047-4e89-951b-d9ff95ceb057
[root@dht-rhs-20 ~]#
[root@dht-rhs-20 ~]# gluster volume rebalance gv0 status
Node Rebalanced-files size scanned failures skipped status run time in secs
--------- ----------- ----------- ----------- ----------- ----------- ------------ --------------
localhost 111 4.2MB 275 0 0 in progress 5.00
10.70.47.101 101 3.9MB 238 0 0 in progress 5.00
volume rebalance: gv0: success:
[root@dht-rhs-20 ~]# gluster volume rebalance gv0 status
Node Rebalanced-files size scanned failures skipped status run time in secs
--------- ----------- ----------- ----------- ----------- ----------- ------------ --------------
localhost 125 4.8MB 329 0 0 completed 10.00
10.70.47.101 148 5.6MB 307 0 0 completed 10.00
volume rebalance: gv0: success:
[root@dht-rhs-20 ~]# mkdir /data/newbrick2/brick-200
[root@dht-rhs-20 ~]# gluster volume add-brick gv0 10.70.47.101:/data/newbrick2/brick-200 force
volume add-brick: success
[root@dht-rhs-20 ~]# gluster volume add-brick gv0 10.70.47.99:/data/newbrick2/brick-200 force
volume add-brick: success
[root@dht-rhs-20 ~]#
[root@dht-rhs-20 ~]#
[root@dht-rhs-20 ~]# mount
/dev/vda2 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/vda1 on /boot type ext4 (rw)
/dev/vdb1 on /data/newbrick1 type xfs (rw)
/dev/vdb2 on /data/newbrick2 type xfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
localhost:gv0 on /var/run/gluster/gv0 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)
10.70.47.101:/gv0 on /mnt/gvmnt type nfs (rw,addr=10.70.47.101)
[root@dht-rhs-20 ~]#
[root@dht-rhs-20 ~]# rpm -qa | grep -i gluster
gluster-swift-container-1.10.0-2.el6rhs.noarch
glusterfs-fuse-3.4.0.70rhs-1.el6rhs.x86_64
gluster-swift-1.10.0-2.el6rhs.noarch
glusterfs-libs-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-geo-replication-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-api-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-rdma-3.4.0.70rhs-1.el6rhs.x86_64
gluster-swift-proxy-1.10.0-2.el6rhs.noarch
gluster-swift-account-1.10.0-2.el6rhs.noarch
gluster-swift-plugin-1.10.0-5.el6rhs.noarch
vdsm-gluster-4.13.0-24.el6rhs.noarch
gluster-swift-object-1.10.0-2.el6rhs.noarch
samba-glusterfs-3.6.9-167.10.el6rhs.x86_64
glusterfs-3.4.0.70rhs-1.el6rhs.x86_64
glusterfs-server-3.4.0.70rhs-1.el6rhs.x86_64
root@dht-rhs-20 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda2 34G 2.5G 30G 8% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/vda1 485M 59M 401M 13% /boot
/dev/vdb1 20G 34M 20G 1% /data/newbrick1
/dev/vdb2 20G 41M 20G 1% /data/newbrick2
localhost:gv0 300G 595M 300G 1% /var/run/gluster/gv0
10.70.47.101:/gv0 300G 595M 300G 1% /mnt/gvmnt
[root@dht-rhs-20 ~]#
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2014-1853.html |