Bug 949625
| Summary: | Peer rejected after upgrading | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | ivano.talamo |
| Component: | unclassified | Assignee: | Kaushal <kaushal> |
| Status: | CLOSED DEFERRED | QA Contact: | |
| Severity: | unspecified | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.3.1 | CC: | bugs, kaushal, vnosov |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2014-12-14 19:40:30 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
ivano.talamo
2013-04-08 15:58:59 UTC
Can you check if the volfiles for volume 'gluster-test', in /var/lib/glusterd/vols/ , are the same on test{02,03,04} (after the probe fails)? If there are any differences, please post them here.
Hi,
the files on /var/lib/glusterd/vols/gluster-test are the same on test{02,03}.
While some of them differ on test04:
[root@t2-test03 gluster-test]# md5sum *
md5sum: bricks: Is a directory
5dfe5b652e3539efa37fc1c8801ea66b cksum
ea87d872aa0870f358dbd8ee03bd8a5c gluster-test-fuse.vol
086f16c89d72fc639876ef2b5d4876cd gluster-test.t2-test02.root-b1.vol
086f16c89d72fc639876ef2b5d4876cd gluster-test.t2-test03.root-b1.vol
27446113de33c6112d41c4306d24e776 info
f1b55d145d2987c1b23c80d3dcc689ed node_state.info
7539d230a861bcba000f71047da6b2b4 rbstate
[root@t2-test04 gluster-test]# md5sum *
md5sum: bricks: Is a directory
4117326b54141c43d5f9f34fc15334c0 cksum
ecdc91a2c769a2867a367946ef2b8897 gluster-test-fuse.vol
086f16c89d72fc639876ef2b5d4876cd gluster-test.t2-test02.root-b1.vol
086f16c89d72fc639876ef2b5d4876cd gluster-test.t2-test03.root-b1.vol
795e55064584312169ebc12f70b0f234 info
f1b55d145d2987c1b23c80d3dcc689ed node_state.info
7539d230a861bcba000f71047da6b2b4 rbstate
ecdc91a2c769a2867a367946ef2b8897 trusted-gluster-test-fuse.vol
Here are the diffs:
[root@t2-test04 ~]# diff gluster-test-t2-test03/cksum /var/lib/glusterd/vols/gluster-test/cksum
1c1
< info=600099517
---
> info=2600514254
[root@t2-test04 ~]# diff gluster-test-t2-test03/gluster-test-fuse.vol /var/lib/glusterd/vols/gluster-test/gluster-test-fuse.vol
40,41c40,41
< volume gluster-test-stat-prefetch
< type performance/stat-prefetch
---
> volume gluster-test-md-cache
> type performance/md-cache
49c49
< subvolumes gluster-test-stat-prefetch
---
> subvolumes gluster-test-md-cache
[root@t2-test04 ~]# diff gluster-test-t2-test03/info /var/lib/glusterd/vols/gluster-test/info
4a5,6
> stripe_count=1
> replica_count=1
Furthermore, only on t2-test04, there's the file trusted-gluster-test-fuse.vol.
Thanks for the update. Gluster v3.3 brings some changes to the volfiles and some options. So, on upgrade from 3.2 to 3.3, these files need to have been regenerated. If the upgrade had been done via rpms, the regeneration should have happened correctly. Since the regeneration hasn't happened, you either did a source install or the rpms you used were faulty (in which case please provide details on the rpms). When you added a new peer to the cluster, it got the volume details from the original peers, but when it saved them to disk it used the newer format. This caused the checksum mismatch, which lead to it being Rejected. To solve this problem, perform step 5 from this article http://vbellur.wordpress.com/2012/05/31/upgrading-to-glusterfs-3-3/ , on the original 2 peers. That should regenerate the volfiles and solve your problems. Hi Kaushal, thank you! I made the procedure you suggested and yes, now the probe was successful. The repo file in the servers is configured to take RPMs from here: http://download.gluster.org/pub/gluster/glusterfs/3.3/3.3.1/EPEL.repo/epel-5/x86_64/ Is it the right one? Anyway a second problem came out. After adding to the existing distributed volume a brick from the new server, all the files disappeared from gluster. If I do an ls I can't see anything, although I can read/write the files giving the exact path and the files are into the bricks. I also tried to restart the gluster daemon and to restart the server but nothing changed... Thanks, Ivano Hi Ivano, It seems like there are problems with the rpms. We'll need to check the rpms to see what caused it. Your new problem shouldn't be happening. Can you give more information (what you did after the upgrade basically) and logs of both servers and the client. I'd suggest that you file a new bug, if this turns out to be a valid issue. Regards, Kaushal The version that this bug has been reported against, does not get any updates from the Gluster Community anymore. Please verify if this report is still valid against a current (3.4, 3.5 or 3.6) release and update the version, or close this bug. If there has been no update before 9 December 2014, this bug will get automatocally closed. |