Bug 1696721 - geo-replication failing after upgrade from 5.5 to 6.0
Summary: geo-replication failing after upgrade from 5.5 to 6.0
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: GlusterFS
Classification: Community
Component: geo-replication
Version: 6
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Sunny Kumar
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-05 13:50 UTC by Chad Cropper
Modified: 2019-09-11 06:22 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2019-09-11 06:22:24 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Chad Cropper 2019-04-05 13:50:20 UTC
Description of problem:
After upgrading Gluster from 5.5 to 6.0, geo-replication stays in initialized status permanently.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Stop all gluster vols/services
2. Upgrade RPMs form 5.5. to 6.0
3. Start all gluster vols/services
4. geo-repl will not start

Actual results:
Not moving past initializing state

Expected results:
geo-replication moves to active/changelog crawl

Additional info:
Last part of log
[2019-04-05 13:38:07.052740] I [socket.c:811:__socket_shutdown] 0-geovol-client-1: intentional socket shutdown(13)
[2019-04-05 13:38:07.053331] W [dict.c:986:str_to_data] (-->/usr/lib64/glusterfs/6.0/xlator/protocol/client.so(+0x40f3a) [0x7f42fb706f3a] -->/lib64/libglusterfs.so.0(dict_set_str+0x16) [0x7f4309ed8bb6] -->/lib64/libglusterfs.so.0(str_to_data+0x71) [0x7f4309ed54d1] ) 0-dict: value is NULL [Invalid argument]
[2019-04-05 13:38:07.053385] I [MSGID: 114006] [client-handshake.c:1238:client_setvolume] 0-geovol-client-0: failed to set process-name in handshake msg
[2019-04-05 13:38:07.053463] W [dict.c:986:str_to_data] (-->/usr/lib64/glusterfs/6.0/xlator/protocol/client.so(+0x40f3a) [0x7f42fb706f3a] -->/lib64/libglusterfs.so.0(dict_set_str+0x16) [0x7f4309ed8bb6] -->/lib64/libglusterfs.so.0(str_to_data+0x71) [0x7f4309ed54d1] ) 0-dict: value is NULL [Invalid argument]
[2019-04-05 13:38:07.053493] I [MSGID: 114006] [client-handshake.c:1238:client_setvolume] 0-geovol-client-1: failed to set process-name in handshake msg
[2019-04-05 13:38:07.054023] I [MSGID: 114046] [client-handshake.c:1107:client_setvolume_cbk] 0-geovol-client-0: Connected to geovol-client-0, attached to remote volume '/glusterfs/geovol_b1/brick'.
[2019-04-05 13:38:07.054314] I [MSGID: 114046] [client-handshake.c:1107:client_setvolume_cbk] 0-geovol-client-1: Connected to geovol-client-1, attached to remote volume '/glusterfs/geovol_b1/brick'.
[2019-04-05 13:38:07.056436] I [fuse-bridge.c:5142:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.24 kernel 7.22
[2019-04-05 13:38:07.056467] I [fuse-bridge.c:5753:fuse_graph_sync] 0-fuse: switched to graph 0
[2019-04-05 13:39:32.048849] C [rpc-clnt-ping.c:155:rpc_clnt_ping_timer_expired] 0-geovol-client-1: server 192.168.xxx.xxx:49153 has not responded in the last 42 seconds, disconnecting.
[2019-04-05 13:39:32.048952] I [socket.c:811:__socket_shutdown] 0-geovol-client-1: intentional socket shutdown(10)

Comment 1 Sunny Kumar 2019-04-15 08:47:52 UTC
Hi,

Can you please check all brick status, it looks like brick/s is/are not up.
Please do a force gluster vol start it should work.

If it does not please share all logs form master and slave volumes.

Comment 2 Amar Tumballi 2019-06-18 09:33:32 UTC
Hi Chad, did you get a chance to try Sunny's suggestion?

Comment 3 Sunny Kumar 2019-09-11 06:22:24 UTC
Closing this bug as I am not seeing this behavior.


Note You need to log in before you can comment on or make changes to this bug.