Bug 1730953
Summary: | mount generates errors after umount | ||
---|---|---|---|
Product: | [Community] GlusterFS | Reporter: | Amgad <amgad.saleh> |
Component: | core | Assignee: | Amar Tumballi <atumball> |
Status: | CLOSED NEXTRELEASE | QA Contact: | |
Severity: | urgent | Docs Contact: | |
Priority: | high | ||
Version: | mainline | CC: | atumball, bugs |
Target Milestone: | --- | Keywords: | EasyFix, StudentProject |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2019-07-23 03:22:21 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Amgad
2019-07-18 02:54:47 UTC
Description of problem: mount a volume after being umount introduces errors, however the mount was successful Version-Release number of selected component (if applicable): 5.5-1 How reproducible: Had 3-cluster nodes. Created a volume and /etc/fstab had the following entry to make it persistent 10.76.155.233:/bcmt-glusterfs /opt/bcmt/storage glusterfs backup-volfile-servers=10.76.155.237:10.76.155.236:10.76.155.233,_netdev 0 0 [root@gfs3 tasks]# umount /opt/bcmt/storage Steps to Reproduce: 1. umount /opt/bcmt/storage 2. mount /opt/bcmt/storage Actual results: [root@gfs3 bricks]# [root@gfs3 bricks]# umount /opt/bcmt/storage glusterfs-client log entry: ------------------------- [2019-07-18 02:29:30.968978] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-bcmt-glusterfs-server: disconnecting connection from CTX_ID:4a34475c-9838-447f-93b2-b5ba33096c9b-GRAPH_ID:0-PID:24514-HOST:gfs3-PC_NAME:bcmt-glusterfs-client-2-RECON_NO:-0 [2019-07-18 02:29:30.969162] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-bcmt-glusterfs-server: Shutting down connection CTX_ID:4a34475c-9838-447f-93b2-b5ba33096c9b-GRAPH_ID:0-PID:24514-HOST:gfs3-PC_NAME:bcmt-glusterfs-client-2-RECON_NO:-0 [root@gfs3 bricks]# mount /opt/bcmt/storage [2019-07-18 02:30:09.182263] E [glusterfsd.c:783:gf_remember_backup_volfile_server] 0-glusterfs: failed to set volfile server: File exists glusterfs-client log entry: -------------------------- [2019-07-18 02:30:09.182263] E [glusterfsd.c:783:gf_remember_backup_volfile_server] 0-glusterfs: failed to set volfile server: File exists [2019-07-18 02:30:09.218383] I [addr.c:54:compare_addr_and_update] 0-/data0/glusterfs: allowed = "*", received addr = "10.76.155.233" [2019-07-18 02:30:09.218430] I [login.c:110:gf_auth] 0-auth/login: allowed user names: f9f0cc51-20cf-42ba-8288-257d57f28bcb [2019-07-18 02:30:09.218443] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-bcmt-glusterfs-server: accepted client from CTX_ID:1b3a75a1-5868-43a2-b2c2-7e79cbb88f01-GRAPH_ID:0-PID:24762-HOST:gfs3-PC_NAME:bcmt-glusterfs-client-2-RECON_NO:-0 (version: 5.5) [root@gfs3 bricks]# Expected results: Should not have the errors Additional info: Expected results: Additional info: Description of problem: mount a volume after being umount introduces errors, however the mount was successful Version-Release number of selected component (if applicable): 5.5-1 How reproducible: Had 3-cluster nodes. Created a volume and /etc/fstab had the following entry to make it persistent 10.76.155.233:/bcmt-glusterfs /opt/bcmt/storage glusterfs backup-volfile-servers=10.76.155.237:10.76.155.236:10.76.155.233,_netdev 0 0 [root@gfs3 tasks]# umount /opt/bcmt/storage Steps to Reproduce: 1. umount /opt/bcmt/storage 2. mount /opt/bcmt/storage Actual results: [root@gfs3 bricks]# [root@gfs3 bricks]# umount /opt/bcmt/storage glusterfs-client log entry: ------------------------- [2019-07-18 02:29:30.968978] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-bcmt-glusterfs-server: disconnecting connection from CTX_ID:4a34475c-9838-447f-93b2-b5ba33096c9b-GRAPH_ID:0-PID:24514-HOST:gfs3-PC_NAME:bcmt-glusterfs-client-2-RECON_NO:-0 [2019-07-18 02:29:30.969162] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-bcmt-glusterfs-server: Shutting down connection CTX_ID:4a34475c-9838-447f-93b2-b5ba33096c9b-GRAPH_ID:0-PID:24514-HOST:gfs3-PC_NAME:bcmt-glusterfs-client-2-RECON_NO:-0 [root@gfs3 bricks]# mount /opt/bcmt/storage [2019-07-18 02:30:09.182263] E [glusterfsd.c:783:gf_remember_backup_volfile_server] 0-glusterfs: failed to set volfile server: File exists glusterfs-client log entry: -------------------------- [2019-07-18 02:30:09.182263] E [glusterfsd.c:783:gf_remember_backup_volfile_server] 0-glusterfs: failed to set volfile server: File exists [2019-07-18 02:30:09.218383] I [addr.c:54:compare_addr_and_update] 0-/data0/glusterfs: allowed = "*", received addr = "10.76.155.233" [2019-07-18 02:30:09.218430] I [login.c:110:gf_auth] 0-auth/login: allowed user names: f9f0cc51-20cf-42ba-8288-257d57f28bcb [2019-07-18 02:30:09.218443] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-bcmt-glusterfs-server: accepted client from CTX_ID:1b3a75a1-5868-43a2-b2c2-7e79cbb88f01-GRAPH_ID:0-PID:24762-HOST:gfs3-PC_NAME:bcmt-glusterfs-client-2-RECON_NO:-0 (version: 5.5) [root@gfs3 bricks]# Expected results: Should not have the errors Additional info: Expected results: Additional info: Can you try below line? '10.76.155.233:/bcmt-glusterfs /opt/bcmt/storage glusterfs backup-volfile-servers=10.76.155.237:10.76.155.236,_netdev 0 0' (change is, no addition of 10.76.155.233 in backup-volfile-servers). Let us know if that fixes the problem. Thanks Amar -- removing the "10.76.155.233" fixed the problem! Does that mean the "backup-volfile-servers" should not include all the servers? The problem is we use a DNS name (glusterfs-server) (instead of IPs) that resolves to any of the three servers in case one is down. That's why needed all servers to be backup -- any suggestion? --> 'glusterfs-server:/bcmt-glusterfs /opt/bcmt/storage glusterfs backup-volfile-servers=10.76.155.237:10.76.155.236,10.76.155.233,_netdev 0 0' Thanks for confirming. The request seems genuine, instead of erroring out a duplicate entry in backup-volfile-server (it happens because 'gluster-server' resolved to one of the IP, and it was not expecting it in backup-volfile-servers. (as it was given as primary server itself). For now, continue with the fix. I will see if we can fix the issue in code, and have a backport done. REVIEW: https://review.gluster.org/23087 (backup-volfile-server: ignore duplicate entries) posted (#1) for review on master by Amar Tumballi REVIEW: https://review.gluster.org/23087 (backup-volfile-server: ignore duplicate entries) merged (#2) on master by Amar Tumballi |