1452766 – VM crashing but there's no apparent reason

Bug 1452766 - VM crashing but there's no apparent reason

Summary: VM crashing but there's no apparent reason

Keywords:
Status:	CLOSED EOL
Alias:	None
Product:	GlusterFS
Classification:	Community
Component:	core
Sub Component:
Version:	3.8
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	high
Target Milestone:	---
Assignee:	bugs@gluster.org
QA Contact:
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2017-05-19 15:05 UTC by Alessandro Briosi
Modified:	2017-11-07 10:35 UTC (History)
CC List:	3 users (show)
Fixed In Version:
Clone Of:
Environment:
Last Closed:	2017-11-07 10:35:48 UTC
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Embargoed:
Dependent Products:

Attachments	(Terms of Use)
GDB KVM log (37.02 KB, text/plain) 2017-05-25 06:53 UTC, Alessandro Briosi	no flags	Details
View All

Description Alessandro Briosi 2017-05-19 15:05:31 UTC

Description of problem:

VM on proxmox cluster which is also gluster server crashes.
The logs do not really report any reason (at least for me)

Gluster version is 3.8.11

How reproducible:
I have not identified any way to reproduce it constantly. It crashes randomly.

Additional info:

The brick log report the following:
[2017-05-19 14:40:25.264115] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-81350-2017/05/10-09:04:55:966276-da
tastore1-client-0-0-0
[2017-05-19 14:40:25.264220] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-81350-2017/05/10-09:04:55:912580-da
tastore1-client-0-0-0
[2017-05-19 14:40:25.264229] W [inodelk.c:399:pl_inodelk_log_cleanup] 0-datastore1-server: releasing lock on 9e66f0d2-501b-4cf9-80db-f423e2e2ef0f held by {client=0x7ffa841014
e0, pid=0 lk-owner=5c807b04f67f0000}
[2017-05-19 14:40:25.264246] W [inodelk.c:399:pl_inodelk_log_cleanup] 0-datastore1-server: releasing lock on bc8f6a7e-31e5-4b48-946c-f779a4b2e64f held by {client=0x7ffa7c005d
20, pid=0 lk-owner=5cd0180df67f0000}
[2017-05-19 14:40:25.264255] I [MSGID: 115013] [server-helpers.c:293:do_fd_cleanup] 0-datastore1-server: fd cleanup on /images/101/vm-101-disk-2.qcow2
[2017-05-19 14:40:25.264259] I [MSGID: 115013] [server-helpers.c:293:do_fd_cleanup] 0-datastore1-server: fd cleanup on /images/101/vm-101-disk-1.qcow2
[2017-05-19 14:40:25.264332] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-81350-2017/05/10-09:04:55:966276-datasto
re1-client-0-0-0
[2017-05-19 14:40:25.264339] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-81350-2017/05/10-09:04:55:912580-datasto
re1-client-0-0-0
[2017-05-19 14:48:15.756077] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:48:15.756158] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144293-2017/05/19-14:48:15:661020-d
atastore1-client-0-0-0 (version: 3.8.11)
[2017-05-19 14:48:15.782046] E [MSGID: 113107] [posix.c:1051:posix_seek] 0-datastore1-posix: seek failed on fd 20 length 539669364736 [No such device or address]
[2017-05-19 14:48:15.782084] E [MSGID: 115089] [server-rpc-fops.c:2007:server_seek_cbk] 0-datastore1-server: 18: SEEK-2 (bc8f6a7e-31e5-4b48-946c-f779a4b2e64f) ==> (No such de
vice or address) [No such device or address]
[2017-05-19 14:48:16.853352] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:48:16.853376] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144293-2017/05/19-14:48:16:834981-d
atastore1-client-0-0-0 (version: 3.8.11)
[2017-05-19 14:48:16.875513] E [MSGID: 113107] [posix.c:1051:posix_seek] 0-datastore1-posix: seek failed on fd 21 length 966582009856 [No such device or address]
[2017-05-19 14:48:16.875547] E [MSGID: 115089] [server-rpc-fops.c:2007:server_seek_cbk] 0-datastore1-server: 18: SEEK-2 (9e66f0d2-501b-4cf9-80db-f423e2e2ef0f) ==> (No such de
vice or address) [No such device or address]
[2017-05-19 14:50:42.399211] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-144293-2017/05/19-14:48:16:834981-d
atastore1-client-0-0-0
[2017-05-19 14:50:42.399261] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-144293-2017/05/19-14:48:16:834981-datast
ore1-client-0-0-0
[2017-05-19 14:50:42.399258] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-144293-2017/05/19-14:48:15:661020-d
atastore1-client-0-0-0
[2017-05-19 14:50:42.399290] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-144293-2017/05/19-14:48:15:661020-datast
ore1-client-0-0-0
[2017-05-19 14:50:43.385444] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:50:43.385464] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144530-2017/05/19-14:50:43:303790-d
atastore1-client-0-0-0 (version: 3.8.11)
[2017-05-19 14:50:43.411762] E [MSGID: 113107] [posix.c:1051:posix_seek] 0-datastore1-posix: seek failed on fd 20 length 539669364736 [No such device or address]
[2017-05-19 14:50:43.411803] E [MSGID: 115089] [server-rpc-fops.c:2007:server_seek_cbk] 0-datastore1-server: 18: SEEK-2 (bc8f6a7e-31e5-4b48-946c-f779a4b2e64f) ==> (No such de
vice or address) [No such device or address]
[2017-05-19 14:50:44.030125] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:50:44.030149] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144530-2017/05/19-14:50:44:15375-da
tastore1-client-0-0-0 (version: 3.8.11)
[2017-05-19 14:50:44.056989] E [MSGID: 113107] [posix.c:1051:posix_seek] 0-datastore1-posix: seek failed on fd 21 length 966582009856 [No such device or address]
[2017-05-19 14:50:44.057016] E [MSGID: 115089] [server-rpc-fops.c:2007:server_seek_cbk] 0-datastore1-server: 18: SEEK-2 (9e66f0d2-501b-4cf9-80db-f423e2e2ef0f) ==> (No such de
vice or address) [No such device or address]
[2017-05-19 14:51:09.079185] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-144530-2017/05/19-14:50:43:303790-d
atastore1-client-0-0-0
[2017-05-19 14:51:09.079186] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-144530-2017/05/19-14:50:44:15375-da
tastore1-client-0-0-0
[2017-05-19 14:51:09.079284] I [MSGID: 115013] [server-helpers.c:293:do_fd_cleanup] 0-datastore1-server: fd cleanup on /images/101/vm-101-disk-1.qcow2
[2017-05-19 14:51:09.079285] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-144530-2017/05/19-14:50:44:15375-datasto
re1-client-0-0-0
[2017-05-19 14:51:09.079325] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-144530-2017/05/19-14:50:43:303790-datast
ore1-client-0-0-0
[2017-05-19 14:51:37.825420] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:51:37.825449] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144656-2017/05/19-14:51:37:744255-d
atastore1-client-0-0-0 (version: 3.8.11)
[2017-05-19 14:51:37.850244] E [MSGID: 113107] [posix.c:1051:posix_seek] 0-datastore1-posix: seek failed on fd 20 length 539669364736 [No such device or address]
[2017-05-19 14:51:37.850291] E [MSGID: 115089] [server-rpc-fops.c:2007:server_seek_cbk] 0-datastore1-server: 18: SEEK-2 (bc8f6a7e-31e5-4b48-946c-f779a4b2e64f) ==> (No such de
vice or address) [No such device or address]
[2017-05-19 14:51:38.083555] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:51:38.083576] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144656-2017/05/19-14:51:38:66829-da
tastore1-client-0-0-0 (version: 3.8.11)
[2017-05-19 14:51:38.108907] E [MSGID: 113107] [posix.c:1051:posix_seek] 0-datastore1-posix: seek failed on fd 21 length 966582009856 [No such device or address]
[2017-05-19 14:51:38.108940] E [MSGID: 115089] [server-rpc-fops.c:2007:server_seek_cbk] 0-datastore1-server: 18: SEEK-2 (9e66f0d2-501b-4cf9-80db-f423e2e2ef0f) ==> (No such de
vice or address) [No such device or address]

This time I also get:
root@srvpve1:/var/log# gluster volume heal datastore1 info
Brick srvpve1g:/data/brick1/brick
/images/101/vm-101-disk-1.qcow2 - Possibly undergoing heal

/images/101/vm-101-disk-2.qcow2 - Possibly undergoing heal

Status: Connected
Number of entries: 2

Brick srvpve2g:/data/brick1/brick
/images/101/vm-101-disk-1.qcow2 - Possibly undergoing heal

/images/101/vm-101-disk-2.qcow2 - Possibly undergoing heal

Status: Connected
Number of entries: 2

Brick srvpve3g:/data/brick1/brick
/images/101/vm-101-disk-1.qcow2 - Possibly undergoing heal

/images/101/vm-101-disk-2.qcow2 - Possibly undergoing heal

Status: Connected
Number of entries: 2

Comment 1 Alessandro Briosi 2017-05-19 15:32:43 UTC

this is a brief description of my setup.

3 dell servers with RAID 10 SAS Disks
each server has 2 bonded 1Gbps ethernets dedicated to gluster (2 dedicated to the proxmox cluster and 2 for comunication with the hosts on the LAN) (each on it's VLAN in the switch)
Also jumbo frames are enabled on ethernets and switches.

each server is a proxmox host which has gluster installed and configured as server and client.

The RAID has a LVM thin provisioned which is divided into 3 bricks (2 big for the data and 1 small for the arbiter).
each Thin LVM is XFS formatted and mounted as brick.
There are 3 volumes configured which replicate 3 with arbiter (so 2 really holding the data).
Volumes are:
datastore1: data on srv1 and srv2, arbiter srv3
datastore2: data on srv2 and srv3, arbiter srv1
datastore3: data on srv1 and srv3, arbiter srv2

On each datastore basically there is a main VM (plus some others which though are not so important). (3 VM are mainly important)

datastore1 was converted from 2 replica to 3 replica with arbiter, the other 2 were created as described.

Comment 2 Alessandro Briosi 2017-05-19 15:37:25 UTC

This is the brick log on the other server. Are those disconnects/reconnects normal? 

[2017-05-19 14:40:25.262715] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-81350-2017/05/10-09:04:55:966276-da
tastore1-client-1-0-0
[2017-05-19 14:40:25.262766] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-81350-2017/05/10-09:04:55:912580-da
tastore1-client-1-0-0
[2017-05-19 14:40:25.262775] W [inodelk.c:399:pl_inodelk_log_cleanup] 0-datastore1-server: releasing lock on 9e66f0d2-501b-4cf9-80db-f423e2e2ef0f held by {client=0x7f8f980196
90, pid=0 lk-owner=5c807b04f67f0000}
[2017-05-19 14:40:25.262787] W [inodelk.c:399:pl_inodelk_log_cleanup] 0-datastore1-server: releasing lock on bc8f6a7e-31e5-4b48-946c-f779a4b2e64f held by {client=0x7f8fa00f10
e0, pid=0 lk-owner=5cd0180df67f0000}
[2017-05-19 14:40:25.262797] I [MSGID: 115013] [server-helpers.c:293:do_fd_cleanup] 0-datastore1-server: fd cleanup on /images/101/vm-101-disk-2.qcow2
[2017-05-19 14:40:25.262800] I [MSGID: 115013] [server-helpers.c:293:do_fd_cleanup] 0-datastore1-server: fd cleanup on /images/101/vm-101-disk-1.qcow2
[2017-05-19 14:40:25.262870] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-81350-2017/05/10-09:04:55:966276-datasto
re1-client-1-0-0
[2017-05-19 14:40:25.262876] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-81350-2017/05/10-09:04:55:912580-datasto
re1-client-1-0-0
[2017-05-19 14:48:15.755145] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:48:15.755175] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144293-2017/05/19-14:48:15:661020-d
atastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 14:48:16.852338] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:48:16.852365] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144293-2017/05/19-14:48:16:834981-d
atastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 14:50:42.397796] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-144293-2017/05/19-14:48:16:834981-d
atastore1-client-1-0-0
[2017-05-19 14:50:42.397838] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-144293-2017/05/19-14:48:15:661020-d
atastore1-client-1-0-0
[2017-05-19 14:50:42.397857] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-144293-2017/05/19-14:48:16:834981-datast
ore1-client-1-0-0
[2017-05-19 14:50:42.397860] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-144293-2017/05/19-14:48:15:661020-datast
ore1-client-1-0-0
[2017-05-19 14:50:43.384883] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:50:43.384926] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144530-2017/05/19-14:50:43:303790-d
atastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 14:50:44.029387] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:50:44.029408] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144530-2017/05/19-14:50:44:15375-da
tastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 14:51:09.077723] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-144530-2017/05/19-14:50:44:15375-da
tastore1-client-1-0-0
[2017-05-19 14:51:09.077784] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-144530-2017/05/19-14:50:44:15375-datasto
re1-client-1-0-0
[2017-05-19 14:51:09.077778] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-144530-2017/05/19-14:50:43:303790-d
atastore1-client-1-0-0
[2017-05-19 14:51:09.077846] I [MSGID: 115013] [server-helpers.c:293:do_fd_cleanup] 0-datastore1-server: fd cleanup on /images/101/vm-101-disk-1.qcow2
[2017-05-19 14:51:09.077929] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-144530-2017/05/19-14:50:43:303790-datast
ore1-client-1-0-0
[2017-05-19 14:51:37.824379] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:51:37.824410] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144656-2017/05/19-14:51:37:744255-d
atastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 14:51:38.082608] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:51:38.082629] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-144656-2017/05/19-14:51:38:66829-da
tastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 14:57:03.854043] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 14:57:03.854076] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-145199-2017/05/19-14:57:03:839698-d
atastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 14:57:04.521427] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-145199-2017/05/19-14:57:03:839698-d
atastore1-client-1-0-0
[2017-05-19 14:57:04.521467] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-145199-2017/05/19-14:57:03:839698-datast
ore1-client-1-0-0
[2017-05-19 15:03:43.823915] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 15:03:43.823966] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-145657-2017/05/19-15:03:43:808931-d
atastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 15:03:43.868332] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-145657-2017/05/19-15:03:43:808931-d
atastore1-client-1-0-0
[2017-05-19 15:03:43.868359] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-145657-2017/05/19-15:03:43:808931-datast
ore1-client-1-0-0
[2017-05-19 15:03:48.976440] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 15:03:48.976488] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-145676-2017/05/19-15:03:48:956066-d
atastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 15:03:49.022813] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-145676-2017/05/19-15:03:48:956066-d
atastore1-client-1-0-0
[2017-05-19 15:03:49.022844] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-145676-2017/05/19-15:03:48:956066-datast
ore1-client-1-0-0
[2017-05-19 15:03:49.959391] I [login.c:76:gf_auth] 0-auth/login: allowed user names: b10d5e19-7220-4f42-b7c4-480eebe1f118
[2017-05-19 15:03:49.959426] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-datastore1-server: accepted client from srvpve1-145695-2017/05/19-15:03:49:943014-d
atastore1-client-1-0-0 (version: 3.8.11)
[2017-05-19 15:03:50.001231] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-datastore1-server: disconnecting connection from srvpve1-145695-2017/05/19-15:03:49:943014-d
atastore1-client-1-0-0
[2017-05-19 15:03:50.001268] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-datastore1-server: Shutting down connection srvpve1-145695-2017/05/19-15:03:49:943014-datast
ore1-client-1-0-0

Comment 3 Alessandro Briosi 2017-05-22 13:24:07 UTC

Is bug #1442933 related to this one?

Comment 4 Alessandro Briosi 2017-05-25 06:52:52 UTC

Attached the log from a gdb session of the KVM process.
The problem seems to be on a "No such file or directory" error on a assertion.

It crashed already a couple of times (as you can see from the log).

Comment 5 Alessandro Briosi 2017-05-25 06:53:42 UTC

Created attachment 1282134 [details]
GDB KVM log

Log of the VM runned with gdb and crashed more times.

Comment 6 Mattia Lambertini 2017-07-26 16:07:09 UTC

I'm experiencing what it seems the same issue.

Gluster version is 3.10.3

My current setup is quite simple, I'm starting with just one node (it's planned to add 2 more in the near future) and one volume with one brick:

Volume Name: gv0
Type: Distribute
Volume ID: ba11f016-3ae5-4fc2-b5a1-550bbcc74e39
Status: Started
Snapshot Count: 0
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: pr1-cluster:/data/brick1/gv0
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
diagnostics.brick-log-level: WARNING

Sharding is disabled.

From the brick log:

[2017-07-26 15:49:05.765830] E [MSGID: 113107] [posix.c:1080:posix_seek] 0-gv0-posix: seek failed on fd 31 length 40273903616 [No such device or address]
[2017-07-26 15:49:05.765883] E [MSGID: 115089] [server-rpc-fops.c:2070:server_seek_cbk] 0-gv0-server: 16: SEEK-2 (c66abe5a-ff28-4e4b-8391-d9a3e7aa7838), client: pr1-16895-2017/07/26-15:49:05:750324-gv0-client-0-0-0, error-xlator: gv0-posix [No such device or address]
[2017-07-26 15:49:05.768248] E [MSGID: 113107] [posix.c:1080:posix_seek] 0-gv0-posix: seek failed on fd 31 length 53687091200 [No such device or address]
[2017-07-26 15:49:05.768289] E [MSGID: 115089] [server-rpc-fops.c:2070:server_seek_cbk] 0-gv0-server: 20: SEEK-2 (1f7a84d5-8383-41f0-8816-bfd55b105f9d), client: pr1-16895-2017/07/26-15:49:05:750324-gv0-client-0-0-0, error-xlator: gv0-posix [No such device or address]

Comment 7 Mattia Lambertini 2017-07-26 16:09:10 UTC

Forgettin:

I'm on proxmox 5 with qemu+libgfapi

(In reply to Mattia Lambertini from comment #6)
> I'm experiencing what it seems the same issue.
> 
> Gluster version is 3.10.3
> 
> My current setup is quite simple, I'm starting with just one node (it's
> planned to add 2 more in the near future) and one volume with one brick:
> 
> Volume Name: gv0
> Type: Distribute
> Volume ID: ba11f016-3ae5-4fc2-b5a1-550bbcc74e39
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 1
> Transport-type: tcp
> Bricks:
> Brick1: pr1-cluster:/data/brick1/gv0
> Options Reconfigured:
> transport.address-family: inet
> nfs.disable: on
> diagnostics.brick-log-level: WARNING
> 
> Sharding is disabled.
> 
> From the brick log:
> 
> [2017-07-26 15:49:05.765830] E [MSGID: 113107] [posix.c:1080:posix_seek]
> 0-gv0-posix: seek failed on fd 31 length 40273903616 [No such device or
> address]
> [2017-07-26 15:49:05.765883] E [MSGID: 115089]
> [server-rpc-fops.c:2070:server_seek_cbk] 0-gv0-server: 16: SEEK-2
> (c66abe5a-ff28-4e4b-8391-d9a3e7aa7838), client:
> pr1-16895-2017/07/26-15:49:05:750324-gv0-client-0-0-0, error-xlator:
> gv0-posix [No such device or address]
> [2017-07-26 15:49:05.768248] E [MSGID: 113107] [posix.c:1080:posix_seek]
> 0-gv0-posix: seek failed on fd 31 length 53687091200 [No such device or
> address]
> [2017-07-26 15:49:05.768289] E [MSGID: 115089]
> [server-rpc-fops.c:2070:server_seek_cbk] 0-gv0-server: 20: SEEK-2
> (1f7a84d5-8383-41f0-8816-bfd55b105f9d), client:
> pr1-16895-2017/07/26-15:49:05:750324-gv0-client-0-0-0, error-xlator:
> gv0-posix [No such device or address]

Comment 8 Niels de Vos 2017-11-07 10:35:48 UTC

This bug is getting closed because the 3.8 version is marked End-Of-Life. There will be no further updates to this version. Please open a new bug against a version that still receives bugfixes if you are still facing this issue in a more current release.

Note You need to log in before you can comment on or make changes to this bug.