Bug 1027113 - Service glusterd failes to start after the most recent update (3.11.6-201 kernel)
Service glusterd failes to start after the most recent update (3.11.6-201 ker...
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: glusterfs (Show other bugs)
19
x86_64 Linux
unspecified Severity unspecified
: ---
: ---
Assigned To: Kaleb KEITHLEY
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-06 02:58 EST by Boris Derzhavets
Modified: 2013-11-06 08:52 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-06 08:52:58 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Boris Derzhavets 2013-11-06 02:58:45 EST
Description of problem:

[root@ovirt1 ~]# service glusterd status
Redirecting to /bin/systemctl status  glusterd.service
glusterd.service - GlusterFS an clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled)
   Active: failed (Result: exit-code) since Wed 2013-11-06 11:32:44 MSK; 20min ago
  Process: 2739 ExecStart=/usr/sbin/glusterd -p /run/glusterd.pid (code=exited, status=255)
   CGroup: name=systemd:/system/glusterd.service
           ├─1108 /usr/sbin/glusterd -p /run/glusterd.pid
           ├─2858 /usr/sbin/glusterfsd -s 192.168.1.135 --volfile-id cinder-vol.192.168.1.135.mnt-brick1-cinder-vol -p /var/lib/glusterd/...
           ├─2866 /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nf...
           └─2892 /sbin/rpc.statd

Nov 06 11:32:44 ovirt1.localdomain systemd[1]: Stopping GlusterFS an clustered file-system server...
Nov 06 11:32:44 ovirt1.localdomain systemd[1]: Starting GlusterFS an clustered file-system server...
Nov 06 11:32:44 ovirt1.localdomain systemd[1]: glusterd.service: control process exited, code=exited status=255
Nov 06 11:32:44 ovirt1.localdomain systemd[1]: Failed to start GlusterFS an clustered file-system server.
Nov 06 11:32:44 ovirt1.localdomain systemd[1]: Unit glusterd.service entered failed state.
Nov 06 11:32:47 ovirt1.localdomain rpc.statd[2892]: Version 1.2.7 starting
Nov 06 11:32:47 ovirt1.localdomain sm-notify[2894]: Version 1.2.7 starting

Attempt to debug :

[root@ovirt1 ~]# /usr/sbin/glusterd --debug
[2013-11-06 07:54:34.935415] I [glusterfsd.c:1910:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.4.1 (/usr/sbin/glusterd --debug)
[2013-11-06 07:54:34.936254] D [glusterfsd.c:563:get_volfp] 0-glusterfsd: loading volume file /etc/glusterfs/glusterd.vol
[2013-11-06 07:54:34.939370] I [glusterd.c:962:init] 0-management: Using /var/lib/glusterd as working directory
[2013-11-06 07:54:34.939544] D [glusterd.c:332:glusterd_rpcsvc_options_build] 0-: listen-backlog value: 128
[2013-11-06 07:54:34.939688] D [rpcsvc.c:1900:rpcsvc_init] 0-rpc-service: RPC service inited.
[2013-11-06 07:54:34.939705] D [rpcsvc.c:1666:rpcsvc_program_register] 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
[2013-11-06 07:54:34.939753] D [rpc-transport.c:249:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.4.1/rpc-transport/socket.so
[2013-11-06 07:54:34.941652] I [socket.c:3480:socket_init] 0-socket.management: SSL support is NOT enabled
[2013-11-06 07:54:34.941672] I [socket.c:3495:socket_init] 0-socket.management: using system polling thread
[2013-11-06 07:54:34.941706] D [name.c:557:server_fill_address_family] 0-socket.management: option address-family not specified, defaulting to inet
[2013-11-06 07:54:34.942288] E [socket.c:695:__socket_server_bind] 0-socket.management: binding to  failed: Address already in use
[2013-11-06 07:54:34.942305] E [socket.c:698:__socket_server_bind] 0-socket.management: Port is already in use
[2013-11-06 07:54:34.942339] W [rpcsvc.c:1396:rpcsvc_transport_create] 0-rpc-service: listening on transport failed
[2013-11-06 07:54:34.942361] E [glusterd.c:1056:init] 0-management: creation of listener failed
[2013-11-06 07:54:34.942374] E [xlator.c:390:xlator_init] 0-management: Initialization of volume 'management' failed, review your volfile again
[2013-11-06 07:54:34.942393] E [graph.c:292:glusterfs_graph_init] 0-management: initializing translator failed
[2013-11-06 07:54:34.942408] E [graph.c:479:glusterfs_graph_activate] 0-graph: init failed
[2013-11-06 07:54:34.942479] W [glusterfsd.c:1002:cleanup_and_exit] (-->/usr/sbin/glusterd(main+0x3df) [0x7fef80d4c8ef] (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xb0) [0x7fef80d4f6f0] (-->/usr/sbin/glusterd(glusterfs_process_volfp+0x103) [0x7fef80d4f603]))) 0-: received signum (0), shutting down
[2013-11-06 07:54:34.942501] D [glusterfsd-mgmt.c:2273:glusterfs_mgmt_pmap_signout] 0-fsd-mgmt: portmapper signout arguments not given

Version-Release number of selected component (if applicable):

3.4.1

How reproducible:


Steps to Reproduce:
1.Run yum -y update
2.Run  yum install glusterfs glusterfs-server glusterfs-fuse
3.Try to start glusterd

Actual results:

failure

Expected results:

Success
Additional info:
Comment 1 Niels de Vos 2013-11-06 04:33:00 EST
Important bits from the logs:

[2013-11-06 07:54:34.942288] E [socket.c:695:__socket_server_bind] 0-socket.management: binding to  failed: Address already in use
[2013-11-06 07:54:34.942305] E [socket.c:698:__socket_server_bind] 0-socket.management: Port is already in use
[2013-11-06 07:54:34.942339] W [rpcsvc.c:1396:rpcsvc_transport_create] 0-rpc-service: listening on transport failed


This sounds as if there is a process listening on tcp/24007. Could you check which process that is?

# netstat -tlpn | grep 24007
Comment 2 Kaleb KEITHLEY 2013-11-06 07:46:44 EST
You also need to install glusterfs-libs and glusterfs-cli.

(dependencies for libvirt/oVirt/vdsm and other packaging considerations preclude making these dependencies of the other RPMs.)
Comment 3 Boris Derzhavets 2013-11-06 08:48:59 EST
(In reply to Niels de Vos from comment #1)
> Important bits from the logs:
> 
> [2013-11-06 07:54:34.942288] E [socket.c:695:__socket_server_bind]
> 0-socket.management: binding to  failed: Address already in use
> [2013-11-06 07:54:34.942305] E [socket.c:698:__socket_server_bind]
> 0-socket.management: Port is already in use
> [2013-11-06 07:54:34.942339] W [rpcsvc.c:1396:rpcsvc_transport_create]
> 0-rpc-service: listening on transport failed
> 
> 
> This sounds as if there is a process listening on tcp/24007. Could you check
> which process that is?
> 
> # netstat -tlpn | grep 24007

Thank you very much port was busy by my fault
Glusterd works fine.

Note You need to log in before you can comment on or make changes to this bug.