Bug 1027113 - Service glusterd failes to start after the most recent update (3.11.6-201 kernel)
Summary: Service glusterd failes to start after the most recent update (3.11.6-201 ker...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: glusterfs
Version: 19
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Kaleb KEITHLEY
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-11-06 07:58 UTC by Boris Derzhavets
Modified: 2013-11-06 13:52 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2013-11-06 13:52:58 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Boris Derzhavets 2013-11-06 07:58:45 UTC
Description of problem:

[root@ovirt1 ~]# service glusterd status
Redirecting to /bin/systemctl status  glusterd.service
glusterd.service - GlusterFS an clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled)
   Active: failed (Result: exit-code) since Wed 2013-11-06 11:32:44 MSK; 20min ago
  Process: 2739 ExecStart=/usr/sbin/glusterd -p /run/glusterd.pid (code=exited, status=255)
   CGroup: name=systemd:/system/glusterd.service
           ├─1108 /usr/sbin/glusterd -p /run/glusterd.pid
           ├─2858 /usr/sbin/glusterfsd -s 192.168.1.135 --volfile-id cinder-vol.192.168.1.135.mnt-brick1-cinder-vol -p /var/lib/glusterd/...
           ├─2866 /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nf...
           └─2892 /sbin/rpc.statd

Nov 06 11:32:44 ovirt1.localdomain systemd[1]: Stopping GlusterFS an clustered file-system server...
Nov 06 11:32:44 ovirt1.localdomain systemd[1]: Starting GlusterFS an clustered file-system server...
Nov 06 11:32:44 ovirt1.localdomain systemd[1]: glusterd.service: control process exited, code=exited status=255
Nov 06 11:32:44 ovirt1.localdomain systemd[1]: Failed to start GlusterFS an clustered file-system server.
Nov 06 11:32:44 ovirt1.localdomain systemd[1]: Unit glusterd.service entered failed state.
Nov 06 11:32:47 ovirt1.localdomain rpc.statd[2892]: Version 1.2.7 starting
Nov 06 11:32:47 ovirt1.localdomain sm-notify[2894]: Version 1.2.7 starting

Attempt to debug :

[root@ovirt1 ~]# /usr/sbin/glusterd --debug
[2013-11-06 07:54:34.935415] I [glusterfsd.c:1910:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.4.1 (/usr/sbin/glusterd --debug)
[2013-11-06 07:54:34.936254] D [glusterfsd.c:563:get_volfp] 0-glusterfsd: loading volume file /etc/glusterfs/glusterd.vol
[2013-11-06 07:54:34.939370] I [glusterd.c:962:init] 0-management: Using /var/lib/glusterd as working directory
[2013-11-06 07:54:34.939544] D [glusterd.c:332:glusterd_rpcsvc_options_build] 0-: listen-backlog value: 128
[2013-11-06 07:54:34.939688] D [rpcsvc.c:1900:rpcsvc_init] 0-rpc-service: RPC service inited.
[2013-11-06 07:54:34.939705] D [rpcsvc.c:1666:rpcsvc_program_register] 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
[2013-11-06 07:54:34.939753] D [rpc-transport.c:249:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3.4.1/rpc-transport/socket.so
[2013-11-06 07:54:34.941652] I [socket.c:3480:socket_init] 0-socket.management: SSL support is NOT enabled
[2013-11-06 07:54:34.941672] I [socket.c:3495:socket_init] 0-socket.management: using system polling thread
[2013-11-06 07:54:34.941706] D [name.c:557:server_fill_address_family] 0-socket.management: option address-family not specified, defaulting to inet
[2013-11-06 07:54:34.942288] E [socket.c:695:__socket_server_bind] 0-socket.management: binding to  failed: Address already in use
[2013-11-06 07:54:34.942305] E [socket.c:698:__socket_server_bind] 0-socket.management: Port is already in use
[2013-11-06 07:54:34.942339] W [rpcsvc.c:1396:rpcsvc_transport_create] 0-rpc-service: listening on transport failed
[2013-11-06 07:54:34.942361] E [glusterd.c:1056:init] 0-management: creation of listener failed
[2013-11-06 07:54:34.942374] E [xlator.c:390:xlator_init] 0-management: Initialization of volume 'management' failed, review your volfile again
[2013-11-06 07:54:34.942393] E [graph.c:292:glusterfs_graph_init] 0-management: initializing translator failed
[2013-11-06 07:54:34.942408] E [graph.c:479:glusterfs_graph_activate] 0-graph: init failed
[2013-11-06 07:54:34.942479] W [glusterfsd.c:1002:cleanup_and_exit] (-->/usr/sbin/glusterd(main+0x3df) [0x7fef80d4c8ef] (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xb0) [0x7fef80d4f6f0] (-->/usr/sbin/glusterd(glusterfs_process_volfp+0x103) [0x7fef80d4f603]))) 0-: received signum (0), shutting down
[2013-11-06 07:54:34.942501] D [glusterfsd-mgmt.c:2273:glusterfs_mgmt_pmap_signout] 0-fsd-mgmt: portmapper signout arguments not given

Version-Release number of selected component (if applicable):

3.4.1

How reproducible:


Steps to Reproduce:
1.Run yum -y update
2.Run  yum install glusterfs glusterfs-server glusterfs-fuse
3.Try to start glusterd

Actual results:

failure

Expected results:

Success
Additional info:

Comment 1 Niels de Vos 2013-11-06 09:33:00 UTC
Important bits from the logs:

[2013-11-06 07:54:34.942288] E [socket.c:695:__socket_server_bind] 0-socket.management: binding to  failed: Address already in use
[2013-11-06 07:54:34.942305] E [socket.c:698:__socket_server_bind] 0-socket.management: Port is already in use
[2013-11-06 07:54:34.942339] W [rpcsvc.c:1396:rpcsvc_transport_create] 0-rpc-service: listening on transport failed


This sounds as if there is a process listening on tcp/24007. Could you check which process that is?

# netstat -tlpn | grep 24007

Comment 2 Kaleb KEITHLEY 2013-11-06 12:46:44 UTC
You also need to install glusterfs-libs and glusterfs-cli.

(dependencies for libvirt/oVirt/vdsm and other packaging considerations preclude making these dependencies of the other RPMs.)

Comment 3 Boris Derzhavets 2013-11-06 13:48:59 UTC
(In reply to Niels de Vos from comment #1)
> Important bits from the logs:
> 
> [2013-11-06 07:54:34.942288] E [socket.c:695:__socket_server_bind]
> 0-socket.management: binding to  failed: Address already in use
> [2013-11-06 07:54:34.942305] E [socket.c:698:__socket_server_bind]
> 0-socket.management: Port is already in use
> [2013-11-06 07:54:34.942339] W [rpcsvc.c:1396:rpcsvc_transport_create]
> 0-rpc-service: listening on transport failed
> 
> 
> This sounds as if there is a process listening on tcp/24007. Could you check
> which process that is?
> 
> # netstat -tlpn | grep 24007

Thank you very much port was busy by my fault
Glusterd works fine.


Note You need to log in before you can comment on or make changes to this bug.