Description of problem: Systemd units, provided by centos storage sig are not able to stop all gluster daemons, I guess these units are developed by gluster developers Version-Release number of selected component (if applicable): 3.12.6 How reproducible: Always [root@vmhost1 ~]# systemctl stop glusterfsd [root@vmhost1 ~]# systemctl stop glusterd [root@vmhost1 ~]# ps ax | grep glust 2478 ? Ssl 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /var/run/gluster/glustershd/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/d63ed11ad6712579ba0116e69f0b5a90.socket --xlator-option *replicate*.node-uuid=0172d180-e203-45e6-b447-79117a4765f2 2555 ? Ssl 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/run/gluster/bitd/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/de223d0e1e02670bb0a176ebd143beff.socket --global-timer-wheel 2595 ? Ssl 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/run/gluster/scrub/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1ceaa9fdc4ca79dcc6da7779af8407d4.socket --global-timer-wheel 2952 pts/39 S+ 0:00 grep --color=auto glust Steps to Reproduce: [root@vmhost1 ~]# systemctl stop glusterfsd [root@vmhost1 ~]# systemctl stop glusterd [root@vmhost1 ~]# ps ax | grep glust 2478 ? Ssl 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /var/run/gluster/glustershd/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/d63ed11ad6712579ba0116e69f0b5a90.socket --xlator-option *replicate*.node-uuid=0172d180-e203-45e6-b447-79117a4765f2 2555 ? Ssl 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/run/gluster/bitd/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/de223d0e1e02670bb0a176ebd143beff.socket --global-timer-wheel 2595 ? Ssl 0:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/run/gluster/scrub/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1ceaa9fdc4ca79dcc6da7779af8407d4.socket --global-timer-wheel 2952 pts/39 S+ 0:00 grep --color=auto glust Actual results: some daemons are running Expected results: all daemons are stopped Additional info:
"systemctl stop glusterd" command will stop only glusterd daemon If you want to stop all gluster daemon, same can be achieve by using a script stop-all-gluster-processes.sh. This script is available in /usr/local/share/glusterfs/scripts this directory. This is not a valid bug, hence I am closing this bug.
Well, I don't think that bug is invalid. If processes are started by systemctl start glusterd systemctl start glusterfsd they have to be stopped by systemctl stop glusterd systemctl stop glusterfsd just because administrator uses systemd for system management. external script is good, but it breaks unification. Also, looks like these processes are not restarted during upgrade. Thank you!
(In reply to Need Real Name from comment #2) > Well, I don't think that bug is invalid. > > If processes are started by > systemctl start glusterd This will start glusterd. > systemctl start glusterfsd we dont have any service called glusterfsd. glusterfsd will get started after volume creation followed by volume start. > > they have to be stopped by > systemctl stop glusterd It will stop glusterd alone. glusterd is a management daemon which manages the cluster configuration. To answer your query let use consider a case where IO's are going on and in between systemctl stop glusterd is executed, Now if glusterd kills glusterfsd, IO's will be hampered. This is as per design, I don't see an issue here. Its not a valid bug, hence I am closing this bug
No, you have systemctl status glusterfsd ● glusterfsd.service - GlusterFS brick processes (stopping only) Loaded: loaded (/usr/lib/systemd/system/glusterfsd.service It is valid bug, because , as you just admitted , there is no way to stop gluster services using systemd. Thank you!
> Now if glusterd kills glusterfsd, IO's will be hampered. yes, and if glusterfsd will be killed while system reboot what will happen? you just said that we need correct way to stop glusterfsd and it doesn't work.
(In reply to Need Real Name from comment #5) > > Now if glusterd kills glusterfsd, IO's will be hampered. > > yes, and if glusterfsd will be killed while system reboot what will happen? > you just said that we need correct way to stop glusterfsd and it doesn't > work. Brick processes killed during system reboot are restarted by glusterd once the system is up. You don't need to explicitly start the bricks. To kill all gluster services you could do a `pkill gluster`; followed by `systemctl start glusterd` to restart the services.
Yes, but, as I wrote before, rhel and derivatives ( centos ) are systemd based and there is no unit which can do this 'pkill gluster'. Although most of other daemons are systemd-aware. This is why I created this bug report. Thank you!
Release 3.12 has been EOLd and this bug was still found to be in the NEW state, hence moving the version to mainline, to triage the same and take appropriate actions.
> systemctl start glusterfsd [root@localhost ~]# systemctl start glusterfsd Failed to start glusterfsd.service: Unit glusterfsd.service not found. with the later releases, glusterfsd is not allowed to be started directly, but always will start from glusterd. (and is recommended that way). With that, we are closing the issue.
I'm talking about stopping, not starting, this is not fixed!
Dmitry, my point is, from glusterfs project, we are not supplying glusterfsd.service file itself, so, fixing this issue is not valid in the project, because systemd only bothers about glusterd, and as we would like to cleanup all the processes started by glusterd too, glusterfsd too is stopped. https://github.com/gluster/glusterfs/tree/master/extras/systemd
Please fix glusterd.service, so while stopping glusterd glusterfsd will be stopped too. Thank you!
Dmitry - as per the design, stopping glusterd process doesn't mean we need to necessary stop all glusterfsd services. The design consideration we have in glusterfs is that management and I/O plane to remain as seggregated, so at any given point of time if the glusterd daemon i.e. management service needs to be brought down the brick processes (glusterfsd) should remain up and running. I hope this justifies the reason for the bug closure.
"stopping glusterd process doesn't mean we need to necessary stop all glusterfsd services." Not a problem, but there is no way to correctly stop glusterfsd in case of server shutdown. Please provide it!
Have you explored stop-all-gluster-processes.sh script which is packaged as part of glusterfs-server?
This script is not called by systemd during system shutdown...
Could you clarify how does an ungraceful shutdown of glusterfsd process (if not a graceful through systemd) affect your application? We'd be interested to know the same.
Well, frankly, main problem here is not server reboot, but updating gluster, in this case you do not correctly restart it... Anyway, any service/daemon have to be correctly stopped during system shutdown, or you disagree? Just killing process is OK?
We are not planning to work on this in the near future, So there is no point in keeping the bug open. Closing this as won't fix. Thanks, Sanju
But you have to fix this!
Based on above comments, closing this as won't fix.