Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1109094

Summary: init script fails to start/stop service
Product: Red Hat Enterprise Linux 7 Reporter: Mahendra Takwale <mahendra_takwale>
Component: systemdAssignee: systemd-maint
Status: CLOSED NOTABUG QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.0CC: lnykryn, mahendra_takwale, msekleta, systemd-maint-list
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-08 12:02:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
debug logs
none
vcs init script
none
systemctl show vcs.service output after starting vcs services. none

Description Mahendra Takwale 2014-06-13 08:47:40 UTC
Description of problem: Symantec vcs init script fails to start/stop services in case of abrupt process termination on RHEL7. On a fresh system, vcs init script works fine and starts services properly. 
......................
(Linux)(rhel7-lfg-vm1) log{195} systemctl list-units | grep vcs
vcs.service                                                                                      loaded failed failed    LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.
(Linux)(rhel7-lfg-vm1) log{196} /etc/init.d/vcs start
Starting vcs (via systemctl):                              [  OK  ]
(Linux)(rhel7-lfg-vm1) log{197} ps -ef | grep had
root     21217     1  3 02:10 ?        00:00:00 /opt/VRTSvcs/bin/had -onenode
root     21223     1  0 02:11 ?        00:00:00 /opt/VRTSvcs/bin/hashadow
root     21229 17121  0 02:11 pts/5    00:00:00 grep --color=auto had
(Linux)(rhel7-lfg-vm1) log{198} systemctl list-units | grep vcs
vcs.service                                                                                      loaded active running   LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.
(Linux)(rhel7-lfg-vm1) log{199}
......................

After this, a process gets terminated abruptly, the "vcs.service" unit file status is still "running"

......................
(Linux)(rhel7-lfg-vm1) log{204} kill -9 21217 21223
(Linux)(rhel7-lfg-vm1) log{205} systemctl list-units | grep vcs
vcs.service                                                                                      loaded active running   LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.
(Linux)(rhel7-lfg-vm1) log{206} systemctl list-units | grep vcs
vcs.service                                                                                      loaded active running   LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.
(Linux)(rhel7-lfg-vm1) log{207} ps -ef | grep had
root     21288 17121  0 02:11 pts/5    00:00:00 grep --color=auto had
(Linux)(rhel7-lfg-vm1) log{208} systemctl list-units | grep vcs
vcs.service                                                                                      loaded active running   LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.

......................

Any attempt to start service using "/etc/init.d/vcs start" doesn't give any error but fails to start service.

......................
(Linux)(rhel7-lfg-vm1) log{209} /etc/init.d/vcs start
Starting vcs (via systemctl):                              [  OK  ]
(Linux)(rhel7-lfg-vm1) log{210} ps -ef | grep had
root     21300 17121  0 02:11 pts/5    00:00:00 grep --color=auto had
(Linux)(rhel7-lfg-vm1) log{211} ps -ef | grep had
root     21302 17121  0 02:11 pts/5    00:00:00 grep --color=auto had
(Linux)(rhel7-lfg-vm1) log{212} ps -ef | grep had
root     21304 17121  0 02:11 pts/5    00:00:00 grep --color=auto had
(Linux)(rhel7-lfg-vm1) log{213} systemctl list-units | grep vcs
vcs.service                                                                                      loaded active running   LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.
(Linux)(rhel7-lfg-vm1) log{214}
......................


Version-Release number of selected component (if applicable):


How reproducible:

This is reproducible every time a process is stopped without using "init" script or process is abruptly terminated

Steps to Reproduce:
1. Start a service using init script
2. kill process 
3. once again try to start service using init script

Actual results:


Expected results:


Additional info:

Comment 2 Michal Sekletar 2014-06-13 09:29:38 UTC
Can you please add systemd.log_level=debug to kernel command line, reproduce the issue and upload output of "journalctl -b" command. Thanks!

Comment 3 Mahendra Takwale 2014-06-13 12:28:10 UTC
Is there any alternative way to enable debug logs? After updating kernel command line, system is not able to boot.

Comment 4 Lukáš Nykrýn 2014-06-16 07:39:30 UTC
kill -56 1
this will set systemd log level to debug.

By the way can you also try to boot just with "debug" on kernel cmdline?

Comment 5 Mahendra Takwale 2014-06-17 06:26:32 UTC
Created attachment 909387 [details]
debug logs

Comment 6 Mahendra Takwale 2014-06-17 06:28:24 UTC
Log file attached.

Following is the sequence of opeation

1. start VCS service using init script
#/etc/init.d/vcs start
Starting vcs (via systemctl):                              [  OK  ]
#systemctl list-units | grep vcs
vcs.service                                                                                      loaded active running   LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.
#
2. check process status
#ps -ef | grep had
root     10492     1  9 23:56 ?        00:00:00 /opt/VRTSvcs/bin/had -onenode
root     10501     1  0 23:56 ?        00:00:00 /opt/VRTSvcs/bin/hashadow
root     10560  5352  0 23:56 pts/2    00:00:00 grep --color=auto had
#
3. kill processes
#kill -9 10492 10501
#
#systemctl list-units | grep vcs
vcs.service                                                                                      loaded active running   LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.
# ps -ef | grep had
root     11137  5352  0 23:58 pts/2    00:00:00 grep --color=auto had
#
4. Once again start VCS service using init script
#/etc/init.d/vcs start
Starting vcs (via systemctl):                              [  OK  ]
#systemctl list-units | grep vcs
vcs.service                                                                                      loaded active running   LSB: VCS is a Cluster Server, which is the program that provides high availability of applications.
#ps -ef | grep had
root     11149  5352  0 23:59 pts/2    00:00:00 grep --color=auto had
#

Comment 7 Lukáš Nykrýn 2014-06-17 06:57:24 UTC
Can you also provide us the initscript itself?

Also after you start the service, could you send us output of systemctl show vcs.service? Also can you check in that output that ExecMainPID matches to some main process of vcs?

Comment 8 Mahendra Takwale 2014-06-17 11:45:08 UTC
Created attachment 909546 [details]
vcs init script

vcs init script attached here.

Comment 9 Mahendra Takwale 2014-06-17 11:46:56 UTC
Created attachment 909558 [details]
systemctl show vcs.service output after starting vcs services.

systemctl show vcs.service output after starting vcs services.

Please note, processes are started

Comment 10 Michal Sekletar 2014-06-17 15:21:56 UTC
Since your service consists of bunch of processes it would be great if you could change your daemon so it writes out pidfile and reference that in initscript. Do you think it would be possible?

Comment 11 Mahendra Takwale 2014-06-19 08:01:33 UTC
Hi Michal,

Thanks for your help.

Could you please share more details of why it is failing? and how the writing pid will solve the issue?

What are the per-requisites  for supporting standard application init scripts with systemd?

Thanks & Regards,
Mahendra Takwale

Comment 12 Lukáš Nykrýn 2014-06-19 11:03:51 UTC
Simply put, for every service systemd register one pid as a ExecMainPID and when this process is down whole service is consider finished. With initscripts the service can say which process is the main one by a pidfile. If pidfile is missing systemd will try to guess. And my suspicion is that systemd picked wrong process  which is still running.

Comment 13 Michal Sekletar 2014-06-20 16:33:11 UTC
For initscripts systemd doesn't attempt to guess the MainPID. In any case, it looks like systemd doesn't have a full picture here thus reports wrong service status. Can you please provide output of systemd-cgls --full before and after killing processes by hand?

Comment 14 Michal Sekletar 2014-06-20 16:44:09 UTC
(In reply to Mahendra Takwale from comment #11)
> Hi Michal,
> 
> Thanks for your help.
> 
> Could you please share more details of why it is failing? and how the
> writing pid will solve the issue?
> 
> What are the per-requisites  for supporting standard application init
> scripts with systemd?

None. It should work out of the box. However if initscript doesn't include reference to pidfile systemd treats it as one-shot script rather then long running process. But your initscript doesn't do former but later behaviour is expected which is wrong. So adding pidfile to initscript should fix the issue.

> 
> Thanks & Regards,
> Mahendra Takwale

Comment 15 Lukáš Nykrýn 2014-08-08 12:02:40 UTC
So as mentioned earlier this is more a configuration issue, the initscript should use pidfile -> NOTABUG.

Comment 16 Red Hat Bugzilla 2023-09-14 02:10:01 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days