Bug 797913 - tgtd hangs at startup - systemd related
tgtd hangs at startup - systemd related
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: scsi-target-utils (Show other bugs)
16
Unspecified Unspecified
unspecified Severity urgent
: ---
: ---
Assigned To: Andy Grover
Fedora Extras Quality Assurance
:
: 800386 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-02-27 09:21 EST by Pádraig Brady
Modified: 2016-01-04 09:43 EST (History)
13 users (show)

See Also:
Fixed In Version: scsi-target-utils-1.0.18-5.fc16
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-03-06 14:43:48 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Pádraig Brady 2012-02-27 09:21:53 EST
This hangs: /etc/init.d/tgtd start
This works: SYSTEMCTL_SKIP_REDIRECT=1 /etc/init.d/tgtd start

Multiple people in the openstack dev team have seen this issue

I'm fairly sure tgtd used to work on F16, so it's probably a recent update.
Comment 1 Derek Higgins 2012-02-27 09:34:09 EST
On F16 /etc/init.d/tgtd start works on a fresh install 

the problem only appears after 
> yum update

By updating packages individually I narrowed it down to this update
> yum update systemd
Updating:
 systemd         x86_64   37-13.fc16   updates           728 k
Updating for dependencies:
 systemd-sysv    x86_64   37-13.fc16   updates            15 k
 systemd-units   x86_64   37-13.fc16   updates           150 k

Trying to narrow it down further the problem appears to have been introduced in 
37-6.fc16.x86_64
http://pkgs.fedoraproject.org/gitweb/?p=systemd.git;a=commit;h=e43452dfe678264ac81c087d2a8735c973f99686

thats all I have
Comment 2 Pádraig Brady 2012-02-27 09:49:45 EST
Same issue with systemd-37-14.fc16
Comment 3 Michal Schmidt 2012-02-27 10:03:11 EST
What is the status of the service when it hangs?:
systemctl status tgtd.service

Do you get anything interesting in /var/log/messages after the attempt to start the service?
Comment 4 Jóhann B. Guðmundsson 2012-02-27 10:20:12 EST
Wondering if those remount order patches changes might be causing this anyway if you can it would be good if you could test with the latest release in koji [1] which contains native systemd unit files ( it's enough to just copy the units from git into the /etc/systemd/system directory and run systemctl daemon-reload if upstream did not do any systemd specific patching ). 

We probably need you to follow 2. and provide the requested info from there.


1.http://koji.fedoraproject.org/koji/buildinfo?buildID=296816
2.http://fedoraproject.org/wiki/How_to_debug_Systemd_problems
Comment 5 Derek Higgins 2012-02-27 10:54:30 EST
Michal More info below

====== While on systemd-37-5

[root@test systemd]# rpm -qa | grep -i -e systemd -e scsi-target-utils
systemd-37-5.fc16.x86_64
systemd-sysv-37-5.fc16.x86_64
scsi-target-utils-1.0.18-4.fc16.x86_64
systemd-units-37-5.fc16.x86_64

[root@test systemd]# /etc/init.d/tgtd start
Starting tgtd (via systemctl):                             [  OK  ]

[root@test systemd]# /etc/init.d/tgtd status
tgtd.service - LSB: Starts and stops the generic storage target daemon
	  Loaded: loaded (/etc/rc.d/init.d/tgtd)
	  Active: active (running) since Mon, 27 Feb 2012 15:20:41 +0000; 3s ago
	 Process: 18361 ExecStart=/etc/rc.d/init.d/tgtd start (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/tgtd.service
		  ├ 18369 tgtd
		  └ 18370 tgtd

====== While on systemd-37-6

[root@test systemd]# rpm -qa | grep -i -e systemd -e scsi-target-utils
systemd-sysv-37-6.fc16.x86_64
scsi-target-utils-1.0.18-4.fc16.x86_64
systemd-37-6.fc16.x86_64
systemd-units-37-6.fc16.x86_64

[root@test systemd]# date
Mon Feb 27 15:23:12 UTC 2012

[root@test systemd]# /etc/init.d/tgtd start ; date
Starting tgtd (via systemctl):  Job failed. See system logs and 'systemctl status' for details.
                                                           [FAILED]
Mon Feb 27 15:28:21 UTC 2012

[root@test systemd]# /etc/init.d/tgtd status
tgtd.service - LSB: Starts and stops the generic storage target daemon
	  Loaded: loaded (/etc/rc.d/init.d/tgtd)
	  Active: failed since Mon, 27 Feb 2012 15:28:20 +0000; 7min ago
	 Process: 18518 ExecStop=/etc/rc.d/init.d/tgtd stop (code=exited, status=0/SUCCESS)
	 Process: 18536 ExecStart=/etc/rc.d/init.d/tgtd start (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/tgtd.service
		  ├ 18542 tgtd
		  └ 18543 tgtd


[root@test systemd]# systemctl start tgtd.service
< doesn't return for ~5 minutes >
[root@test systemd]# systemctl status tgtd.service                                                                                                            
tgtd.service - LSB: Starts and stops the generic storage target daemon
	  Loaded: loaded (/etc/rc.d/init.d/tgtd)
	  Active: activating (start) since Mon, 27 Feb 2012 15:36:08 +0000; 7min ago
	  CGroup: name=systemd:/system/tgtd.service
		  ├ 18542 tgtd
		  └ 18543 tgtd

[root@test systemd]# tail -n 26 /var/log/messages 
Feb 27 15:22:28 test dbus-daemon[470]: dbus[470]: [system] Reloaded configuration
Feb 27 15:22:28 test systemd[1]: Reexecuting.
Feb 27 15:22:29 test systemd[1]: systemd 37 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +SYSVINIT +LIBCRYPTSETUP; fedora)
Feb 27 15:22:29 test yum[18449]: Updated: systemd-37-6.fc16.x86_64
Feb 27 15:22:29 test yum[18449]: Updated: systemd-sysv-37-6.fc16.x86_64
Feb 27 15:22:29 test systemd[1]: Reloading.
Feb 27 15:22:29 test systemd-logind[18468]: New seat seat0.
Feb 27 15:22:29 test systemd-logind[18468]: New user root logged in.
Feb 27 15:22:29 test systemd-logind[18468]: New session 1 of user root.
Feb 27 15:23:01 test tgtd: tgtd logger stopped, pid:18370
Feb 27 15:23:01 test tgtd[18518]: Stopping SCSI target daemon: [  OK  ]
Feb 27 15:23:21 test tgtd: libibverbs.so: cannot open shared object file: No such file or directory - iser transport not used
Feb 27 15:23:21 test tgtd: semkey 0x6110a50a
Feb 27 15:23:21 test tgtd: tgtd daemon started, pid:18542
Feb 27 15:23:21 test tgtd: tgtd logger started, pid:18543 debug:0
Feb 27 15:23:21 test tgtd[18536]: Starting SCSI target daemon: [  OK  ]
Feb 27 15:23:21 test tgtd: work_timer_start(146) use timer_fd based scheduler
Feb 27 15:23:21 test tgtd: bs_init(312) use signalfd notification
Feb 27 15:23:21 test systemd[1]: PID file /var/run/tgtd.pid not readable (yet?) after start.
Feb 27 15:28:20 test systemd[1]: tgtd.service operation timed out. Terminating.
Feb 27 15:28:20 test systemd[1]: Unit tgtd.service entered failed state.
Feb 27 15:36:08 test systemd[1]: PID file /var/run/tgtd.pid not readable (yet?) after start.
Feb 27 15:40:38 test systemd[1]: Reexecuting.
Feb 27 15:40:38 test systemd[1]: systemd 37 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +SYSVINIT +LIBCRYPTSETUP; fedora)
Feb 27 15:45:38 test systemd[1]: tgtd.service operation timed out. Terminating.
Feb 27 15:45:38 test systemd[1]: Unit tgtd.service entered failed state.
[root@test systemd]#
Comment 6 Michal Schmidt 2012-02-27 11:04:50 EST
(In reply to comment #5)
> Feb 27 15:23:21 test systemd[1]: PID file /var/run/tgtd.pid not readable (yet?)
> after start.
> Feb 27 15:28:20 test systemd[1]: tgtd.service operation timed out. Terminating.

tgtd failed to write its declared PID file. Its initscript declares:
# pidfile: /var/run/tgtd.pid

It is required that the pidfile information given in initscripts (or PIDFile=... in native systemd units) is correct.

For more information see a similar bug in another package: bug 786625
Comment 7 Derek Higgins 2012-02-28 07:19:24 EST
Thanks Michal,

When I take the line 
# pidfile: /var/run/tgtd.pid
out of the init script, tgtd comes up and appears to do so correctly (at least for my case)

as far as I can see there is no option in tgtd to write out a pid file

Would removing the line above be an acceptable solution for the init script? Or is there a better option (in the absence of a pid file)?
Comment 8 Michal Schmidt 2012-02-28 09:38:51 EST
If the daemon really cannot create a pid file, then removing the "# pidfile:" header is correct, yes.
Comment 9 Dave Allan 2012-03-01 15:03:56 EST
(In reply to comment #7)
> When I take the line 
> # pidfile: /var/run/tgtd.pid
> out of the init script, tgtd comes up and appears to do so correctly (at least
> for my case)

FWIW taking out the pidfile line fixes the hang on my system as well.  I have no idea what the implications are, however.
Comment 10 Fedora Update System 2012-03-01 21:29:00 EST
scsi-target-utils-1.0.24-2.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/scsi-target-utils-1.0.24-2.fc17
Comment 11 Fedora Update System 2012-03-01 21:30:39 EST
scsi-target-utils-1.0.18-5.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/scsi-target-utils-1.0.18-5.fc16
Comment 12 Andy Grover 2012-03-06 11:50:14 EST
*** Bug 800386 has been marked as a duplicate of this bug. ***
Comment 13 Fedora Update System 2012-03-06 14:43:48 EST
scsi-target-utils-1.0.18-5.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.