Bug 1464355 - Starting varnishncsa during boot fails
Summary: Starting varnishncsa during boot fails
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: varnish
Version: epel7
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Ingvar Hagelund
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-23 08:28 UTC by Gerald Vogt
Modified: 2020-02-24 10:44 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-02-24 10:44:55 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Gerald Vogt 2017-06-23 08:28:33 UTC
Description of problem:

We have a server which is running varnishd and varnishncsa for logging. However, during boot varnishncsa fails because varnishd isn't ready, yet.


Version-Release number of selected component (if applicable):

varnish-4.0.4-3.el7.x86_64

How reproducible:

Install varnish, enable varnishncsa with systemctl and reboot


Steps to Reproduce:
1. Install varnish and configure it.
2. systemctl enable varnishncsa
3. reboot

Actual results:

varnishncsa fails during boot.

Expected results:

varnishncsa is running

Additional info:

Logs show the problem:

Jun 22 08:22:04 test systemd: Failed to read PID from file /var/run/varnish.pid: Invalid argument
Jun 22 08:22:04 test systemd: Started Varnish Cache, a high-performance HTTP accelerator.
Jun 22 08:22:04 test systemd: Starting Varnish Cache HTTP accelerator NCSA logging daemon...
zun 22 08:22:04 test varnishd[936]: Platform: Linux,3.10.0-514.21.2.el7.x86_64,x86_64,-sfile,-smalloc,-hcritbit
Jun 22 08:22:04 test systemd: varnishncsa.service: control process exited, code=exited status=1
Jun 22 08:22:04 test systemd: Failed to start Varnish Cache HTTP accelerator NCSA logging daemon. 
Jun 22 08:22:04 test systemd: Unit varnishncsa.service entered failed state.
Jun 22 08:22:04 test systemd: varnishncsa.service failed. 
Jun 22 08:22:05 test varnishd[936]: child (939) Started 
Jun 22 08:22:05 test varnishd[936]: Child (939) said Child starts
Jun 22 08:22:05 test varnishd[936]: Child (939) said SMF.s0 mmap'ed 2147483648 bytes of 2147483648

So it seems to me varnishncsa is started while varnishd is still starting up. Thus, either varnishncsa has to wait or the varnishd start during boot should only return once varnishd is up...

Comment 1 Ingvar Hagelund 2017-06-26 07:11:12 UTC
The problem here is that varnishncsa starts too fast, that is, before varnishd has gotten around to open its vsm file. A simple workaround could be just to add a pause to the startup of varnishd like "ExecStartPost=sleep 5" in varnish.service, or just make varnishncsa try restarting until it succeeds, that is "Restart=on-failure" in varnishncsa.service.

A more correct solution would be to figure out some systemd probe-able resource that is not available before varnishd has started proberly, and make varnishncsa wait for that. I've got to look more into this, and ask upstream.

Ingvar

Comment 2 Ingvar Hagelund 2017-06-26 07:19:39 UTC
Upstream issue: https://github.com/varnishcache/pkg-varnish-cache/issues/79

Comment 3 Dridi Boukelmoune 2018-02-27 16:18:58 UTC
I closed the upstream issue because as far as we are concerned, 4.0 is EOL and the verdict is WONTFIX. There's no easy workaround until 4.1 (enabled by default). 5.2 finishes daemonizing when varnishd is completely ready and shouldn't need any workaround.

Comment 4 Ingvar Hagelund 2020-02-24 10:44:55 UTC
I'm closing this as WONTFIX, as it is a corner case, and I can't get an upstream fix.

If anybody else reaches this, a workaround might be something along one of these two:


## Make systemd wait 5s after varnishd has started before starting varnishncsa
# mkdir -p /etc/systemd/system/varnish.service.d
# echo -e '[Service]\nExecStartPost=sleep 5' > /etc/systemd/system/varnish.service.d/wait_for_vsm_open.conf
# systemctl daemon-reload

## OR

## Make varnishncsa restart until varnishd answers
# mkdir -p /etc/systemd/system/varnishncsa.service.d
# echo -e '[Service]\nRestart=on-failure\nRestartSec=5' > /etc/systemd/system/varnishncsa.service.d/restart.conf
# systemctl daemon-reload


Note You need to log in before you can comment on or make changes to this bug.