I have configured varnish to listen on port 80 and nginx to listen on 8080. After around 24 hours of uptime, my site has been down for 22 hours. I checked in and found varnish isn't listening on port 80.
When site is up:
abc@abc:~$ sudo netstat -anp --tcp --udp | grep LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 571/varnishd
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 376/nginx
tcp 0 0 0.0.0.0:9171 0.0.0.0:* LISTEN 376/nginx
tcp 0 0 publicip:6082 0.0.0.0:* LISTEN 569/varnishd
tcp6 0 0 :::80 :::* LISTEN 376/nginx
tcp6 0 0 ::1:6082 :::* LISTEN 569/varnishd
When site is down:
abc@abc:~$ sudo netstat -anp --tcp --udp | grep LISTEN
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 376/nginx
tcp 0 0 0.0.0.0:9171 0.0.0.0:* LISTEN 376/nginx
tcp 0 0 publicip:6082 0.0.0.0:* LISTEN 745/varnishd
tcp6 0 0 :::80 :::* LISTEN 376/nginx
tcp6 0 0 ::1:6082 :::* LISTEN 745/varnishd
This is my /etc/default/varnish:
## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request. Use a 1GB
# fixed-size cache file.
#
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,96m"
Is there a specific reason why varnish is not listening on port 80 in the second case? I can probably just check and if varnish is not up, restart it but that would still mean a few minutes of down time.
My varnish.vcl file: http://pastebin.com/UH2c8KdH I'm on ubuntu 12.04 x86
It happened again after 2 hours or so and this is what I found from syslog.
Feb 14 18:16:00 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:16:51 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:17:49 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:18:06 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:19:33 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:21:25 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:22:34 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:28:28 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:29:41 abc varnishd[745]: Child (749) not responding to CLI, killing it.
Feb 14 18:29:48 abc last message repeated 2 times
Feb 14 18:29:48 abc varnishd[745]: Child (749) died signal=3
Feb 14 18:29:49 abc varnishd[745]: Child cleanup complete
Feb 14 18:29:55 abc varnishd[745]: child (1380) Started
Feb 14 18:29:58 abc varnishd[745]: Pushing vcls failed: CLI communication error (hdr)
Feb 14 18:29:58 abc varnishd[745]: Stopping Child
Feb 14 18:29:58 abc varnishd[745]: Child (1380) said Child starts
Feb 14 18:29:59 abc varnishd[745]: Child (1380) said Child dies
Feb 14 18:30:02 abc varnishd[745]: Child (1380) died status=1
Feb 14 18:30:04 abc varnishd[745]: Child cleanup complete
I'm not sure why the process ids are different than the ones I posted earlier. Maybe I restarted it while troubleshooting. I can't really make out much from these logs. Any help is appreciated.
Adding more logs:
Details from /etc/log/messages:
First time it stopped:
Feb 13 17:40:44 dragon75 varnishd[581]: Child (583) died signal=3
Feb 13 17:41:09 dragon75 varnishd[581]: child (2682) Started
Feb 13 17:42:31 dragon75 varnishd[581]: Child (2682) said Child starts
Feb 13 17:51:48 dragon75 varnishd[581]: Child (2682) died status=1
Feb 13 17:51:48 dragon75 varnishd[581]: Child (-1) said Child dies
Second time it stopped:
Feb 14 18:29:48 dragon75 varnishd[745]: Child (749) died signal=3
Feb 14 18:29:55 dragon75 varnishd[745]: child (1380) Started
Feb 14 18:29:58 dragon75 varnishd[745]: Child (1380) said Child starts
Feb 14 18:29:59 dragon75 varnishd[745]: Child (1380) said Child dies
Feb 14 18:30:02 dragon75 varnishd[745]: Child (1380) died status=1
According to messages, at 16:31 varnish started and then there are 5 MARK messages in /var/log/messages and then the varnish child died message at 18:29. There is nothing in between.
I don't think resources are a bottleneck. This is a new website and still in testing phase. I haven't really put anything on it. There is no traffic except the uptime script that I have on another server which just checks for the homepage. This is the first time I'm using varnish.
