38
Foreword
Update in 2016. Things are evolving, all servers are getting better, they all support SSL and the web is more amazing than ever.
Unless stated, the following is targeted toward professionals in business and start-ups, supporting thousands to millions of users.
These tools and architectures require a lot of users/hardware/money. You can try this ...
20
Note that the above answers are for Varnish 3. As the question does not not specify version information, it seems an appropriate time to include the answer for Version 4 also as it has changed.
Hopefully this will save people from reading the above answers and putting vcl_error into their V4 VCL :)
Builtin VCL for Varnish 4.0
sub vcl_synth {
set resp....
19
I my case nginx listens to port 80 inside a docker container but it's mapped to port 8080 (or any random port) outside the container. There is no reverse proxy in-between that can add proper headers for port and also don't want to hardcode it in the nginx configuration.
Example of wrong redirect:
http://localhost:8080/directory -> http://localhost/...
17
You need to change the varnish start parameters in the systemd service definition as well. You could edit the line starting with ExecStart in the service defintion file:
sudo vi /lib/systemd/system/varnish.service
Modifying this file however, has the disadvantage that it will not be updated in future updates of the package. Alternatively, as suggested in ...
15
I would like to propose an alternative... please find below a sample default.vcl file
vcl 4.0;
import std;
backend default {
.host = "127.0.0.1";
.port = "8080";
}
sub vcl_backend_response {
if (beresp.status == 503 && bereq.retries < 5 ) {
return(retry);
}
}
sub vcl_backend_error {
if (beresp.status == 503 &&...
14
All other answers are pre-2010, hence adding an updated comparison.
Nginx
A full web server, other features can also be used. Eg: HTTP
Compression
SSL Support
Very light weight as Nginx was designed to be light from the start.
Near Varnish caching performance
Close to HAProxy load balancing performance
Varnish
best for complex caching scenarios and ...
14
In Varnish 4
varnishtop -i BereqURL
will provide you with an updating list of URL requests going to the back-end (Misses).
varnishtop -i ReqURL
will provide you with an updating list of ALL requests.
I cannot figure out a simple request for a "HIT". The documentation for Varnishtop is at https://www.varnish-cache.org/docs/4.0/reference/varnishtop.html ....
14
RFC 7231 says about the Server header:
An origin server MAY generate a Server field in its responses.
MAY is interpreted as in RFC 2119:
MAY This word, or the adjective "OPTIONAL", mean that an item is
truly optional. One vendor may choose to include the item because a
particular marketplace requires it or because the vendor feels that
it ...
11
For those who land here in frustration looking for solution to the same problem in Varnish 4, see below:
varnishlog -a -w /var/log/varnish/varnish50x.log -q "RespStatus >= 500 or BerespStatus >= 500"
Varnish 4 did away with the -m option in varnishlog and instead came up with a new filter query language called VSL which is available in the new option ...
10
I've encounter 503 errors the time that my server is not much of load. I tried to avoid that problem by increasing the timeout in vcl conf
first_byte_timeout = 300s; (by default this is set to 60s) <- probably this one can answer your question (or not) :)
Kindly check this out
http://vincentfretin.ecreall.com/articles/varnish-guru-meditation-on-...
9
You'd want something like this in your vcl_recv:
if (req.url ~ "^/path/to/exclude/") {
return (pass);
}
You'll probably need to familiarize yourself with the basics of what the default VCL is doing and adjust for your content - for example, by default, it'll avoid caching anything when the client sends any cookies on the assumption that your content ...
9
Configure it to run on any port you like! 81. 8081. Anything really.
7
You need to name storage respectively as follows and in vcl you specify which backend storage you want to use with beresp.storage = storage_name.
.
Varnish 3.* process options
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s foo=malloc,512m \
...
7
if you use CentOS or Red Hat, use this:
service varnish stop
if you want to uninstall varnish, try this:
yum erase varnish
or refer to https://stackoverflow.com/questions/27739853/varnish-cache-server-error-503-service-unavailable
7
Try the following for Varnish 4 while updating the TLD with your respective values:
sub vcl_recv {
if ( (req.http.host ~ "^(?i)www.domain.com") && req.http.X-Forwarded-Proto !~ "(?i)https") {
return (synth(750, ""));
}
}
sub vcl_synth {
if (resp.status == 750) {
set resp.status = 301;
set resp....
7
I don't know whose instructions you were following, but either the instructions were wrong, or they were the wrong instructions.
On a clean CentOS 7 system you can install Varnish 4 simply by:
yum install epel-release
yum install varnish
Since your system is no longer clean, you will have to clean it up first. In particular:
You installed the jemalloc ...
7
In your case, I think your best shot would be the command varnishadm vcl.show boot (replace boot by active configuration's name, see varnishadm vcl.list output).
6
I had the same error after updating my server and found the following error in /var/log/nginx/error.log:/var/run/php5-fpm.sock failed (13: Permission denied)
What solved the issue was uncommenting the following lines in /etc/php5/fpm/pool.d/www.conf and restart php5-fpm
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
6
We use Squid for this; the nice thing about squid is that you can set individual expiry of objects based on a pattern match, fairly easily, which allows the metadata from the yum repo to be purged fairly quickly. The config we have which implements this:
refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880
refresh_pattern (\.xml|xml\.gz)$ ...
6
Note that the drop-in should have an empty ExecStart=
Otherwise you will get an error starting the service (duplicate ExecStart)
sudo mkdir /etc/systemd/system/varnish.service.d
sudo nano /etc/systemd/system/varnish.service.d/varnishd.conf
With
[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T <YOUR WEBSERVER IP>:...
5
Varnish actually supports multiple storage backends. You can cache something to RAM, whereas something to disk:
-s default=malloc,256m \
-s static=file,/var/lib/varnish/varnish_storage.bin,1G"
This is most useful to conditionally store static files cache (.css, .js files) to disk, and storing full page cache into RAM. You will need to adjust your .vcl file ...
5
This can be done using std.ip, assuming ClientIP header is already set in vcl_recv(). For example:
vcl 4.0;
import std;
acl admin_net {
"10.10.1.160/27";
}
sub vcl_deliver {
if (std.ip(req.http.ClientIP,"0.0.0.0") ~ admin_net) {
// do something ...
}
}
5
I agree to Michael's short response to abuse/use vcl_error for this purpose and like to show you what we meant in the example code below.
To abuse vcl_error I have used here error codes outside of the HTTP standard and implement a special treatment for this kind of errors.
Example in VCL_RECV:
sub vcl_recv {
...
# respond HTTP 200 to /ping requests
...
5
That article is over 3 years old and there's been 2 stable releases of HAProxy since then, so I suspect that's why it's not working verbatim.
If I had to guess your exact problem, I'd say it has to do with the syntax for the monitor fail statement, where it expects an ACL name after the if.
You have to options to fix this.
Change the ACL to be an "...
5
Varnish does not support SSL
Running a site/service on SSL is a necessity for anything half serious, so thats one reason to use haproxy which will do SSL termination for you.
5
Ended up with this table:
**Nginx**: ngx_http_sub_module --> http://p.defau.lt/?A6UukYFKZrYLalJ5KOuTKg
**Nginx**: Nginx + lua. LUA body_filter --> http://p.defau.lt/?0ZxLWENoEkFIRD17QlEtbQ
**Apache**: Apache Module mod_substitute --> http://p.defau.lt/?IS8u2mnAQTSHs4qEl_MSFQ
**Apache**: As of version 2.3 LUA can be used as a scripting language ...
5
That's a definitive use case for a proxy. A normal proxy, not a reverse-proxy (aka. load balancers).
The most well-known and free and open-source is squid. Luckily it's one of the few good open-source software that can easily be installed with a single apt-get install squid3 and configured with a single file /etc/squid3/squid.conf.
We'll go over the good ...
5
@Michael Hampton's answer is missing some points so here it goes:
Varnish is the one software that does HTTP/2 in Hitch+Varnish combo, but most of the browsers requires TLS connection in order for HTTP/2 to work. That is, TLS connection is required for HTTP/2, its base requirement nowadays.
Varnish Plus does support TLS, while Varnish open source doesn't.
...
4
x-forwarded-for header
Of course, we get the IP of the ELB instead of the real client ip...
You'll want to make use of the x-forwarded-for header which will give you the client ip.
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/TerminologyandKeyConcepts.html#x-forwarded-for
The X-Forwarded-For request header helps you identify ...
4
Actually Varnish handles client.ip as a different data type and so you can use ACLs against this value. This doesn't work with text values like req.http.ClientIP (or all req.http.* parameters). So a regex with this text value seems to be a good solution.
Alternatively you can use a specific module to set client.ip. I found this for example: http://...
Only top voted, non community-wiki answers of a minimum length are eligible