0

I am using Nginx as an SSL terminator in front of varnish. Backends are litespeed. I have two wordpress sites in which both of them uses HTTPS protocol. I have a Haproxy in front of my infrastructure which only passes SSL traffics to Nginx.

One of my websites is loading fast and when I check the status of cached items through firebug, everything get a 'HIT' like .js .css and the domain name: domain1.com.

When I check these items for the second wordpress site, .js and HTML file get 'MISS' the same as the domain name domain2.com.

The configuration of Nginx and Varnish and the backend is the same for both web sites. I also disabled all of its plugins and the ones I think affect the caching :WP Fastest Cache, 404 to 301,Broken Link Checker and some others.

The only difference is that the second web site has a robot which gathers news from other sites and put them on the sites(some kind of web crawler).

What causes the varnish not to cache cache everything on the second site?


Edit:
I added the varnish log for a requests which is not cached:

varnishlog -g request -q "ReqUrl ~ '/wp-content/themes/farda/js/jquery.min.js'"
*   << Request  >> 149       
-   Begin          req 148 rxreq
-   Timestamp      Start: 1499626236.025652 0.000000 0.000000
-   Timestamp      Req: 1499626236.025652 0.000000 0.000000
-   ReqStart       192.168.1.108 57470
-   ReqMethod      GET
-   ReqURL         /wp-content/themes/farda/js/jquery.min.js
-   ReqProtocol    HTTP/1.0
-   ReqHeader      X-Real-IP: 192.168.1.105
-   ReqHeader      X-Forwarded-For: 192.168.1.105
-   ReqHeader      X-Forwarded-Proto: https
-   ReqHeader      X-Nginx: on
-   ReqHeader      Host: mytourguide.ir
-   ReqHeader      Connection: close
-   ReqHeader      User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0
-   ReqHeader      Accept: */*
-   ReqHeader      Accept-Language: en-US,en;q=0.5
-   ReqHeader      Accept-Encoding: gzip, deflate, br
-   ReqHeader      Referer: https://mytourguide.ir/
-   ReqHeader      Pragma: no-cache
-   ReqHeader      Cache-Control: no-cache
-   ReqUnset       X-Forwarded-For: 192.168.1.105
-   ReqHeader      X-Forwarded-For: 192.168.1.105, 192.168.1.108
-   VCL_call       RECV
-   ReqHeader      Cookie: 
-   ReqUnset       Cookie: 
-   ReqHeader      Cookie: 
-   ReqUnset       Cookie: 
-   ReqHeader      Cookie: 
-   ReqUnset       Cookie: 
-   ReqHeader      Cookie: 
-   ReqUnset       Cookie: 
-   ReqUnset       Accept-Encoding: gzip, deflate, br
-   ReqHeader      Accept-Encoding: gzip
-   ReqUnset       X-Forwarded-For: 192.168.1.105, 192.168.1.108
-   ReqHeader      X-Forwarded-For: 192.168.1.108
-   ReqUnset       Accept-Language: en-US,en;q=0.5
-   ReqUnset       User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0
-   ReqHeader      cookie: 
-   ReqUnset       cookie: 
-   ReqHeader      cookie: 
-   VCL_return     hash
-   VCL_call       HASH
-   VCL_return     lookup
-   Debug          "XXXX MISS"
-   VCL_call       MISS
-   VCL_return     fetch
-   Link           bereq 150 fetch
-   Timestamp      Fetch: 1499626236.029726 0.004075 0.004075
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespHeader     Cache-Control: max-age=2592000, private
-   RespHeader     Expires: Mon, 09 Jul 2018 18:50:18 GMT
-   RespHeader     Last-Modified: Tue, 17 Jan 2017 14:46:21 GMT
-   RespHeader     Content-Type: text/javascript; charset=UTF-8
-   RespHeader     Content-Length: 33342
-   RespHeader     Content-Encoding: gzip
-   RespHeader     Vary: Accept-Encoding,User-Agent
-   RespHeader     Date: Sun, 09 Jul 2017 18:50:18 GMT
-   RespHeader     Accept-Ranges: bytes
-   RespHeader     Server: LiteSpeed
-   RespHeader     X-Varnish: 149
-   RespHeader     Age: 0
-   RespHeader     Via: 1.1 varnish-v4
-   VCL_call       DELIVER
-   RespHeader     X-Cache: MISS
-   RespUnset      X-Varnish: 149
-   RespUnset      Via: 1.1 varnish-v4
-   RespHeader     X-Configured-By: ServerSetup.ir
-   VCL_return     deliver
-   Timestamp      Process: 1499626236.029739 0.004087 0.000012
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: close
-   RespHeader     Accept-Ranges: bytes
-   Timestamp      Resp: 1499626236.030691 0.005039 0.000952
-   Debug          "XXX REF 2"
-   ReqAcct        445 0 445 448 33342 33790
-   End            
**  << BeReq    >> 150       
--  Begin          bereq 149 fetch
--  Timestamp      Start: 1499626236.025722 0.000000 0.000000
--  BereqMethod    GET
--  BereqURL       /wp-content/themes/farda/js/jquery.min.js
--  BereqProtocol  HTTP/1.0
--  BereqHeader    X-Real-IP: 192.168.1.105
--  BereqHeader    X-Forwarded-Proto: https
--  BereqHeader    X-Nginx: on
--  BereqHeader    Host: mytourguide.ir
--  BereqHeader    Accept: */*
--  BereqHeader    Referer: https://mytourguide.ir/
--  BereqHeader    Pragma: no-cache
--  BereqHeader    Accept-Encoding: gzip
--  BereqHeader    X-Forwarded-For: 192.168.1.108
--  BereqHeader    cookie: 
--  BereqProtocol  HTTP/1.1
--  BereqHeader    X-Varnish: 150
--  VCL_call       BACKEND_FETCH
--  VCL_return     fetch
--  Backend        17 apache web1(192.168.1.11,,5050)
--  Timestamp      Bereq: 1499626236.025798 0.000076 0.000076
--  Timestamp      Beresp: 1499626236.027971 0.002249 0.002173
--  BerespProtocol HTTP/1.1
--  BerespStatus   200
--  BerespReason   OK
--  BerespHeader   Cache-Control: max-age=2592000, private
--  BerespHeader   Expires: Mon, 09 Jul 2018 18:50:18 GMT
--  BerespHeader   Last-Modified: Tue, 17 Jan 2017 14:46:21 GMT
--  BerespHeader   Content-Type: text/javascript; charset=UTF-8
--  BerespHeader   Content-Length: 33342
--  BerespHeader   Content-Encoding: gzip
--  BerespHeader   Vary: Accept-Encoding,User-Agent
--  BerespHeader   Date: Sun, 09 Jul 2017 18:50:18 GMT
--  BerespHeader   Accept-Ranges: bytes
--  BerespHeader   Server: LiteSpeed
--  BerespHeader   Connection: Keep-Alive
--  TTL            RFC 2592000 -1 -1 1499626236 1499626236 1499626218 1531162218 2592000
--  VCL_call       BACKEND_RESPONSE
--  TTL            VCL 120 10 0 1499626236
--  VCL_return     deliver
--  Storage        malloc Transient
--  ObjProtocol    HTTP/1.1
--  ObjStatus      200
--  ObjReason      OK
--  ObjHeader      Cache-Control: max-age=2592000, private
--  ObjHeader      Expires: Mon, 09 Jul 2018 18:50:18 GMT
--  ObjHeader      Last-Modified: Tue, 17 Jan 2017 14:46:21 GMT
--  ObjHeader      Content-Type: text/javascript; charset=UTF-8
--  ObjHeader      Content-Length: 33342
--  ObjHeader      Content-Encoding: gzip
--  ObjHeader      Vary: Accept-Encoding,User-Agent
--  ObjHeader      Date: Sun, 09 Jul 2017 18:50:18 GMT
--  ObjHeader      Accept-Ranges: bytes
--  ObjHeader      Server: LiteSpeed
--  Fetch_Body     3 length stream
--  Gzip           u F - 33342 95992 80 206366 266672
--  BackendReuse   17 web1(192.168.1.11,,5050)
--  Timestamp      BerespBody: 1499626236.030637 0.004915 0.002666
--  Length         33342
--  BereqAcct      291 0 291 375 33342 33717
--  End            
3
  • If you can add the output of varnishlog -g request from a request you expect to be cached, but which isn't that'll make it much easier to figure out what's wrong. Jul 9 '17 at 15:04
  • I doubt anyone can tell you based on the information you've provided. You're probably going to need to edit your question to provide a few things, including a) curl showing response headers to each layer (not just Haproxy, also direct curls to the web server(s), b) configurations c) possibly logs. I suggest you look at the caching headers, and also you need to format your question better - I've fixed it up twice now.
    – Tim
    Jul 9 '17 at 18:54
  • Thank you. you are right. I added the varnish log for a requests.
    – Sinai
    Jul 9 '17 at 18:56
1

The problem is with backend (LiteSpeed).

Note this line in your varnishlog:

--  ObjHeader      Cache-Control: max-age=2592000, private

Varnish built-in VCL will not cache URLs which send Cache-Control: private. Thus it's always being not cached, and MISS-ed later on.

Proper solution: configure LiteSpeed to enable public cache as opposed to private cache.

5
  • Thank you @Daniel V. I think you are right. But there is another problem. My litespeed license is for VPS with one core ,so I cannot enable public cache! because cache configuration is not accessible with this license. Should I change the web server to something else like nginx or apache? Or there is another way to enabling public cache in litespeed.
    – Sinai
    Jul 10 '17 at 5:32
  • Another matter is that, I am using Litespeed for two wordpress sites. Why litespeed sends private for Cache-Control for one site and does not send anything for the other one?!! I think something more is changing the Cache-Control parameter in the backend, am I right?
    – Sinai
    Jul 10 '17 at 5:49
  • @Sinai, this header could very well be sent by PHP (via Wordpress plugin for example), but I don't see any rationale in any plugin doing that in a way that it will affect entire website. So I think in most probability the difference is with LiteSpeed settings. I have little understanding about LiteSpeed settings Jul 10 '17 at 13:58
  • Nginx doesn't mess with with Cache-Control header unless you tell it too so it might be worth using it instead. Anyway, have you checked the possible ways to set public cache on the linked page in pargraph "Set cache policy at LSWS Server level"? You should be able to enable proper header with .htaccess <IfModule LiteSpeed> CacheLookup public on </IfModule> Jul 10 '17 at 13:58
  • Thank you @Daniel. The litespeed configurations for both web sites are the same because it is a shared hosting server which has several web sites. Yes I tested CachLookup but it did not work. It is because of the license. I switched to nginx and now everything is cached fine. I also use fastcgi_cache of the nginx too. But now I have other problems such as rewrite rules for a prestashop website in nginx!! I am struggling with that:D
    – Sinai
    Jul 11 '17 at 8:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.