0

I have set up nginx reverse proxy and added caching to it using nginx caching.

I can verify whether the requested copy is cached or not using added header.

add_header X-Cache-Status $upstream_cache_status;

i have set to cache page if its get accessed more than twice.

proxy_cache_min_uses    2;

Nginx caching is working as it should, i can see the X-Cache-Status: MISS twice and later X-Cache-Status: HIT if i try to access the webpage from any browser.

But if i try to check same with curl , if the page is not cached i am getting

X-Cache-Status: MISS

and if the page is cached i am getting

X-Cache-Status: HIT

But my issue is , i cant able to warm up cache using curl. i mean i cant make webserver make cache specific page by accessing page using curl even if i access it 10 times using curl.

why is that ?

So far i have tried .

curl -I https://example.com/page1
curl -v https://example.com/page1
curl --verbose https://example.com/page1
curl -svo /dev/null https://example.com/page1

using latest centos 7x64

latest Nginx

[root]# curl -V
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.19.1 Basic ECC zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz

update1: here is my nginx config file

proxy_cache_path    /usr/local/nginx/cache levels=1:2   keys_zone=mycache:10m       max_size=10g    use_temp_path=off;

server {

access_log  /dev/null;
listen my_ip:80;
server_name example.com ;

location / {
    access_log off;

    proxy_cache             mycache;
    proxy_cache_min_uses    2;
    proxy_cache_valid       200     10m;
    proxy_cache_valid       404     1m;

    proxy_cache_use_stale   error timeout invalid_header updating http_500 http_502 http_503 http_504;  
    proxy_cache_lock        on;

    proxy_pass http://box1.example.com/;

    proxy_set_header    X-Real-IP   $remote_addr;
    proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;

    #hide headers 
    proxy_hide_header Cache-Control;

    #ignore header
    proxy_ignore_headers Cache-Control Expires;

    #add custom headers for users
    add_header X-Cache-Status $upstream_cache_status;
    expires 3h;
    add_header Cache-Control "public, max-age=10800, s-maxage=10800";

    #add no mobile redirect cookie
    add_header Set-Cookie "__cf_mob_redir=0;Domain=.example.com;";
}
}

update 2:
I am using cloudflare on my domain. I have tried disabling cloudflare service altogether and only using it as a DNS server . Tried again with curl and got same response.

It does not matter whether cloudflare is enabled or not curl cant make pages cache.


Update 3 :

Tried using different User-agnts but received same response.

curl -A "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3" -svo /dev/null http://example.com/some-page
  • Please share your Nginx config, especially the part around caching. – Tim Nov 17 '16 at 4:28
  • You're using old http1 cache control headers, have a read of a tutorial I wrote about that photographerstechsupport.com/tutorials/… . This doesn't answer your question but is worth considering. Short version: only header required is "Cache-Control". Unfortunately I can't answer your curl question. If it's not causing a problem I'd probably let it go, but it'd be interesting to get an answer. – Tim Nov 17 '16 at 18:47
  • @Tim Thanks updated config file. and still the same. – AMB Nov 23 '16 at 13:49
  • Do you see cache misses from web browsers? Try Firefox and maybe Chrome with the plug "Live HTTP Headers" it's ideal for this. If web browsers work but curl doesn't then there's no actual problem, just curiosity. – Tim Nov 23 '16 at 18:22
  • @Tim yes, i can see MISS twice then HIT afterwards in browser. practically its working perfectly. but i would like to warm up cache from time to time. so i need curl to cache pages too. and i am using cloudflare on my website, i even tried disabling cloudflare and trying with curl, but still the same. – AMB Nov 25 '16 at 5:25
0

0) As I can see you check https:// site but configuration for http:// version. I guess it's not a problem in your case but better to check that you edit right configuration file.
1) You should check what you got from upstream curl -I http://box1.example.com/page1 and I guess that there will be Set-Cookie header. So you should either add Set-Cookie to proxy_ignore_headers or remove cookies from upstream's reply. Or use separate locations: one for pages where you could ignore cookies for caching and another for pages where you can't ignore cookies.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.