Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to setup a cache server for Rubygems, as I'm currently based in Vietnam and the international internet connections are pretty slow. I have been trying to get this done via Varnish, but after hours of googling and trying various things I'm still stuck and can't get it to work properly.

My goal

This a sample request group when I'm installing a gem:

302 Moved Temporarily
200 OK

I want to setup a reverse proxy cache server (e.g. where I can perform the following request and the cache server would follow any redirects internally.

The redirect location will link to various domains (some rubygems subdomains, Amazon S3, rubygems mirror).

Current State

After fiddling around with nginx, I found this blog post which is pretty close what I wanna achieve. However, I have too little knowledge about how Varnish works to get it working properly.

That's my current config file

import std;

backend rubygems {
    .host = "";
    .port = "80";

sub vcl_recv {
    std.syslog(180, "RECV: " + + req.url);
    if (!req.url  ~ "^http") {
      std.syslog(180, "FETCH");
      set req.backend = rubygems;
      return (lookup);

sub vcl_fetch {
    if (beresp.status == 302) {
        set beresp.http.X-Magic-Redirect = "1";

sub vcl_hit {
    if (obj.http.X-Magic-Redirect == "1") {
        set req.url = obj.http.Location;
        return (restart);

sub vcl_deliver {
    if (resp.http.X-Magic-Redirect == "1") {
        unset resp.http.X-Magic-Redirect;

I can perform a request, but it responds with an error:

curl -is http://localhost:8080/latest_specs.4.8.gz
HTTP/1.1 302 Found
Server: Varnish
Content-Type: text/html; charset=utf-8
Retry-After: 5
Content-Length: 376
Accept-Ranges: bytes
Date: Sat, 01 Feb 2014 02:33:47 GMT
X-Varnish: 933109322
Age: 1
Via: 1.1 varnish
Connection: close

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    <title>302 Found</title>
    <h1>Error 302 Found</h1>
    <h3>Guru Meditation:</h3>
    <p>XID: 933109322</p>
    <p>Varnish cache server</p>

and that's the corresponding syslog output for the request:

Jan 31 18:33:46 precise64 varnishd[2387]: RECV: localhost:8080/latest_specs.4.8.gz
Jan 31 18:33:46 precise64 varnishd[2387]: FETCH
Jan 31 18:33:47 precise64 varnishd[2387]: RECV: localhost:8080/latest_specs.4.8.gz
Jan 31 18:33:47 precise64 varnishd[2387]: FETCH
Jan 31 18:33:47 precise64 varnishd[2387]: RECV: localhost:8080

So, the request to Rubygems is working fine, but following the redirect does not work as expected. I would be glad if somebody could point me in the right direction.

share|improve this question
I think you're missing an adequate backend once the request have been restarted. Anyway, you should also change fill and req.url appropiately when following the redirect – NITEMAN Feb 11 '14 at 8:37

At this point when you will receive a 302 status from your rubygems backend, you have to make a request again at the new location specify by the HTTP header Location in the response.

You should end up with something like that:

vcl_fetch {
   if (beresp.status == 302) {  /* The content is on another location */

      /* First change the host of the request*/
      set = regsub(regsub(beresp.http.Location, "^http://", ""), "^([^/]+)/.*$", "\1");

      /* Then change the url of the request */
      set req.url = regsub(beresp.http.Location, "^http://[^/]+/(.*)$", "/\1");
      return (restart);
share|improve this answer

If you get that working, please share.
Especially whether it works with the bundler dependency protocol.

There is some good information here Alternatively, you may want to take a look at geminabox

share|improve this answer
While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. – kasperd Nov 22 '14 at 11:23

Another approach is which provides a proxy that will cache the gems on the way through.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.