0

I'm currently using nginx to cache Steam downloads for my local network as detailed by Multiplay. This works great, but Steam has a feature to watch other users games via a stream not unlike Twitch but on demand. I'm having trouble with my proxy failing to cache the stream. I don't mind (and maybe it is preferable) to not cache this content, but I cannot figure this out as I'm new to nginx.

The server being hit is valve#.cs.steampowered.com and all look for under /broadcast/... Here's a sample request and response:

GET /broadcast/2671935884594669886/manifest/94/?broadcast_origin=br02.broadcast.iad.steamstatic.com:80&viewer=10502638835558921467 HTTP/1.1
Host: valve65.cs.steampowered.com
Accept: application/json, text/javascript, */*; q=0.01
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-CA,en;q=0.8,en-US;q=0.6
Origin: http://steamcommunity.com
Referer: http://steamcommunity.com/broadcast/watch/76561198065147403
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36

HTTP/1.1 404 Not Found
Connection: keep-alive
Content-Length: 570
Content-Type: text/html
Date: Sun, 07 Feb 2016 03:15:58 GMT
Server: nginx/1.6.2

Here is a sample request and response with the traffic diverted from the proxy:

GET /broadcast/1432112925536035508/manifest/94/?broadcast_origin=valve66.broadcast.sea.steamstatic.com:80&viewer=10502638835558921467 HTTP/1.1
Host: valve63.cs.steampowered.com
Accept: */*
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-CA,en;q=0.8,en-US;q=0.6
Origin: http://steamcommunity.com
Referer: http://steamcommunity.com/broadcast/watch/76561198015566908
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36

HTTP/1.1 200 OK
Access-Control-Allow-Headers: 
Access-Control-Allow-Methods: GET,HEAD,OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Date
cache-control: no-cache,must-revalidate
content-length: 2621
content-type: application/xml
Date: Sun, 07 Feb 2016 03:25:20 GMT
expires: Mon, 26 Jul 1997 05:00:00 GMT

The expires looks kind of funny (1997?) but no-cache is set. The relevant configs I'm using can be found on my GitHub with node-steam being the appropriate one being used.

I've tried adding the following with no luck (before location /):

location /broadcast/ {
    proxy_cache_bypass $arg_nocache;
    proxy_no_cache $arg_nocache;
}

What can I do to bypass proxying these requests?

Edit: Here's some more detail on the actual problem. I use nginx as a reverse proxy to cache Steam downloads, which come from *.cs.steampowered.com and this works fine. Steam also has a broadcasting feature, where users can watch others play their games on a livestream. There's a list of them here. When trying to watch one, the broadcast will sit there loading indefinitely, but the chat and rest of the page will load. Going around nginx removes this problem.

After looking at the requests for the livestream, I see that any requests to *.cs.steampowered.com/broadcast/ return 404's. There's some unfortunate overlap here between the content servers and the broadcasting servers. There are no rules for location / so I would have thought that these would just get passed through the proxy anyway.

  • 1
    Could you create a separate paragraph and explain what the actual issue is in more detail? – 030 Feb 7 '16 at 14:57
  • @Alfred Just updated the problem with some more detail. – drglove Feb 7 '16 at 20:43
  • What do the logs say? – 030 Feb 7 '16 at 22:17
  • The logs weren't very helpful. access.log showed the 404 response, and error.log complained about not being able to find the cached file. For some reason, clearing the cache that was being used before seems to have fixed my problems. – drglove Feb 7 '16 at 22:39
0

There were a few things going on. location / was being cached which caught the traffic initially.

Adding the following rules worked, but only after clearing the cache used initially for some reason I still have yet to figure out.

location /broadcast/ {
    # Proxy
    proxy_next_upstream error timeout http_404;
    proxy_pass http://$host$request_uri;
    proxy_redirect off;

    # Upstream request headers
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    # Useful headers for debugging / stats
    add_header X-Upstream-Status $upstream_status;
    add_header X-Upstream-Response-Time $upstream_response_time;
    add_header X-Upstream-Cache-Status $upstream_cache_status;

    # New settings - 2014-04-12 (i52)
    proxy_ignore_client_abort on;

    # Increase proxy timeout to increase throughput
    proxy_read_timeout 300;
}

The contents of this location are the same way I handle proxying everywhere else.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.