I have a pretty good handle on load balancing web applications with nginx/HAProxy/etc. As I understand it, in these cases you're mostly limited by things like the number of concurrent connections and TLS handshakes, but each request is a relatively small amount of data transferred.
I'm currently working on a service that transfers a lot of data per request. Think video streaming or peer-to-peer file transfers proxied through my server.
I'm wondering what the typical way to load balance something like this is? Even if HAProxy could handle the bandwidth, having everything going through a single VPS would saturate its network fairly easily (at least on DigitalOcean; maybe an AWS 25Gbps instance would be enough). I'm thinking redirects might be the way to go, but I'd like to avoid that and wanted to see if there's a better way.
One other piece of information about my service is that requests to the same URL must go to the same upstream server. But it only cares about the path. Query params, headers, etc don't matter.
I did a quick check on youtube, and it looks like they use redirects to almost random-looking domains like r5---sn-qxo7rn7l.googlevideo.com
, r1---sn-qxoedn7e.googlevideo.com
.
EDIT: Additional details at Tim's request:
The data should be considered uncacheable. Imagine peer1 has a 4GB video file they want to share with peer2. peer1 connects to lb.example.com/path
and waits for peer2 to connect. peer2 connects to lb.example.com/path
, and and the data is streamed through the server from peer1 to peer2.
The way I would do this with redirects is peer1 connects to lb.example.com/path
. path is hashed, and the value of the hash is used to determine whether to redirect peer1 to instance1.example.com
or instance2.example.com
. When peer2 connects with the same path, it ends up on the same upstream instance. The hash space would be evenly divided between instance1 and instance2.
I agree AWS egress is too expensive which is a big part of why I'm trying to design a scalable solution that doesn't depend on a single huge network pipe.