0

I am trying to get nginx working to load balance udp traffic. The upstream servers are configured for DSR and do not pass traffic back through nginx. I have a group of ports that I need to forward to the upstream servers while preserving the server port. So I need the traffic to get to nginx, nginx look at the list of ips of upstream servers and pick one using "hash $remote_addr consistent", then send the traffic to the ip of the chosen upstream server with the ip of the incoming client and the original destination port it came to nginx on. Then the upstream server will receive the traffic as if it didn't go through nginx. Any thoughts?

I have tried using a range with listen 9000-9999; but it doesn't work and gives an error "host not found in "9000-9999" of the "listen" directive." So I have a listen line for each port which is a real pain.

stream {

    upstream stream_backend {
        hash $remote_addr consistent;
        server 10.10.10.14:8999;
    }
    
    
    server { #use this for upstream lb
        listen     8999;
        proxy_pass stream_backend;
        proxy_bind $remote_addr:$remote_port transparent;
        proxy_responses 0;
    }
    
    server { #test going directly to ip
        listen 9000;
        listen 9001;
        listen 9002;
        listen 9003;
        #listen lines continue for whole port range
        proxy_pass 10.10.10.30:$server_port; #used this to go directly to a server for testing
        proxy_bind $remote_addr:$remote_port transparent;
        proxy_responses 0;
    }
    
}

Is there any way to have a range of ports? Instead of:

upstream stream_backend {
        hash $remote_addr consistent;
        server 10.10.10.14:8999;
    }

something like this?

upstream stream_backend {
        hash $remote_addr consistent;
        server 10.10.10.14; or server 10.10.10.14:8999-9003 or server 10.10.10.14:$server_port
    }

1 Answer 1

0

After re-reading your question, I see requirements that cannot be filled by a proxy like nginx.

then send the traffic to the ip of the chosen upstream server with the ip of the incoming client and the original destination port it came to nginx on

nginx uses normal socket functions for initiating TCP connections / sending UDP datagrams. This means that the network stack in kernel chooses the IP addresses for the outgoing packets.

nginx cannot send packets where the source address is something else than an address the server has.

You can use IPTables DNAT feature to accomplish your goal. DNAT replaces the destination IP address and UDP/TCP port in incoming packets, and then forwards them to the destination. This method preserves source IP address.

An example rule set could be:

iptables -t nat -I PREROUTING -p udp -m multiport --dports 9000:9003 -m statistic --mode nth --every 3 --packet 0 -j DNAT --to-destination 10.0.0.2:8999
iptables -t nat -I PREROUTING -p udp -m multiport --dports 9000:9003 -m statistic --mode nth --every 2 --packet 0 -j DNAT --to-destination 10.0.0.3:8999
iptables -t nat -I PREROUTING -p udp -m multiport --dports 9000:9003 -m statistic --mode nth -j DNAT --to-destination 10.0.0.3:8999

These rules match incoming traffic to ports 9000:9003, and to a destination NAT to distribute them to different servers.

The -m statistic match does the round-robin distribution. The --every parameter is decreasing because of the sequential way iptables processes rules.

Original answer below


Your post title mentions UDP, but your configuration does not include udp in the listen directive.

So, try using:

server {
    listen 8999 udp;
    proxy_pass stream_backend;
    proxy_bind $remote_addr:$remote_port transparent;
    proxy_responses 0;
}
1
  • Sorry, the config I posted didn't explicitly say UDP, but I have tested that and either way has the same result.
    – hotdog
    Apr 19, 2021 at 19:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.