0

My current implementation is as follow:

  1. In front, an AWS Load Balancer (Network TCP/UDP type) on dual-stack, that redirects to a fleet of EC2 instances
  2. These EC2 instances are running HaProxy to receive the requests to a list of processes
  3. The list of processes are aiosmtpd instances (from Python).

Since the purpose is to connect via SMTP, I need to know the Client's IP, but I also need to send the response first.

What I noticed is that if

  • I bind the front-end without accept-proxy
  • but redirect the backed server by sending send-proxy
  • And disable Proxy v2 on AWS Load Balancer

it works perfectly ... for IPv4 only (??) !

IPv6 doesn't work, and the Proxy returns me the IP of the AWS Load balancer instead.

So I've tried enabling Proxy v2 at AWS level, setting accept-proxy on the frontend bind and send-proxy on the server backend property.

This time, it works for both IPv4/v6, BUT, it never sends the initial response: No connections are made to the python code until the first line is sent from the client!

In SMTP protocol, this isn't possible: As a server, I need to be the first to send a response.

What is happening?

Here's my HaProxy configuration:

global
    log /dev/log    local0
    log /dev/log    local1 notice
    chroot /var/lib/haproxy
    user haproxy
    group haproxy
    daemon

    maxconn 60000

    # Default ciphers to use on SSL-enabled listening sockets.
    # For more information, see ciphers(1SSL). This list is from:
    #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
    # An alternative list with additional directives can be obtained from
    #  https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options prefer-client-ciphers no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets

    ssl-default-server-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
    ssl-default-server-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets


defaults
    timeout connect 5s
    timeout client 30s
    timeout server 30s
    mode tcp


frontend smtp
    bind :25 accept-proxy
    bind :::25 accept-proxy

    default_backend smtp_backend


backend smtp_backend
    mode tcp
    timeout server 1m
    timeout connect 5s

    server srv1 127.0.0.1:2525 send-proxy maxconn 500

I tried adding/removing either or both or none of accept-proxy and send-proxy and even send-proxy-v2. Without any luck!

The closest I've been to having it work is with no Proxy v2 enabled on AWS's end, no accept-proxy on the frontend part and send-proxy on the client, but it doesn't work for IPv6.


I've built a basic script to describe the issue:

# -*- config:utf-8 -*-
import socket, argparse


class TestingServer(object):
    def __init__(self, host, port):
        self.sock = socket.socket(socket.AF_INET6)
        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
        self.sock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, False)
        self.sock.bind((host, port))
        print('Setting up listener on {}:{}'.format(host, port))

    def listen(self):
        print('Waiting on connections ...')
        self.sock.listen()

        while True:
            try:
                client, address = self.sock.accept()
                print('Got a connection!', address)
                client.send(b'250 Sending initial data.\r\n')
                data = client.recv(1024)
                print('Received data:', data)
                client.send(b'250 Gotcha')
                data = client.recv(1024)
                print('Received data 2:', data)
                client.send(b'250 Second gotcha')
                client.close()
            except Exception as e:
                print ('Got an exception!', e)
                pass
            finally:
                try:
                    client.close()
                except Exception:
                    pass


if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Start an SMTPD daemon.')
    parser.add_argument('--host', nargs='?', default='localhost', type=str)
    parser.add_argument('--port', nargs='?', default=2552, type=int)
    args = parser.parse_args()

    TestingServer(args.host, args.port).listen()

When setting AWS Proxy v2 enabled, with accept-proxy for bind and send-proxy for server, here's what's happening:

I start the server :

$> python testserver.py --port 2525 --host ::

Setting up listener on :::2525

Waiting on connections ...

On my local machine, I do a :

$> telnet {aws-loadbalancer-name}.elb.eu-west-3.amazonaws.com 25

Trying {ipv6}...

Connected to {aws-loadbalancer-name}.elb.eu-west-3.amazonaws.com.

Escape character is '^]'.

Nothing on the server yet.

On the client's end (telnet), I write anything:

$> a [enter]

Then, as soon as I hit enter, the server shows the following:

Address is: ('::ffff:127.0.0.1', 42494, 0, 0)

Got a connection! <socket.socket fd=4, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, laddr=('::ffff:127.0.0.1', 2525, 0, 0), raddr=('::ffff:127.0.0.1', 42494, 0, 0)> ('::ffff:127.0.0.1', 42494, 0, 0)

Received data: b'PROXY TCP6 {clients_ip} {loadbalancer_ip} 44012 25\r\ns\r\n'

So clearly, for some reason, HaProxy is not sending the connection to the Python process as soon as one is created, but it waits for a first data to come.

How can I avoid that?

Thank you in advance!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.