3

I currently have HAProxy running in a Docker container. Docker is just running on one host with IPv4 and IPv6 enabled. HAProxy is the only container with host ports shared and only 80/443 come through the firewall on the external/public interface.

Currently web traffic comes in fine. I have forwardfor set in HAProxy and my backends get the real ClientIP for logging and analytics .. except for IPv6. Although port 80/443 get mapped/translated in such a way by docker so that HAProxy gets the real client IP on IPv4, anything over IPv6 somehow gets translated to IPv4! HAProxy gets a private 172.17.0.1 address as the client IP for all IPv6 connections.

My daemon.json for Docker looks like the following:

{
    "tls": true,
    "tlsverify": true,
    "tlscacert": "/etc/docker/ca.pem",
    "tlscert": "/etc/docker/server.crt",
    "tlskey": "/etc/docker/server-key.pem",
    "ipv6": true,
    "fixed-cidr-v6": "2001:19f0:6001:1c12::/80",
    "hosts": ["127.0.0.1:2376", "10.10.6.10:2376", "fd://"]
}

I've assigned Docker a /80 from my IPv6 subnet. I'm using Vultr as a provider and they assign me 2001:19f0:6001:1c12:: for my IPv6 subnet and 2001:19f0:6001:1c12:5400:01ff:fe49:876e for my host IP address (the machine running docker). I also my DEFAULT_FORWARD policy as ACCEPT in ufw, and have ndppd setup with the following configuration:

proxy ens3 {
  timeout 500
  ttl 30000
  rule 2001:19f0:6001:1c12::/80 {
    static
  }
}

So with IPv6 enabled, I realize my containers all get their own IPv6 addresses. So for my HAProxy instance to get a real Client IP, I need to point the DNS to the container IPv6 address and open a firewall rule for it. But the trouble is, I need that address to be static. Of course this means I can't use the default bridge network and I need to have a user defined network.

I realize I should have done this to being with, and then I remembered what my issue was. I keep getting the following:

.gem/ruby/2.2.0/gems/docker-api-1.33.6/lib/docker/connection.rb:50:in `rescue in request': could not find an available, non-overlapping IPv6 address pool among the defaults to assign to the network (Docker::Error::ServerError)

This is from the Ruby Docker-API library, but the I get the exact same error with the command line tool. I've tried various slices of IPv6 subnets, but none seem to work and people have told me I'll have problems if I use less than a /64.

Remember, the original problem is that I can't get the client-ip in my web server logs for IPv6 connections. If there's a simpler way to solve this, I'm all open to it. If not, what settings do I need to use for my user defined network? I wish I could just delete the default bridge and use its ranges, but that doesn't seem to be an option. Can I keep the default bridge from getting that IPv6 subnet that's defined in the daemon.json and use that range instead for my user defined network (and then statically assign just the HAProxy container so I could use that container's IP in the DNS/AAAA record?) Or is there a simpler way to get IPv6 host ports to connect directly to the container without translation in the same way IPv4 is done?

1

I ended up having to create a user defined network. After I refactored everything, I did discover an IPv6 NAT container that might have accomplished the same thing, but I'm glad I did it right instead of trying to use IPv6 NAT to hack it.

First, I split my IPv6 range so my daemon.json got the following:

"fixed-cidr-v6": "2001:19f0:6001:1c12::1:0:0/96",

and my user defined network got 2001:19f0:6001:1c12::2:0:0/96. I defined a static IPv6 address within the ::2:0:0/96 subnet and used that IP when I created my container using the Docker Engine API:

 ...
 "NetworkingConfig"=>
  {"EndpointsConfig"=>
    {"my-custom-network"=>
      {"IPAMConfig"=>{"IPv6Address"=>"2001:19f0:6001:1c12::2:0:1000"}}}},
 ...

Then I set my AAAA records in DNS to that custom IPv6 address. Finally, I made sure that HAProxy was listening on IPv4 and IPv6 using bind :::80 v4v6 and bind :::443 v4v6 <insert cert stuff> configuration options.

  • Is this in swarm mode? – Meezaan-ud-Din Jul 14 at 9:52
  • @Meezaan-ud-Din No, this isn't in Swarm mode. – djsumdog Jul 14 at 16:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.