5

I have a server running multiple docker containers in the following configuration:

  • One of the containers is a reverse proxy binding to the exposed ports of the other containers. This is the only container accepting connections from the outside world on http port 80
  • All other containers are development environments running tty shells. These have exposed ports on the host, but those ports are not exposed to the outside world due to EC2 security group configuration on the host. So the only way to communicate with them is through the reverse proxy. The reverse proxy communicates with them through their exposed ports.
    n.b. I cannot use container links here as I don't want to restart the reverse proxy container with each new development container.

                    |                                DEV Env Docker
                    |                              / 
OUTSIDE WORLD <-----|------> REVERSE PROXY DOCKER  - DEV Env Docker
                    |                              \ 
                    |                                DEV Env Docker

The idea is that a user can access the development container through the reverse proxy and run commands inside that container.

I want to prevent users from running commands connecting to the outside world by whitelisting the domains they can connect to.

I have installed squid3 and setup a white list by adding the following lines to the config file:

acl whitelist dstdomain "/etc/squid3/whitelist.txt"
http_access allow whitelist

I also managed to redirect traffic from the docker containers to squid using the following iptables command iptables -t nat -A PREROUTING -i docker0 -p tcp -d 0/0 -j REDIRECT --to-port 3128

The issue I have is that I only want outgoing traffic from my Dev Env containers to go through squid, but since "I believe" I need to use PREROUTING all traffic is going through squid even the ones incoming and destined for my reverse proxy.

When I try to access my reverse proxy over the web I get the following error from squid

ERROR
The requested URL could not be retrieved
The following error was encountered while trying to retrieve the URL: /
Invalid URL
Some aspect of the requested URL is incorrect.
Some possible problems are:
Missing or incorrect access protocol (should be http:// or similar)
Missing hostname
Illegal double-escape in the URL-Path
Illegal character in hostname; underscores are not allowed.
Your cache administrator is webmaster.
Generated Fri, 06 Nov 2015 18:56:54 GMT by ip-10-0-1-201 (squid/3.3.8)

QUESTION: How can I get squid to ignore all traffic related to the reverse proxy container both incoming and outgoing?

My iptables is like this

# Generated by iptables-save v1.4.21 on Fri Nov  6 18:54:09 2015
*nat
:PREROUTING ACCEPT [30:1796]
:INPUT ACCEPT [28:1680]
:OUTPUT ACCEPT [37:2388]
:POSTROUTING ACCEPT [46:2964]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A PREROUTING -i docker0 -p tcp -d 0/0 -j REDIRECT --to-port 3128
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 8000 -    j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 80 -j     MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 9000 -j DNAT --to-destination 172.17.0.3:8000
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 8192 -j DNAT --to-destination 172.17.0.3:80
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.17.0.2:80
COMMIT
# Completed on Fri Nov  6 18:54:09 2015
# Generated by iptables-save v1.4.21 on Fri Nov  6 18:54:09 2015
*filter
:INPUT ACCEPT [1891:3910112]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1500:1500230]
:DOCKER - [0:0]
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8000 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
COMMIT
# Completed on Fri Nov  6 18:54:09 2015

---- EDIT ----- My Squid Conf in full after making change to http_port as indicated in the comments.

acl SSL_ports port 443
acl Safe_ports port 80      # http
acl Safe_ports port 21      # ftp
acl Safe_ports port 443     # https
acl Safe_ports port 70      # gopher
acl Safe_ports port 210     # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280     # http-mgmt
acl Safe_ports port 488     # gss-http
acl Safe_ports port 591     # filemaker
acl Safe_ports port 777     # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access allow localhost manager
http_access deny manager
acl allowed_ips src 172.17.0.0-172.17.0.254
http_access allow allowed_ips
http_access allow localhost
http_access deny all
http_port 3128 accel vhost allow-direct
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:       1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .       0   20% 4320

I Appreciate all the help you can provide.

  • I managed to get around the squid error by changing the squid http_port directive from http_port 3128 to http_port 3128 accel vhost allow-direct. I then had another error about restricted url which I managed to bypass with acl allowed_ips src 172.17.0.0-172.17.0.254 http_access allow allowed_ips but now all connections from the dev env containers are allowed to the outside world. I am at the point of guessing here, so still need help setting squid up correctly. – ajaali Nov 6 '15 at 22:37
1

The trick is to get iptables to redirect only the connections from the DEV Env containers. We can do this by adding a rule to accept all connections from the Reverse Proxy. So the IP table rules will now become:

-A PREROUTING -i docker0 -s 172.17.0.2/32 -j ACCEPT
-A PREROUTING -i docker0 -s 172.17.0.1/32 -j ACCEPT
-A PREROUTING -i docker0 -p tcp -d 0/0 -j REDIRECT --to-port 3128

Since docker dynamically allocates IPs. The IPs used will need to be updated if the docker containers are rerun or the server is restarted. I also added the rule for 172.17.0.1 which is the docker0 ip.

These rules mean that all other traffic originating from the docker0 interface, other than the reverse proxy container and docker host itself, get redirected to squid.

Within squid we can whitelist domain as we like by using the following lines

acl allowed_domain dstdomain google.com
http_access allow allowed_domain

full iptables rules are:

# Generated by iptables-save v1.4.21 on Fri Nov  6 18:54:09 2015
*nat
:PREROUTING ACCEPT [30:1796]
:INPUT ACCEPT [28:1680]
:OUTPUT ACCEPT [37:2388]
:POSTROUTING ACCEPT [46:2964]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A PREROUTING -i docker0 -s 172.17.0.2/32 -j ACCEPT
-A PREROUTING -i docker0 -s 172.17.0.1/32 -j ACCEPT
-A PREROUTING -i docker0 -p tcp -d 0/0 -j REDIRECT --to-port 3128
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 8000 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 9000 -j DNAT --to-destination 172.17.0.3:8000
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 8192 -j DNAT --to-destination 172.17.0.3:80
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.17.0.2:80
COMMIT
# Completed on Fri Nov  6 18:54:09 2015
# Generated by iptables-save v1.4.21 on Fri Nov  6 18:54:09 2015
*filter
:INPUT ACCEPT [1891:3910112]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1500:1500230]
:DOCKER - [0:0]
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8000 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
COMMIT
# Completed on Fri Nov  6 18:54:09 2015

full squid rules are:

acl SSL_ports port 443
acl Safe_ports port 80      # http
acl Safe_ports port 21      # ftp
acl Safe_ports port 443     # https
acl Safe_ports port 70      # gopher
acl Safe_ports port 210     # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280     # http-mgmt
acl Safe_ports port 488     # gss-http
acl Safe_ports port 591     # filemaker
acl Safe_ports port 777     # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access allow localhost manager
http_access deny manager
acl allowed_domain dstdomain google.com
http_access allow allowed_domain
http_access allow localhost
http_access deny all
http_port 3128 accel vhost allow-direct
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:       1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .       0   20% 4320

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.