2

For couple of days I'm trying to figure out how to get a transparent HTTPs proxy to work with Squid. What I'm trying to achieve is a proxy that accepts internet traffic from ports 80 & 443, routes them through Squid to Privoxy and finally through Tor and returns back the data. So essentially I want to "automatically" revert some traffic through Tor without the user needing to add a proxy to their connection.

I know how to setup the Privoxy and Tor part, but I'm struggling with the Squid & IP tables configuration.

Here is my setup

Download latest version

curl -O http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.22.tar.gz && tar zxvf squid-3.5.22.tar.gz && cd squid-3.5.22

Install all needed packages

apt install devscripts build-essential openssl libssl-dev fakeroot libcppunit-dev libsasl2-dev cdbs ccze libfile-readbackwards-perl libcap2 libcap-dev libcap2-dev libnetfilter-conntrack-dev htop ccze sysv-rc-conf -y

Configure the build and make and install

./configure \
CHOST="x86_64-pc-linux-gnu" \
CFLAGS="-march=core2 -O2 -pipe" \
CXXFLAGS="${CFLAGS}" \
--build=x86_64-linux-gnu \
--prefix=/usr \
--exec-prefix=/usr \
--bindir=/usr/bin \
--sbindir=/usr/sbin \
--libdir=/usr/lib \
--sharedstatedir=/usr/com \
--includedir=/usr/include \
--localstatedir=/var \
--libexecdir=/usr/lib/squid \
--srcdir=. \
--datadir=/usr/share/squid \
--sysconfdir=/etc/squid \
--infodir=/usr/share/info \
--mandir=/usr/share/man \
--x-includes=/usr/include \
--x-libraries=/usr/lib \
--with-default-user=proxy \
--with-logdir=/var/log/squid \
--with-pidfile=/var/run/squid.pid \
--enable-err-languages=English \
--enable-default-err-language=English \
--enable-storeio=ufs,aufs,diskd \
--enable-linux-netfilter \
--enable-removal-policies=lru,heap \
--enable-gnuregex \
--enable-follow-x-forwarded-for \
--enable-x-accelerator-vary \
--enable-zph-qos \
--enable-delay-pools \
--enable-snmp \
--enable-underscores \
--with-openssl \
--enable-ssl-crtd \
--enable-http-violations \
--enable-async-io=24 \
--enable-storeid-rewrite-helpers \
--with-large-files \
--with-libcap \
--with-netfilter-conntrack \
--with-included-ltdl \
--with-maxfd=65536 \
--with-filedescriptors=65536 \
--with-pthreads \
--without-gnutls \
--without-mit-krb5 \
--without-heimdal-krb5 \
--without-gnugss \
--disable-icap-client \
--disable-wccp \
--disable-wccpv2 \
--disable-dependency-tracking \
--disable-auth --disable-epoll \
--disable-ident-lookups \
--disable-icmp

Allow ip4 forwarding

echo -e "net.ipv4.ip_forward = 1\nnet.ipv4.conf.default.rp_filter = 0\nnet.ipv4.conf.all.rp_filter = 0\nnet.ipv4.conf.eth0.rp_filter = 0\n" >> /etc/sysctl.conf

Generate certificates

mkdir /etc/squid/ssl_certs && cd /etc/squid/ssl_certs
openssl genrsa -out squid.key 2048
openssl req -new -key squid.key -out squid.csr -nodes
openssl x509 -req -days 3652 -in squid.csr -signkey squid.key -out squid.crt
cat squid.crt squid.key > squid.pem

Generate certificate cache

mkdir /var/lib/squid && chown -R proxy:proxy /var/lib/squid/
/usr/lib/squid/ssl_crtd -c -s /var/lib/squid/ssl_db

Change ownership and rights to folders

mkdir -p /var/spool/squid

chown -R proxy:proxy /etc/squid/squid.conf | chown -R proxy:proxy /usr/lib/squid | chown -R proxy:proxy /var/lib/squid/ssl_db/ | chown -R proxy:proxy /var/spool/squid | chown -R proxy:proxy /var/log/squid  | chmod 777 /var/spool/squid | chmod 777 /var/log/squid  | chmod 755 /var/lib/squid/ssl_db/certs | chown proxy:proxy /var/log/squid/

Change configuration (bellow) and initialize the cache

squid -f /etc/squid/squid.conf -z

Redirect ports 80 and 443

iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 3128
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-ports 3129

My actual squid configuration

acl localnet src all

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

never_direct allow all
always_direct allow all

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

http_access allow localnet
http_access allow localhost

debug_options ALL,2

visible_hostname squid

# stop squid taking forever to restart.
shutdown_lifetime 3
# for clients with a configured proxy.
http_port 3127
# for clients who are sent here via iptables ... REDIRECT.
http_port 3128 tproxy
# for https clients who are sent here via iptables ... REDIRECT
https_port 3129 tproxy ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.pem

sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB sslcrtd_children 8 startup=1 idle=1

# acl step1 at_step SslBump1
# ssl_bump peek step1
# ssl_bump bump all

ssl_bump server-first all
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER

via off
forwarded_for off

request_header_access From deny all
request_header_access Server deny all
request_header_access WWW-Authenticate deny all
request_header_access Link deny all
request_header_access Cache-Control deny all
request_header_access Proxy-Connection deny all
request_header_access X-Cache deny all
request_header_access X-Cache-Lookup deny all
request_header_access Via deny all
request_header_access X-Forwarded-For deny all
request_header_access Pragma deny all
request_header_access Keep-Alive deny all

cache_dir ufs /var/spool/squid 1024 16 256
coredump_dir /var/cache/squid

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320

You can notice how benevolent I'm with the settings for Squid. It's only for testing.

So where I got now is that nor intercept nor tproxy works. If I use accel for the non-HTTPS traffic it works, but nothing else. If I use it as it is, the result is that it will end up hanging for the client's timeout period and then timeout.

Here is an example. I changed in /etc/hosts the IP for httpbin.org and redirected it through the squid box.

❯ curl -vk https://httpbin.org/ip
*   Trying *******...
* Connected to httpbin.org (*******) port 443 (#0)
* TLS 1.2 connection using TLS_RSA_WITH_AES_256_GCM_SHA384
* Server certificate: ******
* Server certificate: Universe
> GET /ip HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.49.1
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable
< Server: squid/3.5.22
< Mime-Version: 1.0
< Date: Mon, 05 Dec 2016 05:43:50 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3498
< X-Squid-Error: ERR_CONNECT_FAIL 110
< Vary: Accept-Language
< Content-Language: en
< X-Cache: MISS from pipik
< Connection: close

On the squid side

2016/12/05 05:42:50.362 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New connection on FD 28
2016/12/05 05:42:50.362 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection on local=[::]:3129 remote=[::] FD 28 flags=25
2016/12/05 05:42:50.363 kid1| 33,2| client_side.cc(3911) httpsSslBumpAccessCheckDone: sslBump needed for local=*******:3129 remote=############# FD 11 flags=17 method 3
2016/12/05 05:42:50.363 kid1| 11,2| client_side.cc(2347) parseHttpRequest: HTTP Client local=*******:3129 remote=############# FD 11 flags=17
2016/12/05 05:42:50.363 kid1| 11,2| client_side.cc(2348) parseHttpRequest: HTTP Client REQUEST:
---------
CONNECT *******:3129 HTTP/1.1
Host: *******:3129


----------
2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(744) clientAccessCheckDone: The request CONNECT *******:3129 is ALLOWED; last ACL checked: localnet
2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(720) clientAccessCheck2: No adapted_http_access configuration. default: ALLOW
2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(744) clientAccessCheckDone: The request CONNECT *******:3129 is ALLOWED; last ACL checked: localnet
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 17,2| FwdState.cc(133) FwdState: Forwarding client request local=*******:3129 remote=############# FD 11 flags=17, url=*******:3129
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for '*******:3129'
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths:   always_direct = ALLOWED
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths:    never_direct = DUNNO
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(288) peerSelectDnsPaths:    ORIGINAL_DST = local=############# remote=*******:3129 flags=25
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths:        timedout = 0
2016/12/05 05:43:50.645 kid1| 4,2| errorpage.cc(1261) BuildContent: No existing error page language negotiated for ERR_CONNECT_FAIL. Using default error file.
2016/12/05 05:43:50.645 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable
2016/12/05 05:43:50.645 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable
2016/12/05 05:43:50.845 kid1| 83,2| client_side.cc(3811) clientNegotiateSSL: clientNegotiateSSL: New session 0x29dda60 on FD 11 (#############:59117)
2016/12/05 05:43:50.943 kid1| 11,2| client_side.cc(2347) parseHttpRequest: HTTP Client local=*******:3129 remote=############# FD 11 flags=17
2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(2348) parseHttpRequest: HTTP Client REQUEST:
---------
GET /ip HTTP/1.1
Host: httpbin.org
User-Agent: curl/7.49.1
Accept: */*


----------
2016/12/05 05:43:50.944 kid1| 33,2| QosConfig.cc(145) doTosLocalMiss: QOS: Preserving TOS on miss, TOS=0
2016/12/05 05:43:50.944 kid1| 33,2| client_side_reply.cc(1534) buildReplyHeader: clientBuildReplyHeader: Connection Keep-Alive not requested by admin or client
2016/12/05 05:43:50.944 kid1| 88,2| client_side_reply.cc(2051) processReplyAccessResult: The reply for GET https://httpbin.org/ip is ALLOWED, because it matched (access_log daemon:/var/log/squid/access.log line)
2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(1393) sendStartOfMessage: HTTP Client local=*******:3129 remote=############# FD 11 flags=17
2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(1394) sendStartOfMessage: HTTP Client REPLY:
---------
HTTP/1.1 503 Service Unavailable
Server: squid/3.5.22
Mime-Version: 1.0
Date: Mon, 05 Dec 2016 05:43:50 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3498
X-Squid-Error: ERR_CONNECT_FAIL 110
Vary: Accept-Language
Content-Language: en
X-Cache: MISS from squid
Connection: close


----------
2016/12/05 05:43:50.944 kid1| 33,2| client_side.cc(817) swanSong: local=*******:3129 remote=############# flags=17
2016/12/05 05:43:50.944 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable
2016/12/05 05:43:50.944 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable

I tried so many different configurations that I'm already lost in what does work and what doesn't. I'm probably not understanding the connection between iptables and squid properly, but no matter what I read I always end up here.

I appreciate any suggestions.

10
  • try intercept instead of tproxy in squid.conf since you are using iptables redirect and see if it works. – Diamond Dec 5 '16 at 9:50
  • As I've mentioned above, nor intercept nor trproxy works. For intercept the result is an immediate 403 from the Squid. – Fomo Dong Dec 5 '16 at 15:26
  • Your posted configs are inconsistent, thats what I tried to point out. But Well, what do you mean by "I changed in /etc/hosts the IP for httpbin.org and redirected it through the squid box." Where are you doing this, in proxy server or client? Why would you need this to check ssl proxy and how should it work? – Diamond Dec 5 '16 at 15:51
  • @bangal inconsistent in what way? I mentioned I've tried both tproxy and intercept but it didn't work. The config is only for tproxy though. What I'm trying to is on a client to route the traffic by adding specific hosts to /etc/hosts and route the traffic through the proxy box. All happening on the internet, no local networks involved. – Fomo Dong Dec 5 '16 at 16:57
  • with all due respect, I must tell you, you have no idea what you are doing. At least from what you are saying it makes no sense. It sounds more like a reverse proxy that you are trying to set up (wiki.squid-cache.org/ConfigExamples/Reverse/…). But it is for websites that you own and have control of. The configs you have is for traditional forward/caching proxy. And it wont work if you just change the ip of a public website to your proxy server. The client will try to get the website from the proxy and the proxy will return nothing. – Diamond Dec 6 '16 at 8:13
0

All you need is install the .crt certificate in your clients, if you are unsing a selfsigned certificate. I take example in this question to compile squid 5 transparent mode with a self-signed certificate transparent in tls mode and is working fine. only google use a diferent certificate, in linux good, in windows good, in Android fifty-fifty, but you can browser in all of them.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.