6

We use to have a filtering/proxy setup that was configured so that about 500 users were routed through a Unix-based system running SQUID with SquidGuard to block questionable content and log what users were doing. What we found was that there were some people able to bypass the block by going to free online proxies that use the HTTPS protocol. I couldn't find a way to block HTTPS traffic without disabling it altogether, which wasn't going to work for purchasing and HR (or regular users with banking sites).

Is there a way to block or filter HTTPS traffic with SQUID/Squidguard? Or some other open source monitoring programs? How do others deal with it without resorting to commercial appliances?

EDIT: We're not trying to watch the traffic. We're trying to monitor the urls and block them as necessary. We don't want credit card numbers and such...we wanted to know when Johnny was accessing https://schoolssucksoweproxyforyou.com/playboy.com, add the domain to a blockfile, and prevent them from doing it again. See some added comments below...

EDIT2: (Re: speaking of HR, banking, etc...) You may not think about it that way, but we have a person that does purchasing, our district covers 7 buildings scattered around the county so we have someone in charge of distributing materials and inventory, we have the same people working with checking and banking records, we have people in charge of unions for staff and another for faculty, we have people in charge of dealing with HR information like insurance and injury claims and liability...there's a lot of things involved in running schools that I don't think the general public ever thinks about. Plus we do have issues where it was considered overbearing to prevent staff/faculty from doing online banking, or even students from being able to access certain sites for online use (such as web quests) that sometimes required SSL connections to work, so banning HTTPS outright at the router would not be feasible.

Anyway, moving on to your suggestion about blocking only certain people, I didn't see a way to integrate the functionality in for authentication on a per-user basis; if credentials could be passed along by Windows so users didn't need to keep authenticating with yet another password it would be a workable solution but anything I found was kludgey and didn't work reliably. Then it's an issue of getting people to remember yet another password (or students/staff stealing/sharing passwords). I personally liked having filtering done fairly across the board for staff and students rather than making it an issue of teachers can do X while students aren't considered enough of a person to do that same thing.

In the end I couldn't find a way to get the server to reliably authenticate against Active Directory (to centralize user management and reduce their passwords to remember), or if I used a different authentication scheme it would mean another database to keep in sync with...what, last count, ~1200 users or so? with a high churn as every year we have kids in and out of district, graduating, and new ones coming in?...

7 Answers 7

1

I'm not going to get into ethics here - maybe you want to do this on all sites you don't "know" the domain for... anyway, ethical issues aside:

It is possible, definitely need to add bits for squid2, but AFAIK squid3 will do this out of the box. As will some commercial web filter vendors. MITM style attack is generally the only way.

7
  • We're a school, so it's mandated that users are filtered by the Government (U.S.A.), and students were hitting SSL proxies to surf porn and game sites that were questionable and we'd be liable for it. Jul 29, 2009 at 14:00
  • Bart, I am slightly biased, I work for a commercial web content filtering outfit - Smoothwall (visit www.smoothwall.com, Alex Nemeth is the guy to speak to if you want to try it out). Anyway - despite my bias, I do think it is quite a big ask to "self build" an HTTPS filter at this stage, unless you are willing to wait for squid3 to stabilise - that will be much easier. I know CIPA is a tough cookie - we have BECTA here in the UK which is similarly prescriptive.
    – Tom Newton
    Jul 29, 2009 at 15:04
  • Bart, fwiw, you can do domain blocking without resorting to MITM, and you can also block the vast majority of SSL proxies using other techniques (certificate validation for example) which may allay the ethical fears of some of the other posters!
    – Tom Newton
    Jul 29, 2009 at 15:06
  • The trouble we had at the time with domain blocking was that the domains weren't being seen with the redirected traffic; the routing in the server just wasn't seeing that traffic at all. Even redirecting the appropriate port traffic instead of just 80, so it couldn't match the domain name. Maybe an implementation issue? @Tom Newton-I've used Smoothwall before for a home project. It's a nice product. Jul 29, 2009 at 15:40
  • Ah... are you trying to do this transparently? This will make life a whole lot harder. I am not sure of any product does this yet... we hope to in the next release, but we have not worked out how yet. Would suggest you block 443 at the gateway and have people who want to use SSL set up a non-transparent proxy. Or go fully non-transparent and use proxy.pac. Dang, using my best answers in comments than no-one gets to vote for ;) Also try asking around on edugeek.net
    – Tom Newton
    Jul 29, 2009 at 16:05
7

The entire point of HTTPS traffic is that it's encrypted between the server and the end-user so no one else can snoop on it - including your filters. You won't be able to do any content filtering on it. The only HTTPS filtering you'll be able to do is blocking the SSL port to specific IP addresses.

If you whitelist, you'll have loads of false positives - banks you didn't think of, useful sites that require HTTPS to login or access, etc. If you blacklist, you'll have loads of false negatives - new proxy sites pop up every second.

This is something that needs to be addressed at a policy level, not a technical one. If someone's goofing off on porn sites at work and using proxies to get around your filters, HR should be smacking them on the hand and threatening termination if it continues.

5
  • political part is right, technical - not necessarly right. there are products like finjan that decrypt traffic, and send it to end-user encrypted again. this goes together with installation of finjan's ssl cert on all desktops - in this way online activity of end users - even if done via https [ think banking ] can be monitored.
    – pQd
    Jul 29, 2009 at 13:15
  • That it can be done doesn't mean it should.
    – ceejayoz
    Jul 29, 2009 at 13:50
  • 1
    While I appreciate people's political views, we're mandated by the government to filter traffic and block questionable content. Take it up with your state reps if you're a US citizen... Otherwise, we still were looking for a way to block URLS, not play big brother over our users finances and secret webmails. Jul 29, 2009 at 14:07
  • Last, we do have it at a policy level. It's difficult to explain the concept to 13 year olds that school computers aren't their personal property to do with as they please. But thank you for the information you provided. Jul 29, 2009 at 14:16
  • 2
    When you mention HR, purchasing, and online-banking, one generally doesn't assume it's a school being talked about. Why not block HTTPS entirely for student accounts, leave it open for employees.
    – ceejayoz
    Jul 29, 2009 at 15:53
5

Here's a very ugly solution that's implemented by a commercial vendor:

  • Replace the browsers' CA certs with your own in-house one

  • When a connection is requested to an unknown address, the proxy connects with its own client, and fetches the sites' cert

  • Then generate a fake cert for that site, signed by your own CA

  • The proxy then effectively acts as a MITM (man-in-the-middle)

You can't do that with stock Squid, but it would take me about a day of mod_perl hacking to implement that with Apache.

3

What solution might be to do what IRC servers does sometimes, if they see a connection from XXX.XXX.XXX.XXX they will try to connect to that IP and see if its a open proxy server and if it is they block that IP.

Thats the closest thing i can think of that would be a fully automatic solution, but would require work on your end. That combined with the other suggetions regarding white listing or just manualy look at logs to see and then check the remote ip of the https and see if its hosting a site might be the only soltion

2

You must NOT monitor the content of HTTPS traffic even if you can, using man-in-the-middle techniques. It will weaken the security of the entire thing and just destroy the point of using HTTPS in the first place. You can, however, monitor the HTTPS traffic externally.

For example, you can know what sites they are connecting to because all HTTPS traffic through a web proxy uses the CONNECT method to initiate, which is the initial bit transmitted in the clear. This information can be used to help decide whether or not to allow a connection to continue.

You can also monitor the bandwidth usage of the HTTPS traffic. This can give you insight into whether they are using it to stream videos or they are using it for online transactions such as banking.

5
  • We don't want to monitor their traffic. We want to block sites by URL. In SquidGuard, I'd just add the name to a textfile and run a quick "recompilation" command to reload the list into memory and then it's blocked. By Monitoring I mean I wanted to know what sites people were visiting when, as we had to run reports as principals requested it. Jul 29, 2009 at 14:02
  • Second note: we couldn't see the sites that went through HTTPS. SquidGuard/Squid looked blind when users went through to those sites. Jul 29, 2009 at 14:02
  • I disagree that a company must not monitor the content of HTTPS traffic flowing on its networks. While such a policy might breed distrust in the workforce, I disagree that well publicized monitoring of trafific destroys the point of using HTTPS in the first plcae
    – pcapademic
    Jul 29, 2009 at 15:36
  • @bart: If you just want to block by URL, look at the CONNECT method. It is in clear text.
    – sybreon
    Jul 30, 2009 at 0:55
  • 1
    @EricJLN: The point of HTTPS is to prevent others from snooping. Otherwise, just do plain HTTP. So, if a company monitors the content, it's defeating the whole point of HTTPS. The employees' banking details are all unsecured now.
    – sybreon
    Jul 30, 2009 at 0:57
1

You may need a http ssl proxy like DeleGate as a Man-In-The-Middle proxy

EDIT: Add a new way probably work

Maybe you can try pass all traffic at the ssl port, then by the mean time start a process to detect if the target server is a ssl proxy or not(this can be detected if no authentication required), then block it.

Most content based firewall use such strategy, first passed it, then check and block it.

4
  • 1
    By doing that at the organisation level, you are also needlessly exposing things like credit card information, if they are doing banking for example. Not a good idea.
    – sybreon
    Jul 29, 2009 at 13:41
  • 1
    @sybreon: it's bad but it does what is asked. There is no other way.
    – niXar
    Jul 29, 2009 at 13:51
  • It depends on what you want to achieve. I've mentioned a couple of other ways in my response.
    – sybreon
    Jul 30, 2009 at 0:54
  • @sybreon, for your "CONNECT" way, if you can not monitor the traffic, how could you know it sent CONNECT request.
    – Sam Liao
    Jul 30, 2009 at 2:07
1

I know this question is old, but I believe that you should use a DNS proxy. You can therefore blacklist all domains you don't want (be it SSL or not).

dnsmasq works well, for example. You can configure it as a transparent DNS proxy. There's an open source Linux distribution called "Endian Firewall". You can use it for that, and it's EXTREMELY easy to set up.

What you surely won't be able to do without MITM is having reports of HTTPS sites (how much they were visited, how often, by whom, etc.).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .