110

PLEASE NOTE: I'm not interested in making this into a flame war! I understand that many people have strongly-held beliefs about this subject, in no small part because they've put a lot of effort into their firewalling solutions, and also because they've been indoctrinated to believe in their necessity.

However, I'm looking for answers from people who are experts in security. I believe that this is an important question, and the answer will benefit more than just myself and the company I work for. I've been running our server network for several years without a compromise, without any firewalling at all. None of the security compromises that we have had could have been prevented with a firewall.

I guess I've been working here too long, because when I say "servers", I always mean "services offered to the public", not "secret internal billing databases". As such, any rules we would have in any firewalls would have to allow access to the whole Internet. Also, our public-access servers are all in a dedicated datacenter separate from our office.

Someone else asked a similar question, and my answer was voted into negative numbers. This leads me to believe that either the people voting it down didn't really understand my answer, or I don't understand security enough to be doing what I'm currently doing.

This is my approach to server security:

  1. Follow my operating system's security guidelines before connecting my server to the Internet.

  2. Use TCP wrappers to restrict access to SSH (and other management services) to a small number of IP addresses.

  3. Monitor the state of this server with Munin. And fix the egregious security problems inherent to Munin-node in its default configuration.

  4. Nmap my new server (also before connecting my server to the Internet). If I were to firewall this server, this should be the exact set of ports incoming connections should be restricted to.

  5. Install the server in the server room and give it a public IP address.

  6. Keep the system secure by using my operating system's security updates feature.

My philosophy (and the basis of the question) is that strong host-based security removes the necessity of a firewall. The overall security philosophy says that strong host-based security is still required even if you have a firewall (see security guidelines). The reason for this is that a firewall that forwards public services to a server enables an attacker just as much as no firewall at all. It is the service itself that is vulnerable, and since offering that service to the entire Internet is a requirement of its operation, restricting access to it is not the point.

If there are ports available on the server that do not need to be accessed by the whole Internet, then that software needed to be shut down in step 1, and was verified by step 4. Should an attacker successfully break into the server through vulnerable software and open a port themselves, the attacker can (and do) just as easily defeat any firewall by making an outbound connection on a random port instead. The point of security isn't to defend yourself after a successful attack - that's already proven to be impossible - it's to keep the attackers out in the first place.

It's been suggested that there are other security considerations besides open ports - but to me that just sounds like defending one's faith. Any operating system/TCP stack vulnerabilities should be equally vulnerable whether or not a firewall exists - based on the fact that ports are being forwarded directly to that operating system/TCP stack. Likewise, running your firewall on the server itself as opposed to having it on the router (or worse, in both places) seems to be adding unnecessary layers of complexity. I understand the philosophy "security comes in layers", but there comes a point where it's like building a roof by stacking X number of layers of plywood on top of each other and then drilling a hole through all of them. Another layer of plywood isn't going to stop the leaks through that hole you're making on purpose.

To be honest, the only way I see a firewall being any use for servers is if it has dynamic rules preventing all connections to all servers from known attackers - like the RBLs for spam (which coincidentally, is pretty much what our mail server does). Unfortunately, I can't find any firewalls that do that. The next best thing is an IDS server, but that assumes that the attacker doesn't attack your real servers first, and that attackers bother to probe your entire network before attacking. Besides, these have been known to produce large numbers of false positives.

8
  • 2
    And so ALL of the traffic that passes between your servers is encrypted?
    – GregD
    Nov 12, 2010 at 22:21
  • 6
    Love it. Local firewall rules are almost always voodoo-only.
    – unixtippse
    Nov 13, 2010 at 4:48
  • 2
    Do you have desktops/employees in your network as well? What do you do with them?
    – Brandon
    Nov 13, 2010 at 16:10
  • 8
    This question would have been well suited for security.stackexchange.com Nov 14, 2010 at 6:56
  • 2
    @routeNpingme: It looks like I didn't include that tidbit in my original post. All of our servers need to be open to the public, and live in a dedicated datacenter. If your office is your datacenter, I suppose it would be necessary to have a firewall between your server network and your office network. In which case, I'm talking about the server network - why firewall something that has complete public access?
    – Ernie
    Nov 15, 2010 at 17:57

20 Answers 20

54

Advantages of firewall:

  1. You can filter outbound traffic.
  2. Layer 7 firewalls (IPS) can protect against known application vulnerabilities.
  3. You can block a certain IP address range and/or port centrally rather than trying to ensure that there is no service listening on that port on each individual machine or denying access using TCP Wrappers.
  4. Firewalls can help if you have to deal with less security aware users/administrators as they would provide second line of defence. Without them one has to be absolutely sure that hosts are secure, which requires good security understanding from all administrators.
  5. Firewall logs would provide central logs and help in detecting vertical scans. Firewall logs can help in determining whether some user/client is trying to connect to same port of all your servers periodically. To do this without a firewall one would have to combine logs from various servers/hosts to get a centralized view.
  6. Firewalls also come with anti-spam / anti-virus modules which also add to protection.
  7. OS independent security. Based on host OS, different techniques / methods are required to make the host secure. For example, TCP Wrappers may not be available on Windows machines.

Above all this if you do not have firewall and system is compromised then how would you detect it? Trying to run some command 'ps', 'netstat', etc. on local system can't be trusted as those binaries can be replaced. 'nmap' from a remote system is not guaranteed protection as an attacker can ensure that root-kit accepts connections only from selected source IP address(es) at selected times.

Hardware firewalls help in such scenarios as it is extremely difficult to change firewall OS/files as compared to host OS/files.

Disadvantages of firewall:

  1. People feel that firewall will take care of security and do not update systems regularly and stop unwanted services.
  2. They cost. Sometimes yearly license fee needs to be paid. Especially if the firewall has anti-virus and anti-spam modules.
  3. Additional single point of failure. If all traffic passes through a firewall and the firewall fails then network would stop. We can have redundant firewalls, but then previous point on cost gets further amplified.
  4. Stateful tracking provides no value on public-facing systems that accept all incoming connections.
  5. Stateful firewalls are a massive bottleneck during a DDoS attack and are often the first thing to fail, because they attempt to hold state and inspect all incoming connections.
  6. Firewalls cannot see inside encrypted traffic. Since all traffic should be encrypted end-to-end, most firewalls add little value in front of public servers. Some next-generation firewalls can be given private keys to terminate TLS and see inside the traffic, however this increases the firewall's susceptibility to DDoS even more, and breaks the end-to-end security model of TLS.
  7. Operating systems and applications are patched against vulnerabilities much more quickly than firewalls. Firewall vendors often sit on known issues for years without patching, and patching a firewall cluster typically requires downtime for many services and outbound connections.
  8. Firewalls are far from perfect, and many are notoriously buggy. Firewalls are just software running on some form of operating system, perhaps with an extra ASIC or FPGA in addition to a (usually slow) CPU. Firewalls have bugs, but they seem to provide few tools to address them. Therefore firewalls add complexity and an additional source of hard-to-diagnose errors to an application stack.
6
  • 6
    Above all this if you do not have firewall and system is compromised then how would you detect it? Intrusion detection is not the job of the firewall. That job is more properly handled by the HIDS (host-based intrusion detection system), which is independent of the firewall. Nov 13, 2010 at 17:54
  • 6
    Syslog servers eliminate the need for item 5. If anything, it's best to send your firewall logs to a syslog server, in case an attacker manages to compromise the firewall and delete its logs. Then the attacker has to compromise two systems just to delete the logs, and they might not be prepared for that (especially with automated attacks). Likewise, if all your systems have centralized logging, you get better detail about the attack than firewall logs can provide.
    – Ernie
    Nov 15, 2010 at 17:19
  • 2
    My point was since HIDS resides on host we cant trust its output. For example even if we use cryptographically secure 'tripwire' as host based IDS, the attacker can always replace all tripwire binaries (twadmin, tripwire, twprint, etc.) with compromised versions which will never report intrusion. Even if we try to copy libraries/binaries from other system, there can be a process running which monitors these compromised binaries and again replaces them with corrupted version in case they are replaced or updated. Firewall being independent from host, can be trusted in such scenarios. Nov 17, 2010 at 3:04
  • 2
    This answer was accepted over the more popular one because it provides better and a more comprehensive set of reasons for using a firewall. And not, for that matter.
    – Ernie
    Jan 17, 2011 at 17:46
  • Stateful packet inspection firewalls do NOT belong in front of servers. They are a huge liability in DDoS attacks, and are typically the first thing to fail under attack.
    – rmalayter
    Aug 9, 2015 at 1:16
32

TCP Wrappers could be arguably called a host-based firewall implementation; you're filtering network traffic.

For the point on an attacker making outbound connections on an arbitrary port, a firewall would provide a means of controlling outgoing traffic as well; a properly configured firewall manages ingress and egress in a way which is appropriate to the risk related to the system.

On the point about how any TCP vulnerability isn't mitigated by a firewall, you're nt familiar with how firewalls work. Cisco has a whole bunch of rules available for download which identify packets constructed in a way that would cause particular operating systems problems. If you grab Snort and start running it with the right rule set, you will also get alerted on this kind of thing. And of course, Linux iptables can filter out malicious packets.

Basically, a firewall is proactive protection. The farther you get away from being proactive, the most likely that you'll find yourself in a situation where you're reacting to a problem rather than preventing the problem. Concentrating your protection at the border, as with a dedicated firewall, makes things easier to manage because you have a central choke point rather than duplicating rules everywhere.

But no single thing is necessarily a final solution. A good security solution generally is multi-layer, where you have a firewall at the border, TCP wrappers at the device, and probably some rules on internal routers as well. You should usually protect the network from the Internet, and protect the nodes from each other. This multi-layer approach isn't like drilling a hole through multiple sheets of plywood, it's more like putting up a pair of doors so an intruder has two locks to break instead of just one; this is called a man trap in physical security, and most every building has one for a reason. :)

4
  • 6
    Also if they sneak inside the building and open the inner door for their friend outside, they then have to also unlock and open the outer door. (i.e. without an external firewall, someone who gets into your server can open it right up, whereas an external firewall would still block the open ports from the outside)
    – Ricket
    Nov 13, 2010 at 15:23
  • @Ricket: Perhaps they could, but modern attackers don't bother with things like this. Your site would have to be of particular interest for an attacker to do anything more than add your server to a zombie farm.
    – Ernie
    Nov 15, 2010 at 17:14
  • 1
    @Ernie - no, it only needs to exist to be automatically probed for free space for Warez, customer databases, financial info, passwords, and being added to a botnet - but even that can be bad enough - some admins will happily blackhole your IP if it looks like you host zombies.
    – Rory Alsop
    Dec 10, 2010 at 21:17
  • TCP Wrappers could be arguably called a host-based firewall implementation +1 for a great answer.
    – sjas
    Aug 8, 2015 at 22:53
14

(You may want to read "Life without Firewalls")

Now: What about having a legacy system for which no patches get published anymore? What about not being able to apply the patches to N-machines at the time you need to do so, while at the same time you can apply them in fewer nodes in the network (firewalls)?

There's no point in debating the firewall's existence or need. What really matters is that you have to implement a security policy. To do so you will use whatever tools will implement it and help you manage, expand and evolve it. If firewalls are needed to do so, that's fine. If they are not needed that's fine too. What really matters is having a working and verifiable implementation of your security policy.

4
  • 1
    Heh. I've been running our server network for the past 8 years without a firewall. I could have written "Life without firewalls", but he did a way better job and runs a way larger network than I, anyway.
    – Ernie
    Nov 12, 2010 at 22:01
  • 2
    @Ernie - I guess you could just have got lucky. How do you know you haven't been compromised? The results of forensic investigations on many of my clients have discovered compromise, sometimes dating back months, while the attackers found personal and financial information, client details, intellectual property, business plans etc. As Sean said - get a proper security audit done.
    – Rory Alsop
    Dec 10, 2010 at 21:26
  • 1
    Actually, aside from the fact that our server network is physically separate from our office network (and thus, no truly sensitive data can be gleaned from it, even with root access on every server), I have been able to discover every single compromise we've ever had since I started here. I could go on about the horror show that existed when I started, but I don't have enough space. :) Suffice to say that most attacks are not subtle, and the subtle ones are discoverable too. Oh yes, and user privilege separation is your friend.
    – Ernie
    Jan 17, 2011 at 17:51
  • 1
    "life without firewalls" is snake-oil since he uses host based stateless firewalling, too: "Most services (including RPC) protected by TCP Wrappers. Some services which should only be used within our networks are limited to just those networks. TCP Wrappers also provides consistent logging of accepted and refused connections for services, even those accessible from anywhere."
    – sjas
    Jan 9, 2023 at 16:23
9

Most of your explanations seem to refute a need for a firewall, but I don't see a con to having one, other than the small amount of time to set one up.

Few things are a "necessity" in a strict meaning of the word. Security is more about setting up all the blockades you can. The more work needed to break into your server means less chance of successful attack. You want to make it more work to break into your machines than somewhere else. Adding a firewall makes more work.

I think a key use is redundancy in security. Another plus of firewalls is you can simply drop attempts to connect to any port rather than responding to rejected requests - this will make nmapping a little more inconvenient for an attacker.

Most important to me on the practical note of your question is you can lock SSH, ICMP, and other internal services down to local subnets as well as rate limit incoming connections to help alleviate DOS attacks.

"The point of security isn't to defend yourself after a successful attack - that's already proven to be impossible - it's to keep the attackers out in the first place."

I disagree. Limiting damages can be just as important. (under this ideal why hash passwords? or stick your database software on a different server than your web applications?) I think the old saying "Don't stick all of your eggs in one basket" is applicable here.

3
  • 1
    Well, you're right, I didn't put any cons in there. Cons: increased network complexity, single point of failure, single network interface through which bandwidth is bottlenecked. Likewise, administrative mistakes made on one firewall can kill your entire network. And gods forbid that you lock yourself out of it in the meantime when it's a 20 minute trip to the server room.
    – Ernie
    Nov 12, 2010 at 21:40
  • 1
    This may be purely rhetorical, but when you say "Security is more about setting up all the blockades you can", I would prefer to hear "Security is more about blocking everything by default, and carefully opening the strict minimum to operate".
    – MatthieuP
    Nov 13, 2010 at 13:59
  • +1 A comprehensive security plan covers prevention, detection and response. Nov 14, 2010 at 2:50
8

Should I firewall my server? Good question. It would seem that there is little point to slapping a firewall on top of a network stack that already rejects connection attempts to all but the handful of ports that are legitimately open. If there is a vulnerability in the OS that allows maliciously crafted packets to disrupt/exploit a host, would a firewall running on that same host prevent the exploit? Well, maybe ...

And that is probably the strongest reason to run a firewall on every host: A firewall might prevent a network stack vulnerability from being exploited. Is that a strong enough reason? I don't know, but I suppose one could say, "No one ever got fired for installing a firewall."

Another reason to run a firewall on a server is to decouple these two otherwise strongly correlated concerns:

  1. From where, and to what ports, do I accept connections on?
  2. Which services are running and listening for connections?

Without a firewall, the set of services running (along with the configurations for tcpwrappers and such) completely determines the set of ports the server will have open, and from whom connections will be accepted. A host-based firewall gives the admin additional flexibility to install and test new services in a controlled way before making them more widely available. If such flexibility is not required, then there is less reason to install a firewall on a server.

On a final note, there is one item not mentioned in your security checklist that I always add, and that is a host-based intrusion detection system (HIDS), such as AIDE or samhain. A good HIDS makes it extremely difficult for an intruder to make unwanted changes to the system and remain undetected. I believe all servers should be running some kind of HIDS.

3
  • 1
    +1 for mention of HIDS systems. Nov 13, 2010 at 4:48
  • 3
    HIDS are great - If you intend to set it and forget it. And never add or delete accounts. Otherwise the vast majority of HIDS logs are going to be long lists of what you did today, and will quickly end up being ignored all the time.
    – Ernie
    Nov 15, 2010 at 17:23
  • No pain, no gain, as they say. On servers with a lot of changes, it is possible to filter out expected noise, leaving you to concentrate on the unexpected. Nov 15, 2010 at 18:56
6

A firewall is a tool. It doesn't make things secure in and of itself, but it can make a contribution as a layer in a secure network. That doesn't mean you need one, and I certainly worry about people who blindly say "I've got to get a firewall" without understanding why they think that way and who don't understand the strengths and weaknesses of firewalls.

There are a lot of tools we can say we don't need... Is it possible to run a Windows computer without Antivirus? Yes it is... but it's a nice layer of insurance to have one.

I'd say the same about firewalls - whatever else you can say about them, they are a nice level of insurance. They are not a substitute for patching, for locking machines down, for disabling services you don't use, for logging, etc... but they can be a useful supplement.

I'd also suggest that the equation changes somewhat depending on whether or not you're talking about placing a firewall in front of a group of carefully tended servers, as you seem to be, or a typical LAN with a mix of workstations and servers, some of which might be running some pretty hairy stuff despite the best efforts and wishes of the IT team.

One more thing to consider is the the benefit of creating an obviously hardened target. Visible security, whether it's bright lights, heavy locks and an obvious alarm box on a building; or an obvious firewall on a business' range of IP addresses can deter the casual intruder - they'll go looking for easier prey. This won't deter the determined intruder who knows you have information they want and is determined to get it, but deterring the casual intruders is still worthwhile - especially if you know that any intruder whose probes persists past the deterrent needs to be taken especially seriously.

2
  • 1
    Thus the reason I said "servers" instead of "office network"? :) In our case especially, the datacenter and office are two physically separate entities.
    – Ernie
    Nov 12, 2010 at 22:38
  • I understand that Ernie, but it's a point that's worth underlining... so I did.
    – Rob Moir
    Nov 12, 2010 at 22:42
5

All great questions. BUT - I'm very surprised PERFORMANCE is not been brought to the table.

For highly (CPU-wise) used Web front ends, local firewall really degrades performance, period. Try a load test and see. I saw this tons of times. Turning off firewall increased performance (request per-sec) by 70% or more.

This trade-off must be considered.

1
  • 2
    It depends very much on the firewall rules in place. Firewall rules are sequentially run on every packet, so you don't want to have hundreds of rules that have to be looked at. Last winter when we managed a site that had an advertisement on the superbowl, the firewall rules were not a problem, for example. But I do agree that you need to understand the performance impact of firewall rules. Nov 14, 2010 at 10:23
4

A firewall is additional protection. Three particular scenarios it protects against are network stack attacks (i.e. your server OS has a vulnerability to on specially crafted packets that never reach the level of ports), a successful intrusion making a connection to "phone home" (or send spam, or whatever), or a successful intrusion opening a server port or, less detectable, watch for a port knocking sequence before opening a port. Granted, the last two have to do with mitigating the damage of an intrusion rather than preventing it, but that doesn't mean it's useless. Remember that security is not an all-or-nothing proposition; one takes a layered approach, belt and suspenders, to achieve a level of security that is sufficient for your needs.

2
  • +1 Absolutely, defence in depth is key. Nov 14, 2010 at 2:46
  • 2
    I fail to see how I can have any expectation of blocking outbound traffic to any effect, especially when our customers expect many of our servers to send mail to random hosts on the internet (as is the case with spam). "Phoning home" is just a matter of connecting to some other random host on the net - and I doubt that blocking all outbound connections save for a few will help anything at all - that is, if you want to do anything on the internet. And blocking just a few ports is kind of like setting up a toll booth in the middle of the desert.
    – Ernie
    Nov 15, 2010 at 17:35
3

I'm no security expert by any means, but it sounds as though you are firewall'ed. It seems as though you've taken some of the core functionality of a firewall and made it part of your policies and procedures. No, you don't need a firewall if you are going to do the same job as a firewall yourself. As for myself, I'd rather do the best I can in keeping up security, but have a firewall looking over my shoulder, but at some point when you can do everything the firewall is doing, it starts to become irrelevant.

2

Blockquote Well, you're right, I didn't put any cons in there. Cons: increased network complexity, single point of failure, single network interface through which bandwidth is bottlenecked. Likewise, administrative mistakes made on one firewall can kill your entire network. And gods forbid that you lock yourself out of it in the meantime when it's a 20 minute trip to the server room.

First, adding at most one additional routed hop through your network is not complex. Second, no firewall solution implemented with any single point of failure is completely useless. Just like you cluster your important server or services and use bonded NICs, you implement highly available firewalls. Not doing so, or not recognizing and knowing that you'd do so is very short sighted. Simply stating that there is a single interface does not automatically make something a bottleneck. That assertion shows that you have no idea how to properly plan and deploy a firewall sized to handle the traffic that would flow through your network. You are correct in saying that a mistake in policy can cause harm to your entire network, but I'd argue that maintaining individual policies on all your servers is far more error prone that a single place.

As for the argument that you keep up with security patching and follow security guides; that's a shaky argument at best. Typically security patches aren't available until after a vulnerability is discovered. That means that the entire time you're running publicly addressable servers they are vulnerable until they are patched. As others have pointed out, IPS systems can help prevent compromises from such vulnerabilities.

If you think your systems are as safe as they can be, that's a good confidence to have. However, I'd recommend you have a professional security audit performed on your network. It may just open your eyes.

1
  • It may just open your eyes. +1 as it will be the last nail in the coffin.
    – sjas
    Aug 8, 2015 at 22:50
2

A firewall is certainly not needed for smaller setups. If you have one or two servers, software firewalls are maintainable. With that said, we don't run without dedicated firewalls, and there are a few reasons why I maintain this philosophy:

Separation of Roles

Servers are for applications. Firewalls are for packet inspection, filtering, and policies. A web server should worry about serving web pages, and that's it. Putting both roles in one device is like asking your accountant to also be your security guard.

Software is a moving target

The software on the host is always changing. Applications can create their own firewall exceptions. The OS is updated and patched. Servers are a high-traffic "admin" area, and your firewall policies/security policies are often way more important to security than your application configurations. In a Windows environment, suppose somebody makes a mistake at some Group Policy level and turns Windows Firewall off on the desktops PCs, and doesn't realize that it's going to get applied to the servers. You're wide open in a matter of clicks.

Just speaking to updates, firewall firmware updates generally come out once or twice a year, while OS and service updates are a constant stream.

Reusable services/policies/rules, manageability

If I set up a service/policy called "Web Server" once (say TCP 80 and TCP 443), and apply it to the "web servers group" at the firewall level, that is much more efficient (a couple of configuration changes) and exponentially less prone to human error than setting up firewall services on 10 boxes, and opening up 2 ports x 10 times. When that policy needs to change, it's 1 change vs. 10.

I can still manage a firewall during an attack, or after a compromise

Say my host-based firewall + application server is getting attacked and CPU is off the chart. To even start to figure out what's happening, I am at the mercy of my load being less than the attacker's to even get in and look at it.

An actual experience - I once messed up a firewall rule (left ports to ANY instead of a specific one, and server had a vulnerable service), and the attacker actually had a live Remote Desktop session to the box. Every time I'd start to get a session, the attacker would kill off/disconnect my session. If it wasn't for being able to shut down that attack from an independent firewall device, that could have been a lot worse.

Independent Monitoring

The logging in dedicated firewall units is usually far superior to host-based software firewalls. Some are good enough that you don't even need external SNMP / NetFlow monitoring software to get an accurate picture.

IPv4 Conservation

There is no reason to have two IP addresses if one is for web and one is for mail. Keep the services on separate boxes, and route the ports appropriately via the device designed to do that.

2

In general, security is an onion thing, as already mentioned. There are reasons firewalls exist, and it's not just all the other lemmings being stupid morons.

This answer comes, as searching for 'fail2ban' on this page did not give me any results. So if I double other content, bear with me. And excuse me, if I rant a little, I provide plain experience as this might come in handy for others. :)

Networking considerations, local vs. external

This is rather Linux specific and concentrates on host based firewalling, which is usually the use case. External firewalling goes hand in hand with a proper network structure and other security considerations usually go into that. Either you know what is implied here, then you will likely not need this posting. Or you don't, then just read on.

Running firewalls externally and locally may seem counter-intuitive and double work. But this also gives the possibility turn of the rules on the external one, without compromising security on all other hosts being behind it. The need could arise from either debugging reasons or because somebody has just fucked up. Another use case will come down there in the 'adaptive global firewalling' section, for which you will also need both global and local firewalls.

Cost and availability and the same story all the time:

Firewalling is just one aspect of a proper secured system. Not installing a firewall as it 'costs' money, introduces a SPOF or whatever is just bullshit, pardon my French here. Just setup a cluster. Oh, but what if the fire cell has an outage? Then setup your cluster on spanning two or more fire compartments.

But what if the whole datacenter is unreachable, as both of the external carriers are out of business (excavator killed your fiber)? Just make your cluster spanning several datacenters, then.

That's expensive? Clusters are too complex? Well, paranoia has to be paid for.

Just whining about SPOFs, but not wanting to pay more money or creating little more complex setups is a clear case of double standards or just a small wallet on company or customer side.

This pattern applies to ALL these discussions, no matter which service is the current matter of the day. No matter if it's a VPN gateway, a Cisco ASA used just for firewalling, a MySQL or PostgreSQL database, a virtual system or server hardware, a storage backend, switches/routers, ...

By now you should get the idea.

Why bother with firewalling?

In theory your reasoning is sound. (Only running services can be exploited.)

But this is only half the truth. Firewalling, especially stateful firewalling can do much more for you. Stateless firewalls are only important if you experience performance issues like others already mentioned.

Easy, central, discrete access control

You mentioned TCP wrappers - these implement basically the same functionality for securing your environment. Because ACLs, by the way, are also just feature-incomplete host-based stateless firewalls.

For the sake of the argument, let's assume someone doesn't know of tcpd and likes using a mouse? fwbuilder might come into mind.

Having full access from your management network is something you should have enabled, which is something of the first use cases of your host based firewall.

How about a multi-server setup, where the database runs somewhere else and you cannot put both/all the machines within a shared (private) subnet for whatever reason? Use the firewall to allow MySQL access on port 3306 only for the single given IP address of the other server, done, simple.

And that also works flawlessly for UDP. Or whatever protocol. Firewalls can be so damn flexible. ;)

Portscan mitigation

Further, with firewalling, general portscans can be detected and mitigated as the amount of connections per timespan can be monitored via the kernel and its networking stack, and the firewall can act upon that.

Also invalid or obscure packets can be handled prior to them ever reaching your application.

Outbound traffic limiting

Filtering outbound traffic is usually a pain in the ass. But it can be a must, depending on contract.

Statistics

Another thing that a firewall can give you, is statistics. (Think watch -n1 -d iptables -vnxL INPUT with having added some rules for special IP addresses right at the top to see if packets are coming through.)

You can see in plain daylight if things work or they do not. Which is VERY useful when troubleshooting connections and being able to tell the other person on the telephone you do not get packets, without having to resort to chatty tcpdump's. Networking is fun, most people just do now know what they are doing, and all to often it's just simple routing errors. Hell, even I don't always know what I am doing. Although I have worked with literally dozens of complex systems and appliances, often with tunneling, too, by now.

IDS/IPS

Layer7 firewalling is usually snake-oil (IPS/IDS), if not attended properly and updated regularly. Plus the licenses are damn expensive, so I'd spare getting one if you have no real need for getting everything money can buy you.

Masqerading

Easy, just try this with your wrappers. :D

Local port forwarding

See masquerading.

Securing password access channels with dynamic IP addresses

What about if customers have dynamic IP addresses and there is not a VPN setup deployed? Or other dynamic approaches to firewalling? This is already hinted at in the question, and here will come a use case for the Unfortunately, I can't find any firewalls that do that. part.

Having disabled the root account access via a password is a must. Even if access is limited to certain IP addresses.

Also, still having another blanko account ready with a password login if ssh keys get lost or deployment fails is very handy if something goes really wrong (a user has administrative access to the machine and 'things happened'?). It is the same idea for network access as it is having single-user mode on Linux or using init=/bin/bash via grub for local access really really bad and cannot use a live disk for whatever reason. Don't laugh, there are virtualization products prohibiting that. Even if the functionality exists, what if an outdated software version is run lacking the functionality?

Anyway, even if you run your ssh daemon on some esoteric port and not on 22, if not having implemented things like port knocking (to open even another port and thus mitigating portscans, slowly bordering on being too impractical), port scans will detect your service eventually.

Usually you also set up all servers with the same configuration, with the same ports and services for efficiency reasons. You cannot set up ssh to a different port on every machine. Also you cannot change it on all machines every time you consider it being 'public' information, because it already is after a scan. The question of nmap being legal or not is not an issue when having a hacked Wi-Fi connection at your disposal.

If this account is not named 'root', people may not be able to guess the user account name of your 'backdoor'. But they will know, if they get another server from your company, or just buy some webspace, and have an unchrooted/unjailed/uncontainered look at /etc/passwd.

For purely theoretical illustration now, they could use a hackable website there to gain access to your server and look up how things are usually run at your place. Your hack search tools might not run 24/7 (usually they do at night for disk performance reasons for the filesystem scans?) and your virus scanners are not updated the second a new zero-day sees the light of the day, so it will not detect these happenings at once, and without other protection measures you may never even know what happened. To get back to reality, if someone has access to zero-day exploits, it's very likely he will not target your servers anyway as these are just expensive. This is just for illustrating that there is always a way into a system if the 'need' arises.

But on the topic again, just don't use an extra passworded account, and don't bother? Please read on.

Even if attackers to get the name and port of this extra account, a fail2ban + iptables combination will stop them short, even if you used only an eight-letter password. Plus fail2ban can be implemented for other services, too, widening the monitoring horizon!

For your own services, if the need ever arose: Basically every service logging failures to a file can get fail2ban support via providing a file, what regex to match and how many failures are allowed, and the firewall will just happily ban every IP address it is told to.

I am not telling to use 8-digit passwords! But if they get banned for 24 hours for five wrong password tries, you can guess how long they will have to try if they do not have a botnet at their disposal even with such lousy security. And you'd be astonished what passwords customers tend to use, not just for ssh. Having a look at peoples' mail passwords via Plesk tells you everything you'd rather not want to know, if you'd ever do, but what I am not trying to imply here of course. :)

Adaptive global firewalling

fail2ban is just one application which uses something along the lines of iptables -I <chain_name> 1 -s <IP> -j DROP, but you can easily build such stuff yourself with some Bash magic quite fast.

To further expand something like this, aggregate all fail2ban IP addresses from servers within your network on an extra server, that curates all the lists and passes it in turn to your core firewalls blocking all the traffic already on the edge of your network.

Such functionality cannot be sold (of course it can, but it will just be a brittle system and suck), but has to be interweaved into your infrastructure.

While at it, you can also use blacklist IP addresses or lists from other sources, be it aggregated by yourself or external ones.

1

In the physical world, people secure valuables by putting them in safes. But there is no safe that cannot be broken into. Safes, or security containers, are rated in terms of how long it takes to force entry. The purpose of the safe is to delay the attacker long enough that they are detected and active measures then stop the attack.

Similarly, the proper security assumption is that your exposed machines will, eventually, be compromised. Firewalls and bastion hosts are not set up to prevent your server (with your valuable data) from compromise, but to force an attacker to compromise them first and allow you to detect (and deter) the attack before the valuables are lost.

Note that neither firewalls nor bank vaults protect against insider threats. That's one reason for bank accountants to take two weeks leave consecutively, and for servers to have full internal security precautions even though protected by bastion hosts.

You seem to imply in the original post that you are forwarding "outside world" packets through your firewall directly to your server. In that case, yes, your firewall isn't doing very much. A better perimeter defense is done with two firewalls and a bastion host, where there is no direct logical connection from outside to inside -- every connection terminates in the DMZ bastion host; every packet is examined appropriately (and possibly parsed out) before forwarding.

5
  • "That's one reason for bank accountants to take two weeks leave consecutively" can you clarify on that one? Might not be important here, but I got interested. Nov 13, 2010 at 18:41
  • -1 because I already covered the fact that you don't have to break into a firewall before you break into a server - The firewall must allow the public to have access to a service that you are offering to the public, therefore the server itself is open to attack from the public. There are no services on any of our servers that we do not already want the public to have access to.
    – Ernie
    Nov 15, 2010 at 17:42
  • @Ernie - You miss the point. A bastion host design DMZ isolates a host by firewalls on both sides. That host is exposed to attack, and will eventually be subverted. But that host is not your server and properly will have minimal amounts of your critical information on it. The DMZ (host+firewalls) slows down an attack on your server long enough that you can respond and prevent its success.
    – mpez0
    Nov 18, 2010 at 20:56
  • 1
    @Per Wiklander - GAAP include regular check audits. An embezzling accountant may be able to cook the numbers and prevent discovery during those check audits when present, but if required to be away from the job for two consecutive weeks someone else will be reporting and their malfeasance can be discovered. It's a form of two-person control.
    – mpez0
    Nov 18, 2010 at 20:59
  • How does the bastion host protect anything on the servers? An example: Port 80, 25, and 110 are the only open ports on a server. The firewall allows traffic to these ports from the entire internet. Therefore, the firewall protects nothing. If your servers are in a separate location from your office, then you need no further protection except for a firewall at your office.
    – Ernie
    Nov 18, 2010 at 22:01
1

Vulnerabilities are hard to predict. It is practically impossible to predict which way your infrastructure is going to get exploited. Having a firewall "raises the bar" for an attacker wanting to exploit a vulnerability, and this is the important part, BEFORE you know what that vulnerability is. Additionally, the ramifications of the firewall can be easily understood in advance, so you are unlikely to CAUSE problems with a firewall which are more severe than the problems you are likely to avoid.

This is why "nobody ever got fired for installing a firewall". Their implementation is very low risk, and has a HIGH likelihood of either preventing or mitigating an exploit. Also, most really nasty vulnerabilities end up being exploited by automation, so anything "unusual" will end up breaking a bot, or at least get it to skip you in favor of an easier target.

Side note; firewalls are not for the internet only. Your accounting dept. doesn't need ssh/whatever to your ldap server, so don't give it to them. Compartmentalizing internal services can make a big difference to the cleanup job in the event that something DOES breach the castle walls.

2
  • 2
    Having a firewall doesn't raise the bar when you need to have firewall rules opening ports 80, 53, 25, 110, 143, 443, 993, 995 and more to the entire internet. Those servers are just as vulnerable with a firewall as without, if you need to have rules like that.
    – Ernie
    Nov 15, 2010 at 18:07
  • 2
    True, but that same server probably has port 3306 (mysql) and a variety of other protocols which would potentially benefit from a firewall. That is not to say that you should have no other protection; just that the firewall isn't going to hurt. Also, I think you missed my point that all traffic doesn't need to be open for all users. Port 22 may need to be open, but not on ALL your hosts, and certainly not to the entire internet (or accounting and HR either). Closing 25 for accounting if they are supposed to operate over 465 may mitigate some spam malware for example.
    – Enki
    Jan 19, 2011 at 5:53
1

Have to say Ernie that whilst you seem do a lot to harden your servers and mitigate against attacks and vulnerabilities, I don't agree with your stance on firewalls.

I treat security a bit like an onion in that ideally you have layers that you have to get through before you get to the core, and personally I think it's grossly misguided not to have some form of hardware firewall at your network perimeter.

I'll admit I'm coming at this from the "what I'm used to" angle, but I know that every single month I get a nice little list of that months patches from Microsoft, many of them being exploited in the wild. I'd imagine the same happens for pretty much any OS and set of applications you care to think of.

Now I'm not suggesting firewalls do away with this, nor are firewalls immune to having bugs/vulnerabilities, obviously a "hardware" firewall is just software running on a hardware stack.

That said, I sleep a lot safer knowing that if I have a server that needs only port 443 to be accessible from the web, my perimeter Juniper ensures that only port 443 is accessible from the web. Not only that but my Palo Alto ensures that the traffic coming in is decrypted, and inspected, and logged, and scanned for IPS/IDS - not that it does away with the need to keep the server(s) secure and up to date, but why would you NOT find that a benefit given that it can mitigate for zero-day exploits and good old human error?

1

First of all, by your talk about port forwarding, I think you're conflating firewalling with NATing. While these days NATs very often function as de facto firewalls, that it is not the purpose for which they were designed. NAT is a routing tool. Firewalls are for regulating access. It's important for clarity of thought to keep these concepts distinct.

It is of course true that putting a server behind a NAT box and then configuring the NAT to forward anything to that server, is no more secure than putting the server directly on the Internet. I don't think anyone would argue with this.

Similarly, a firewall configured to allow all traffic is no firewall at all.

But, is "allow all traffic" really the policy you want? Does anyone ever have a need to ssh to any of yours servers from a Russian IP address? While you're tinkering with the configuration of some new experimental network daemon, does the whole world really need access to it?

The power of a firewall is that it lets you keep open only those services that you know need to be open and maintain a single point of control for implementing this policy.

2
  • 2
    All of your points have been addressed in my question. Yes, "allow all traffic" is really the policy we want for non-management services like SSH or some server's management port. Otherwise people wouldn't be able to see our web pages and send us mail!
    – Ernie
    Nov 12, 2010 at 22:32
  • 2
    As for only keeping open those services that need to be, that's what steps 1 and 4 are for.
    – Ernie
    Nov 12, 2010 at 22:44
1

Stateful packet inspection firewalls do NOT belong in front of public servers. You're accepting all connections, so the state tracking is pretty useless. Traditional firewalls are a huge liability in DDoS attacks and are typically the first thing to fail under DDoS attack, even before link bandwidth or server resources are totally consumed.

Stateless packet filters on routers do make sense in front of public servers, but only if they can handle line rate of all the ingress and egress traffic (such as a hardware ACL in a router).

Google, Facebook, and even Microsoft do not put traditional "firewalls" in front of public servers. Anyone who has run public web services at "more than one server" scale should know this.

Other functions found in traditional firewalls such as Cisco ASAs or whatever are best implemented on the hosts themselves, where they can be scaled out effectively. Logging, IDS, etc. are all software features in firewalls anyway, so they just become a huge bottleneck and DDoS target if put in front of publicly accessible servers.

2
  • 1
    I think context matters. The OP likely wasn't speaking about web-scale systems, where load balancing and firewalling would be implemented much differently than the back-office technology stack of an SMB.
    – ewwhite
    Aug 9, 2015 at 1:51
  • Even in front of a single server, a stateful firewall doesn't do much of anything to help you if you're accepting connections from everywhere. You need to be using TLS or other encryption for everything anyway, so all a firewall can do is say "hey, there goes some more data past me on port 443". Firewalls are pretty much dead: etherealmind.com/why-firewalls-wont-matter-in-a-few-years
    – rmalayter
    Aug 9, 2015 at 3:27
0

Why do all of your servers need a public address?

Install the server in the server room and give it a public IP address.

Out of 14 or so servers that I run on a regular basis, only 2 have publicly accessible interfaces.

Edited to add: In other networks that I've had a hand in managing, we've had the ability to turn off/on services at will, whereas we didn't have access to manage the firewall. I cannot even begin to tell you how many times, inadvertently of course, an unneeded service (SMTP) got turned on and left on and the only thing saving us from becoming a spam dump and getting blacklisted in the process, was a firewall.

Also, is all of the traffic that passes between servers, fully encrypted?

You can certainly drive a car 100 mph with no seatbelts, no airbags and bald tires, but why would you??

3
  • 1
    Because they may all have services that need to be accessed "from the Internet"
    – adamo
    Nov 12, 2010 at 22:31
  • Um, no. More like driving a car at 70 mph with seatbelts and airbags while paying attention to what you're doing. But without locking your doors. There's a security policy in place, and the servers are kept secure, but there's no firewall. Firewalls are not the be-all and end-all of security, you know.
    – Ernie
    Nov 12, 2010 at 23:16
  • 2
    I didn't say, EVER, that firewalls were the be-all or end-all of security. I won't repeat what's already been said here. They are but one LAYER in many LAYERS of security. I've asked you twice now and you haven't answered. Servers on a LAN are chatty things. Do all of your servers only talk over encrypted channels to each other?
    – GregD
    Nov 13, 2010 at 0:22
0

Firewalls can prevent system users from opening up network-accessible services that the administrators aren't aware of, or doing port forwarding to other machines. By making use of the hashlimit module, firewalls can also rate-limit abusers based on the remote IP.

A firewall is another safety net that ensures your policies are adhered to. Sure, don't run services you don't expect to.

I definitely recommend that software updates are applied in a timely manner, for example, but I also recommend firewalls on all machines. It's like when I drive: Sure I try to avoid obstacles and other cars, but I also wear a seatbelt and have airbags just in case the unexpected happens.

0

You may not be realizing how much you are benefiting from firewalls simply because everybody else is using them. In a day when literally everyone, down to home DSL users have firewalls in place port sniffing has been all but given up as a feasible attack vector. A decent hacker isn't going to waste their time checking for such things.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .