I am thinking of using pfsense on my ubuntu server box. I'd virtualise pfsense with virtualbox, and have it route and firewall all traffic both on and destined to my server as well as the the rest of the LAN.
However, I've heard that using pfsense on anything but a dedicated box poses a security risk. Why is this, and is this really an issue?
Theo and others can make claims along those lines, but history suggests it's not a significant concern. Security researchers have been looking for vulnerabilities in hypervisors for years, and by and large they've escaped unscathed from at least major repeated vulnerabilities. Not entirely, you'll have to patch your hypervisor as needed where you generally never have to patch physical hardware for security reasons, but there really hasn't proven to be a significant difference in practice.
It's beyond a home networking question in general. There are numerous critical production pfSense installs running in hypervisors, largely ESX. We have 4 colo datacenters on the *.pfsense.org hosting infrastructure that strictly run virtual firewalls. I like it because we can scale way up on CPU power, RAM, etc. as needed without dedicating expensive servers to firewalls, and based on history I'm not concerned about the security of the hypervisor (but keep an eye on new developments).
You do need to take care to ensure the host OS of your hypervisor cannot bind IPs to the NIC that goes to your unfiltered Internet connectivity. That's the significant risk in running your edge firewalls as VMs, it's easier to screw up the networking on a hypervisor than it is to plug physical cables into the wrong place.
However, I've heard that using pfsense on anything but a dedicated box poses a security risk. Why is this, and is this really an issue?
The ever quotable Theo de Raddt of the OpenBSD project has the answer behind this mentality:
You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.
The recommendation to not run servers or appliances that fulfill security roles in your network on virtualization platform boils down to a fundamental principle in designing secure networks and systems: Functional Separation. Ideally each device or element in your system should have one functional role. This reduces the attack surface, and makes configuration and troubleshooting much simpler. When you run a firewall on top of a virtualization platform, not only do you have to securely configure the firewall, but also the Virtual Machine Monitor and the underlying host operating system.
In the real world, we can't achieve perfect functional separation for any number of reasons (limited resources being an an obvious one) but we can aspire to. And while I don't think running firewalls on virtual machines is the absolute worse thing, I have a strong distaste for it. Think carefully about your risk environment and available resources and make a educated and documented decision.
We have been using virtualized pfSense instances under Linux with KVM for years. It's less secure only in that you need to worry about the security of your host in addition to the pfSense guest, as a compromise of the VM host would allow an attacker to at the very least shut down/kill any guest VMs and at worst complete control over them as well.
