We run exclusively OpenBSD routers/firewalls to serve FogBugz On Demand. Unless you're operating in a transit role and need the extremely high pps throughput that purpose-built hardware and integrated software can provide, OpenBSD on solid hardware will be a more manageable, scalable, and economical solution.
Comparing OpenBSD to IOS or JUNOS (in my experience):
- The pf firewall is unmatched in terms of flexibility, manageable configuration, and integration into other services (works seamlessly with spamd, ftp-proxy, etc). The configuration examples do not do it justice.
- You get all the tools of a *nix on your gateway: syslog, grep, netcat, tcpdump, systat, top, cron, etc.
- You can add tools as necessary: iperf and iftop I've found very useful
- tcpdump. Enough said.
- Intuitive configuration for Unix veterans
- Seamless integration with existing configuration management (cfengine, puppet, scripts, whatever).
- Next gen features are free and require no add-on modules.
- Adding performance is cheap
- No support contracts
- IOS/JUNOS make it simpler to dump/load an entire configuration. Absent any configuration management tools, they will be easier to deploy once your config is written.
- Some interfaces simply aren't available for or stable on OpenBSD (e.g., I know of no well-supported ATM DS3 cards).
- High-end dedicated Cisco/Juniper-type devices will handle higher pps than server hardware
- No support contracts
So long as you're not talking about backbone routers in an ISP-like environment or edge routers interfacing with specialized network connections, OpenBSD should be just fine.
The most important thing to your router performance is your NICs. A fast CPU will quickly get overwhelmed under moderate load if you have shitty NICs that interrupt for every single packet they receive. Look for gigabit NICs that support interrupt mitigation/coalescing at least. I've had good luck with Broadcom (bge, bnx) and Intel (em) drivers.
CPU speed is more important than in dedicated hardware, but not something to fret about. Any modern server-class CPU will handle a ton of traffic before showing any strain.
Grab yourself a decent CPU (multiple cores don't help much just yet, so look at raw GHz) good ECC RAM, a reliable hard drive, and a solid chassis. Then double everything and run two nodes as an active/passive CARP cluster. Since 4.5's pfsync upgrade you can run active/active, but I haven't tested this.
My routers are running side-by-side with our load-balancers in 1U twin-node configurations. Each node has:
- Supermicro SYS-1025TC-TB chassis (built-in Intel Gigabit NICs)
- Xeon Harpertown Quad Core 2GHz CPU (my load balancers use the multiple cores)
- 4GB Kingston ECC Registered RAM
- Dual-port Intel Gigabit add-in NIC
They've been rock-solid since deployment. Everything about this is overkill for our traffic load, but I've tested throughput upwards of 800Mbps (NIC-limited, the CPU was mostly idle). We make heavy use of VLANs, so these routers have to handle a lot of internal traffic too.
Power efficiency is fantastic since each 1U chassis has a single 700W PSU powering two nodes. We've distributed the routers and balancers through multiple chassis so we can lose an entire chassis and have pretty much seamless failover (thank you pfsync and CARP).
Some others have mentioned using Linux or FreeBSD instead of OpenBSD. Most of my servers are FreeBSD, but I prefer OpenBSD routers for a few reasons:
- A tighter focus on security and stability than Linux and FreeBSD
- The best documentation of any Open Source OS
- Their innovation is centered around this type of implementation (see pfsync, ftp-proxy, carp, vlan management, ipsec, sasync, ifstated, pflogd, etc - all of which are included in base)
- FreeBSD is multiple releases behind on their port of pf
- pf is more elegant and manageable than iptables, ipchains, ipfw, or ipf
- Leaner setup/install process
That said, if you're intimately familiar with Linux or FreeBSD and don't have the time to invest, it's probably a better idea to go with one of them.