Load balancing multiple networks

Question

This question in not about how to do it, but about what should be done.

The situation is as follows:

• A small/medium sized company site
• Multiple DSL and Cable internet connections (nothing better available except for 10x the price)
• Multiple internal networks
• The need for the traditional network features: DHCP, VPN, nat, firewall, etc.
• The desire for more advanced features: DPI, filtering of malicious sites, traffic analysis

What is a good way, hardware and software wise, to connect everything together? (I won't elaborate on the current "solution" because it is horrible)

• I want to have gigabit speed inter-lan routing, at least between three of the LANs.
• The total download speed of all internet connections is nearly 500 MBit/s.
• The VPNs are not heavily used; total VPN throughput is in the 50 MBit/s range, but the latency should be kept low.
• The networks (LAN-LAN and WAN-LAN) should be firewalled.
• QoS/Traffic Shaping on the (load balanced) WANs is desireable since the devices have very different load patterns. It ranges from 24/7 video streams with low priority to business applications with low bandwidth consumption who desire a low latency.
• HA would be desirable but is probably not feasible for the PPPOE-based DSL lines with a single IP. But is there something else which would keep the downtime to a minimum in case of failure?

I have considered solutions:

A) Use one big router

This would mean to connect all WANs and all LAN/VLANs to a single router which would do everything. But is it feasible?

Using a software firewall like pfSense or sophos would offer a lot of features. BUT it would require a server with at least 8 gigabit ports and it might be to slow. From all that I could find, using multiple routes at gigabit speed uses quite a lot of CPU power which then becomes the bottleneck.

B) combining two routers

One router could handle the WAN side, do the load balancing, WAN firewalling, Traffic Shaping, etc. Could a software solution like pfSense handle this on a not-so pricey hardware?

Another router would then handy the inner LANs. Maybe something like an 8-Port Edgerouter could handle this. It has a firewall which is complex enough for the inter-lan part (but too simple for the WAN part) and can do hardware offloading.

Would this work?

How about the VPN connections? How should the other functions (DNS, DHCP) be divided between the routers? Would NAT work? (Since the clients and pfSense would be in different networks)

Are there other, better, solutions?

Answer 1

Solution C:

Two big router/firewall in HA with pfsense. Now in my data center I use 2 virtual pfsense to mange all:

• 16VLAN tagged
• 180 virtual servers + 19 physical servers
• 16 public IPs + 254 public IPs
• IPS+IDS
• 8 IPSec VPN
• 21 OpenVPN for remote clinet
• NAT, traffic shaping
• Load balancer and failover (for public web site) ecc ecc...

10Gbit/500Mbit internet feed and 10G/10G LAN speed.

Answer 2

How much is your time worth? I would simplify.

• Use the two fastest, most stable and diverse connections; e.g. coaxial cable and the best DSL.
• Purchase a reasonable new-generation firewall. A Cisco Meraki MX64 or MX84 are good candidates.
• A Meraki MX64 is $1200 with all of the content filtering and IDS features. A standby unit is $400.

Configure your WAN uplinks. PPPoE is supported.

Configure the Meraki for the appropriate VLANs and ACLs.

Set your WAN speeds...

Set a load balancing policy...

Build traffic shaping rules...

Client VPN is built in. Site-to-Site VPN is available. You didn't clarify what you needed.

You didn't explain if you have inbound services to support. Load balancing is transparent for outbound connections. Inbound would need some form of DNS load balancing (I use AWS Route53, 30 second TTL and health checks).