My conclusion to this was to pipe VLAN trunks through EoIP tunnels and encapsulate those in hardware assisted IPSec. Two pairs of fairly inexpensive Mikrotik RB1100AHx2 routers proved capable of saturating a 1 Gbps connection while adding less than 1 ms latency.

I would like to encrypt traffic between two data centres. Communication between the sites is provided as a standard provider bridge (s-vlan/802.1ad), so that our local vlan tags (c-vlan/802.1q) are preserved on the trunk. The communication traverse several layer 2 hops in the provider network.

Border switches on both sides are Catalyst 3750-X with the MACSec service module, but I assume MACSec is out of the question, as I don't see any way to ensure L2 equality between the switches over a trunk, although it may be possible over a provider bridge. MPLS (using EoMPLS) would certainly allow this option, but is not available in this case.

Either way, equipment can always be replaced to accommodate technology and topology choices.

How do I go about finding viable technology options that can provide layer 2 point-to-point encryption over ethernet carrier networks?


To sum up some of my findings:

  • A number of hardware L2 solutions are available, starting at USD 60,000 (low latency, low overhead, high cost)

  • MACSec may in many cases be tunneled through Q-in-Q or EoIP. Hardware starting at USD 5,000 (low-medium latency, low-medium overhead, low cost)

  • A number of hardware assisted L3 solutions are available, starting at USD 5,000 (High latency, high overhead, low cost)

  • 1
    Is there a reason to do it at Layer-2 rather than using IPSec between hosts?
    – mfinni
    Jul 1 '13 at 13:16
  • Layer 2 connectivity is a requirement. One would think that encrypting a layer 2 network on layer 2 rather than doing tunneling and fork lifting would be faster, simpler and more secure. However, IPSec/L2TP or similar (with encryption and encapsulation done in ASIC) may still turn out to be the best available option; that is essentially what I'm trying to figure out.
    – Roy
    Jul 1 '13 at 15:02
  • I might add that the price tag of two ASAs capable of maintaining 1 Gbps full duplex IPSec adds some motivation for exploring the alternatives. By comparison you can get a Catalyst supporting 10 Gbps/wirespeed MACSec for less.
    – Roy
    Jul 1 '13 at 15:09
  • There are a ton of devices that use proprietary ways to do this. I don't think there is a standard or anything. Jul 1 '13 at 15:18
  • Have you actually tried this? I don't understand how your provider adding and then removing a tag would mess up macsec. The frame that the far switch receives should be identical to the frame sent.
    – longneck
    Jul 1 '13 at 15:22

I just did a quick search for "CESG layer 2 encryption" (CESG are a british government agency who specialise in assurance for computer systems), on Google, and found a few options on their list, there's at least one that'll do 1Gbit, and a few that'll do up to 10Gbit.

It'd probably (almost definitely) be overkill, but you'll find that there's quite a lot of milspec products that are capable of Layer 2 encryption, at quite high throughputs.

The first one I found is VLAN and MPLS agnostic, unsurprisingly, but I suspect they're bloody expensive.

  • 1
    I don't know about overkill, the CN1000 was already my backup plan, if a less expensive solution can not be found
    – Roy
    Jul 1 '13 at 16:28
  • What's pricing on those bad boys like? Jul 1 '13 at 19:09
  • In these parts I believe they are listed around $35,000 (+ tax) per unit (1 Gbps ethernet edition)
    – Roy
    Jul 2 '13 at 10:27
  • About as much as I expected, I was pondering if they'd be 100K+ Jul 2 '13 at 10:30
  • Considering you get 20 Gbps worth of MACSec from leading vendors for less than $3,500 I still think the layer 2 devices I know about are insanely overpriced. One might pay 200 times more for the same bandwidth and comparable encryption latency.
    – Roy
    Jul 2 '13 at 14:10

Encryption solutions for Metro/Carrier Ethernet differ quite substantially from MacSec, which was designed for LANs and not for WANs. There is a marketoverview consisting of three documents (intro, P2P, multipoint). Google for "Metro Carrier Ethernet Encryptor" and you'll find it.

Concerning pricing it is imperative to differentiate between list prices and market prices. A 1Gb encryptor will currently cost you around $20K. If you put that into relation with the line costs, it is obvious that encryptor costs are only high if compared to non-comparable solutions.

  • I think part of the point is that WANs and LANs are growing a lot closer together technology wise. About line costs, around these parts, the cost of upgrading from virtual wire to dedicated wire/frequency (where MACSec is obviously fully supported) is a LOT less than acquiring dediacted L2 encryptors. We're talking an order of magnitude.
    – Roy
    Jul 30 '13 at 18:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.