1

I am trying to do some traffic inspection on a lxc container in my host machine. My host machine is connected to a mirror port and being sent duplicate copies of the traffic of 4-5 other machines on a local network.

I can see all the traffic from these other machines through wireshark, so I know that they are reaching my interface, but I am having trouble forwarding these the last hop to my lxc container.

I have tried doing NAT through iptables, but I suspect after reading a few other posts that they are being filtered prior to reaching iptables. I saw one or two suggestions for ebtables and have tried to do some static routing unsuccessfully.

Any help or pointers to existing literature would be greatly appreciated.

2
  • Yes, just read the packets passively no response to original host necessary. It is not a dedicated mirror port. I am duplicating all the packets with an open flow rule. I am getting the whole ethernet frame including original MACs
    – m00sef00t
    Dec 27, 2019 at 22:46
  • No I don't, as I'm working with a small simplistic network and edge device, it's only one hop from the switch. So I can differentiate it based on the fact that it's destination address is the ip of the other machines.
    – m00sef00t
    Dec 27, 2019 at 23:08

1 Answer 1

0

I strongly suggest you encapsulate your traffic somehow. Easiest is probably VLAN (because I know how to handle this easily with tc), but of course ERSPAN is a defacto standard. Without encapsulation, it's not easy to guess if the received traffic is for your system or from the mirror (what about broadcasts? multicast? ARP?)

So I give here a simplistic way to duplicate again all your received traffic to your containers, with the exception of IPv4 packets having your own IP (I won't care about ARP packets or other such less frequent traffic for your host). If the received traffic was encapsulated, more options would be available (starting with redirecting rather than mirroring everything), and this answer would have to be reworked.

You can't expect normal routing, or even bridging to apply when receiving mirrored traffic. You can use a low level tool available to do this: Traffic Control (tc). It's a quite complex tool able to handle various different network features primarily intended for traffic shaping and alike. Suffice to say that there's an action to duplicate packets described there: tc mirred, that an action is used by a filter and that a filter applies on classful queue discipline (qdisc). As it's incoming, the special ingress qdisc is used.

I won't describe how to integrate this automatically with LXC (it's probably possible by supplying a fixed name in lxc.net.[i].veth.pair for the side attached to the bridge intended for LXC rather than letting LXC choose a random name and then using some hook for scripts, as long as the interface are already created when the hook is invoked). Because the interface names must be existing when creating the tc filters, when doing this manually the containers must be started before,

So let's suppose this (edit to suit your configuration):

  • your unique real interface is called eth0,
  • your "external" (assigned to eth0) system IP is 192.0.2.2,
  • you started two containers, and their veth pair's side on the host are called veth123456 and veth89ABCD. The fact that they are enslaved as bridge port (eg: to lxcbr0) doesn't matter.

Let's do the initial plumbing:

tc qdisc add dev eth0 ingress

Let's add the first copy, with a filter priority (to override it later) and provisions to add more:

tc filter add dev eth0 ingress prio 10 matchall \
   action mirred egress mirror dev veth123456 continue

Because the continue control was used, this filter is not final. More can be added (and continue could again be added for even more filters):

tc filter add dev eth0 ingress prio 10 matchall \
   action mirred egress mirror dev veth89ABCD

Since the criteria are the same (match all), the two previous filters could also have been rewritten, using the pipe action control, with this unique filter instead:

tc filter add dev eth0 ingress prio 10 matchall \
   action mirred egress mirror dev veth123456 pipe \
   action mirred egress mirror dev veth89ABCD

Now it's quite easy to insert a filter with an earlier priority to prevent the copy of packets with the destination IP (but not ARP or broadcasts or multicasts or IPv6 ...) intended for the host (pass makes the action final):

tc filter add dev eth0 ingress prio 5 protocol ip \
   u32 match ip dst 192.0.2.2 action pass

Note that the bridge (and almost all the network stack) was entirely bypassed when doing this: the packet was moved from an interface to an other, short-circuiting everything in between. You could also create an additional veth interface with the host side not connected to a bridge (if LXC can allow this? else just use a dummy bridge) and copy the packets there, thus having a dedicated interface in the container.

To display filters with usage statistics:

tc -statistics filter show dev eth0 ingress

To cancel everything, deleting the qdisc will also delete the filters:

tc qdisc delete dev eth0 ingress

UPDATE: the interface is receiving traffic not intended for it, that means the ethernet frame's destination addresses are most likely not the interface's MAC address. By default the interface's hardware will filter out those frames and they'll never be seen by the system, including by tc, leaving only mirrored broadcasts (eg: ARP requests) visible. So better turn the interface in promiscuous mode:

ip link set dev eth0 promisc on
1
  • Also note that for the ingress case only, like here, nftables can do something equivalent when using the netdev address family (which hooks to pre-existing interfaces, just like tc).
    – A.B
    Dec 28, 2019 at 0:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .