I have a pc and two embedded linux based systems that I am trying to chain together through IP forwarding. They span two separate local networks. Both networks are wired. I can ping 10.10.10.6 from the PC and ping 10.10.30.1 from target 1. the problem occurs when I run an application on target 1 that generates a healthy(< 3MBps) amount of UDP traffic, directed toward the PC. The performance of the system can not keep up and I see the RX dropped packet count continuously increase on target 0's 10.10.10.4 interface.
There is plenty of CPU cycles on both targets. So the question is, why is target 0 dropping so many packets?
Is there something I can do with the configuration to get the through put up? Increase buffer sizes?
+-----------+ +--------------------------------------+ +-------------------+
| PC |-->| target 0 |-->| target 1 |
|10.10.30.1 | | eth1 =10.10.30.2 eth0 =10.10.10.4 | | eth0 = 10.10.10.6 |
+-----------+ +--------------------------------------+ +-------------------+
target 0 has two ethernet interfaces and ip_forwarding = 1
And I have the following route's setup: on the PC:
Active Routes:
Network Destination Netmask Gateway Interface Metric
10.10.10.6 255.255.255.255 10.10.30.2 10.10.30.1 1
10.10.30.0 255.255.255.0 10.10.30.1 10.10.30.1 20
and on target 0:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.10.30.0 * 255.255.255.0 U 0 0 0 eth1
10.10.10.0 * 255.255.255.0 U 0 0 0 eth0
default 10.10.10.6 0.0.0.0 UG 0 0 0 eth0
default 10.10.30.1 0.0.0.0 UG 0 0 0 eth1
and on target 1:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.10.30.1 10.10.10.4 255.255.255.255 UGH 0 0 0 eth0
10.10.10.0 * 255.255.255.0 U 0 0 0 eth0
default 10.10.10.4 0.0.0.0 UG 0 0 0 eth0
Edit: target 0
# ip link list | grep mtu
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
target 1
# ip link list | grep mtu
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
Both connections have been confirmed to be set at 100Mbps Full duplex.
Another thought:
As it appears the input to target 0 is getting flooded, can the switch between the two targets, buffer the packets and then flood target 0 with a big burst of data?
Update:
By replacing the 10/100/1000 switch with an old 10M half duplex hub the system is working fine, no dropped packets.
In the bad condition, ethtool register dump indicated RX_DROP count continuously increasing. This is indicating the Processor/Linux is not able to remove the data quickly enough from the ethernet chip. With the 10M hub this is no longer a problem.
So this sounds to me like Linux is not configured correctly to keep up the higher(but not that high) data rate. I would like to understand why the OS/processor can't keep up?