Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We work with devices that are flashed (factory) using TFTP on a usb network connection.

The server has a fixed 192.168.2.100 address and the device a fixed 192.168.2.101 address. When it starts, it connects to download the firmware.

In the current setup, only one device can work at the same time. But I'd like to make it possible to flash as many devices as we can plug (because we can have some massive flashing requirements).

To bypass the routing issue, I made a xinetd version that sets a setsockopt to SO_BIND_DEVICE.

But what I didn't except is that Linux is unable to handle the ARP requests on both interfaces at the same time.

if I do a "ping 192.168.2.101 -I usb0" and "ping 192.168.2.101 -I usb1" at the same time, it will work on one interface:

ARP, Request who-has sk tell 192.168.2.100, length 28
ARP, Reply 192.168.2.101 is-at 7a:0f:66:7c:fc:2c (oui Unknown), length 28
IP 192.168.2.100 > 192.168.2.101: ICMP echo request, id 21807, seq 1, length 64
IP 192.168.2.101 > 192.168.2.100: ICMP echo reply, id 21807, seq 1, length 64

But on the other it won't:

IP 192.168.2.100 > 192.168.2.101: ICMP echo request, id 31071, seq 1, length 64
ARP, Request who-has 192.168.2.100 tell 192.168.2.101, length 28
ARP, Request who-has 192.168.2.100 tell 192.168.2.101, length 28
IP 192.168.2.100 > 192.168.2.101: ICMP echo request, id 31071, seq 2, length 64
IP 192.168.2.100 > 192.168.2.101: ICMP echo request, id 31077, seq 1, length 64
ARP, Request who-has 192.168.2.100 tell 192.168.2.101, length 28

The server doesn't seem to answer to the ARP request.

This is how the connection from the device is handled on the server with a /etc/network/if-up.d/000-first script :

ifconfig $IFACE up
ifconfig $IFACE 192.168.2.100

PID=/var/run/xinetd-$IFACE.pid

# this is the modified xinetd version to bind on one address
kill -9 `cat $PID`
xinetd -pidfile $PID -interface $IFACE

# I tried this to force the handling of ARP table per interface, but it doesn't change anything:
# /usr/sbin/arpd -b /tmp/$IFACE.db -a 3 -k $IFACE 

Here is the modified xinetd version: https://github.com/fclairamb/xinetd/commit/1f5c1e8f9944e372b137e6aa46247f8de807bece#L8R253

share|improve this question
    
I am kinda wondering if you could do some magic with ebtables NAT rules. I know you theoretically can mess aground with arp, and mangle the IP addresses. Perhaps with the right magic you could get the tftp server to believe the remote hosts had different source addresses. –  Zoredache Sep 7 '13 at 8:33
    
The core logic of the device updating process is to contact the 192.168.2.100 server. The ARP request is for 192.168.2.100, I can't change the IP the device asks for on the ARP layer (as far as I know). I'm a little bit frustrated by this situation. Basically I would just need each interface to be able to reply to the ARP request on the same interface. Even arpd doesn't work. Maybe I'm missing something here... –  Florent Sep 8 '13 at 2:00

1 Answer 1

up vote 1 down vote accepted

First, if you haven't tried proxy arp by itself, you should try proxy arp.

echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp

Option 1: some connmark/policy route mangling

If proxy arp alone didn't work, I can't guarantee this will work, but it's worth a shot, I think. What I'm about to describe is how to set up policy routing to bind responses based on input interface using conntrack. I will describe how to set it up for two interfaces, and it should be easy enough to expand to any number. On the other hand, instead of working, it may simply get mightily confused and not do a damn thing (probably because the kernel arp table doesn't know it should cache ip/mac pairs per device). If that happens, try the second, more ugly method.

First, set up some connmark mangle rules:

iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j ACCEPT

In /etc/iproute2/rt_tables add two tables, 100 and 101, label them usb0 and usb1

100    usb0
101    usb1

For each of these tables add the following (replace <N> with the appropriate number):

ip route add 192.168.2.0/24 dev usb<N> table usb<N>
ip rule add fw <N> table usb<N>
iptables -t mangle -A PREROUTING -i usb<N> -j MARK --set-mark <N>

I'm not certain, but you may need to set up a dummy interface like so for your tftp daemon to listen on:

modprobe dummy
ifconfig dummy0 192.168.2.100/32 up
echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp
sysctl net.ipv4.ip_forward=1
iptables -I FORWARD -s 192.168.2.100 -j ACCEPT
iptables -I FORWARD -d 192.168.2.100 -j ACCEPT

Now test with two devices. If it works, excellent, otherwise try the next method:

Option 2: kvm container per usb net device

Option 2 is uglier, involving some bridging and qemu/kvm. virt-manager is probably the easiest way to create these things.

Create tap devices and bridges, one per usb interface.

tunctl -t tap<N>
brctl addbr usb<N>br
brctl addif usb<N>br tap<N>
brctl addif usb<N>br usb<N>  # this may need to be done each time the usb device is connected/disconnected.

Create a kvm image file or boot CD that is designed to be read only and hosts your tftp server and image... or omit read-only and make one image file per vm (using snapshots for this would be better, but that's more out of scope than this answer is getting).

Run a kvm using the tap interface and your image file and test usb network connectivity.

Comparison

If connmark/policy route worked, it's much easier to synchronously maintain the tftp repository. With KVM, you may need to have a repository per VM image (unless you pass a read-only directory through to each VM, which is doable). Connmark/policy route is a bit more finicky, but it also only requires one tftp server, though if your ports aren't random enough, conntrack could get confused and overwrite if there is a port overlap (or it may not). On the other hand, bridging to kvms requires much more memory; an entire VM per attached USB device, but it's much more likely to work every time because the VM has an entire isolated network stack to work with while the host kernel just has to pass packets back and forth over a bridge, especially true if bridge filtering is off.

Hopefully one of these two work for you.

share|improve this answer
    
This is a side project and I still haven't found the time to test option 1. ProxyARP doesn't work. Option 2 is not possible because the goal is to allow anyone to update devices with a small device like a raspberry. –  Florent Sep 16 '13 at 11:59
    
@Florent If you're designing the device-being-configured, you could have it probe for dhcp first and if it returned a tftp image, load from that address and otherwise fall back to a default address. That addresses both your needs for mass configuration and sane end-user defaults. –  Andrew Domaszek Sep 16 '13 at 14:34
    
Yes definitely. This is something that is planned but the current bootloader doesn't support it. And I'm not working on this part of the software. –  Florent Sep 30 '13 at 20:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.