Okay, I just finished resolving a jumbo frames issue between a few Xserves, a Netgear GSM7224, and a Drobo B800i. It turned out that the Xserves (Mac OS X 10.6.8 Server) and the Drobo B800i accept the MTU in bytes as normally expected (1500-9000), but the Netgear seems to have wanted it including the various ethernet headers/footers(trailers) and I ultimately ended up with the Xserves & Drobo configured with an MTU of 9000 and the Netgear ports set to an MTU of 9216.
I've used the following commands for testing & verifying the MTU between the two Xserves over the Netgear (Note: these are the Mac OS X commands, the Windows & Linux ones differ):
ping -D -s <mtu> <ip_address> traceroute -F <ip_address> <mtu>
The former's usage is noted in the
man page as, "Specify the number of data bytes to be sent. The default is 56, which translates into 64 ICMP data bytes when combined with the 8 bytes of ICMP header data." In testing, I found that to
ping -D 1472 <ip_address> was the equivalent of MTU 1500 due to the 8 bytes of ICMP header data plus 20 bytes IP headers (see this and this). That all makes sense.
Now, why is the equivalent command for a 9000 MTU
ping -D -s 8164 <ip_address>? I've verified this is the limit before I star getting "sendto: Message too long" errors, but also that 9000 MTU is working correctly as
traceroute -F <ip_address> 9000 works and
traceroute -F <ip_address> 9001 does not. So, why 8164? I would've expected 8972 (MTU - 28 bytes, just like for 1500 MTU).
Also, why 9216 MTU for the Netgear? I counted 42 bytes for the MAC & ethernet headers (inc. CRC), plus 20 for IP headers (which should eat into the MTU).
I'm really rusty on this math and know I'm just missing something.