Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I attempted to get some help with this question a few days back: Peer to peer connection over SFP+ Passive 10Gbe cable

It was however closed as a duplicate. Now I have setup the physical connections between the two servers and want to see if I can get some help debugging a bit.

On the server I brought up the two devices with the following settings:

eth2      Link encap:Ethernet  HWaddr 00:25:90:55:71:AE  
          inet addr:10.0.0.1  Bcast:10.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:357 errors:0 dropped:0 overruns:0 frame:0
          TX packets:338 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:61212 (59.7 KiB)  TX bytes:27137 (26.5 KiB)

eth3      Link encap:Ethernet  HWaddr 00:25:90:55:71:AF  
          inet addr:10.0.0.2  Bcast:10.255.255.255  Mask:255.0.0.0
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:251 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:37566 (36.6 KiB)  TX bytes:4672 (4.5 KiB)

Then on the NAS I brought up the two interfaces with the following settings:

eth2      Link encap:Ethernet  HWaddr 00:1B:21:D4:8A:E0  
          inet addr:10.0.0.3  Bcast:10.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:261 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:10000 
          RX bytes:0 (0.0 B)  TX bytes:24296 (23.7 KiB)

eth3      Link encap:Ethernet  HWaddr 00:1B:21:D4:8A:E1  
          inet addr:10.0.0.4  Bcast:10.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:197 errors:0 dropped:0 overruns:0 frame:0
          TX packets:200 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:10000 
          RX bytes:18242 (17.8 KiB)  TX bytes:32648 (31.8 KiB)

I am unable to ping from either end of this deal and hoping for some advice on how to further debug this since I was told in the above post that it would work.

Thanks in advance for any and all help and advice.

share|improve this question
    
How are the 4 connections configured? Both cover the same subnet range (10.0.0.0/8 - that's a lot of address space to use for this...), so if they're just straight connections between eth2 <-> eth2 and eth3 <-> eth3 without anything connecting between them, then you can expect communication to fail as the nodes try to talk to the opposite eth3 address node over eth2 or vice-versa. If they're just peer to peer links, assign them two different IP address ranges; /30 will be plenty large. –  Shane Madden Dec 20 '11 at 16:56
    
Shane I actually changed the configuration to be 1.1.1.1 <--> 1.1.1.2 and on the other port 1.1.2.1 <--> 1.1.2.2 with the subnet 255.255.255.252 just as you stated. This worked perfectly and likely I was just tired when I attempted it the night before with the huge subnet. @ShaneMadden thanks. –  Adam Parrish Dec 21 '11 at 13:38

1 Answer 1

up vote 1 down vote accepted

It appears that @ShaneMadden above had good advice.

I tried configuring the Server with eth0 as 1.1.1.1 subnet 255.255.255.252 then on eth0 on the NAS I used 1.1.1.2 with same subnet.

I also setup eth1 as 1.1.2.1 and eth1 on NAS as 1.1.2.2 same subnet.

I was able to see that both NFS mounts from the NAS were using their respective 10gbe connections via some NFS performance testing that I found here: http://nfs.sourceforge.net/nfs-howto/ar01s05.html

Namely these two commands:

Writes:

time dd if=/dev/zero of=/mnt/home/testfile bs=16k count=16384

Reads:

time dd if=/mnt/home/testfile of=/dev/null bs=16k
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.