Background: I have a RHEL6 storage server (lots of HDs) and a RHEL6 Application server (little storage, lots of CPU/memory). They are currently connected to a 1GB Catalyst 2960-X switch. There is also a windows server and 15 client windows computers that pretty much act as terminals also connected. The storage server contains all the data and is linked to the application server via a single NFS4 mount.

Usage: Users SSH (with X11) to the application server to run jobs. Users have the option to run jobs on the storage server as well (because in the past, all we had was the storage server). The storage server will backup all of its data to the windows server nightly.

Question: Since there will be alot of network traffic between the application and storage server, is there a better way to connect the two? For example, I could connect a direct line between the two and mount the NFS4 mount on that dedicated interface. Would it better to use Channel Bonding (I have 4 NICS on each server)? Would that actually help since I'm only using a single NFS mount? Improved fault tolerance is not a concern on this system considering we are only using one line right now. Alternatively, I can used the 2nd NIC on each server for a separate VLAN. That would at least separate the SSH/X11 client traffic from the NFS4/CIFS traffic.


Avoid direct connections between servers, it'll be something of a nightmare to set up and it's horrible for maintenance. Setting up an Etherchannel between the switch and the server is the preferred option, and you can load balance connections over all 4 NICs with a couple different strategies (source MAC, destination MAC, etc.). However, between two hosts, the maximum throughput will still be only 1 Gbps, but you can have 1 Gbps between app and storage, while the app server is communicating with clients on the other NICs.

If the storage on the storage server is only accessed by the application server and the windows server, another option would be to install 10 GE NICs in all the servers, get a cheap 10GE switch, and set up a small storage network.

EDIT: Another option is to separate out the traffic on separate VLAN, that way the storage traffic gets it's own interfaces which better guarantees throughput.

  • Yes, I understand I am limited the speed is the same, I just want a wider pipe. Will NFS4 split across multiple NICs using Etherchannel? I'd get 10gE but we currently dont have the funding. What about connecting the 2nd NIC from each server to the switch as well and putting those on a separate VLAN used only for the mounts? Then at least the mount traffic would be dedicated and split from the client traffic. – eng3 Mar 5 at 12:51
  • A second VLAN for NFS would be preferable, then you aren't at least competing with other traffic over those links. Etherchannel need to send all traffic from one "traffic pair" over the same link, but you can change how it splits the traffic. For example, if you have a webserver that serves a webpage to the Internet, you want to divide the traffic based on source mac address. On the other hand, if you have a server sending a lot of traffic to clients, you want to balance according to destination mac address. However, this doesn't help very much when you have one host talking to another host. – Stuggi Mar 5 at 14:33
  • Yes, the issues seems to be that I just have one host talking to another and it is just one NFS mount. – eng3 Mar 5 at 19:05
  • Then an Etherchannel (or any other port aggregation) isn't going to help you much, except with redundancy. Best bet is to go for a separate VLAN, if you can't go for 10G Ethernet (check the prices though, they are a lot lower these days). – Stuggi Mar 5 at 21:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.