Background: I have a RHEL6 storage server (lots of HDs) and a RHEL6 Application server (little storage, lots of CPU/memory). They are currently connected to a 1GB Catalyst 2960-X switch. There is also a windows server and 15 client windows computers that pretty much act as terminals also connected. The storage server contains all the data and is linked to the application server via a single NFS4 mount.
Usage: Users SSH (with X11) to the application server to run jobs. Users have the option to run jobs on the storage server as well (because in the past, all we had was the storage server). The storage server will backup all of its data to the windows server nightly.
Question: Since there will be alot of network traffic between the application and storage server, is there a better way to connect the two? For example, I could connect a direct line between the two and mount the NFS4 mount on that dedicated interface. Would it better to use Channel Bonding (I have 4 NICS on each server)? Would that actually help since I'm only using a single NFS mount? Improved fault tolerance is not a concern on this system considering we are only using one line right now. Alternatively, I can used the 2nd NIC on each server for a separate VLAN. That would at least separate the SSH/X11 client traffic from the NFS4/CIFS traffic.