Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

The actual deployment of this will happen with a larger number of servers, but for purposes of simplicity, let's assume the following is my future setup (I need +4 reputation to embed images, so here's a link to a diagram: http://stevejenkins.com/downloads/netdiagram.png).

Assumptions:

  • Using CentOS 5.6
  • Each server has 2 NICs (eth0 and eth1)
  • All IPs are static (no DHCP issues involved)

We have two sets of servers on two different public subnets (our ISP provides the default gateways for each).

  • Servers 1 & 2 have their eth0 connected to Switch A on Public Subnet A using Default Gateway 111.222.333.1.
  • Servers 3 & 4 have their eth0 connected to Switch B on Public Subnet B using Default Gateway 444.555.666.1.

Because our ISP charges us for all traffic that goes across subnets, we'd like the servers to be able to use their second NICs to send internal traffic (backups, database queries, etc.) between themselves using statically-assigned local IP addresses (192.168.1.x) on a third switch (Switch C). That way, Server 1 can store backups on Server 4 by sending traffic across Switch C, without incurring any charges.

There's no router or default gateway for the internal network, and from what I've Googled so far, I believe I need to set up static routes on each of the servers for eth1. But I've never had to mess with routing before, and I haven't been able to find anything that explains how to set something like this up (I'm sure it exists and I probably just don't know which keywords to use).

If I can't figure it out, I suppose a lame Option 2 would be to just swap Switch C for a cheap router, which could act as a default gateway at 192.168.1.1. But I'd like to give the routes a shot and learn something new in the process.

So - how do I tell each server to send traffic for 192.168.1.x over eth1 without a default gateway in place?

Thanks in advance!

share|improve this question

1 Answer 1

No. You don't need static routes or a router if the servers' secondary "internal" interfaces are all on the same broadcast domain.

Just put each server's secondary NIC into switch C and give them an IP address in the same subnet mask (255.255.255.0 or whatever) without a default gateway and you're all set.

share|improve this answer
    
Then what should the /etc/sysconfig/network-scripts/ifcfg-eth1 file look like? I tried testing this once before with the following settings, but could never get two test boxes to see each other: DEVICE=eth1 BOOTPROTO=static HWADDR=00:06:xx:xx:xx:xx IPADDR=192.168.1.5 NETMASK=255.255.255.0 ONBOOT=yes #GATEWAY= NETWORK=192.168.1.0 TYPE=Ethernet USERCTL=no IPV6INIT=no #PEERDNS=yes –  SteveJ Aug 8 '11 at 2:57
    
I'm not familiar with CentOS, but it's the same as any other interface: eth1 address 192.168.0.2 netmask 255.255.255.0 without a default gateway. –  gravyface Aug 8 '11 at 2:59
    
That looks ok to me. Is the interface up? what does ifconfig show? –  gravyface Aug 8 '11 at 3:02
    
No, it's not currently set up like this yet - our ISP is migrating us to new IPs, so both NICs are busy servicing two different public IPs. Once the migration is finished, I wanted to set up the internal network. I'll give this a shot once the hardware is avail. Is it the NETWORK=192.168.1.0 that lets it work, even though the GATEWAY= is commented out? –  SteveJ Aug 8 '11 at 3:46
    
It works because they're in the same subnet on the same broadcast domain. No routing is necessary (and this is why you don't need a default gateway on the secondary interfaces). –  gravyface Aug 8 '11 at 11:29

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.