Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm not sure "honor" is the right word for this, but it's the best I could come up with. I have a scenario where I have two servers on the same network. They have primary and secondary IPs, all on the same subnet. For the sake of discussion, they look like this:

server1    eth0
server1-A  eth0:11
server1-B  eth0:12

server2    eth0

Yes, server1 is set to /24, and yes it's a mistake.

I noticed this problem because connections from server1->server2 had a source IP of instead of Since the app originating the connection does not specify a source IP, I was shocked that it wasn't using

That's when I noticed the incorrect netmask. Since the target IP is in a known smaller network, it uses an IP from the same /27 instead of the IP it thinks is from a /24. Oops.

So, I fixed the netmask on server1:eth0 by running the following command:

ifconfig eth0 netmask

ifconfig seemed happy now too:

eth0      Link encap:Ethernet  HWaddr 00:22:19:54:EF:11  
          inet addr:  Bcast:  Mask:
          inet6 addr: fe80::222:19ff:fe54:ef11/64 Scope:Link
          RX packets:1085587580 errors:0 dropped:1355 overruns:0 frame:0
          TX packets:1208356392 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:365708046601 (340.5 GiB)  TX bytes:667099868812 (621.2 GiB)
          Interrupt:169 Memory:f8000000-f8012100 

Also, the routing table cleaned itself up.


server1 0 /home/jj33 ># route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface U     0      0        0 eth0   U     0      0        0 eth0     U     0      0        0 eth0         UG    0      0        0 eth0


server1 0 /home/jj33 ># route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface U     0      0        0 eth0     U     0      0        0 eth0         UG    0      0        0 eth0

The only problem is, after all of that, the OS still seems to choose as the source address for outbound connections to the same network (SMTP doesn't directly figure into this problem, just a convenient way to show the source IP of a connection):

server1 0 /home/jj33 ># telnet server2 25
Connected to
Escape character is '^]'.
220 ESMTP mailer ready at Wed, 23 Dec 2009 12:18:28 -0600
ehlo foo Hello []
250 HELP

(in case it's not obvious, I would expect the mailer to say "Hello []" in response to my ehlo if everything were working properly).

So, now my question. How can I get my OS to take notice of the fact that the netmask on eth0 has changed such that it's a better choice for outbound connections to my local /27? I assume restarting the server or restarting networking services would do it, but I would have to wait a week until my next maintenance window and it seems like something I could do without interrupting service (this is a production system and this improper source IP is a small, tangential problem - the core app is working well).

Any help greatly appreciated. Thanks!

UPDATE 1/8/2010:

So, this problem drew more attention than I was expecting and I ended up getting permission to fail the application over to the standby silo and restart networking services on the affected server outside of our standard window, meaning I can't test any of the theories below.

In general though I believe Juliano's response covered the most detail. I didn't copy and paste it but in playing with ip it generally seemed to corroborate what he posited.

Also, enough people piled on about using ip in preference to ifconfig that I spent some time playing with it and tip my hat to you all, I certainly should be using ip. Thanks for the pointers.

share|improve this question
Please, do not use ifconfig and route, they were written 15+ years ago when Linux had a very different network stack. Instead, you should use the ip family of commands. Also, please, post the results of ip addr show. – Juliano Dec 24 '09 at 13:23
up vote 3 down vote accepted

First things first, do not use ifconfig and route. These commands are generally regarded as obsolete today; they were written too long ago when Linux had a very different network stack, and have been patched ever since. The very idea of interface aliases (e.g. ethX:YY) in order to have multiple addresses is obsolete today, they still exist mostly to please ifconfig itself. Today, the ip command should satisfy all your needs.

Now, understand your original situation: Your eth0 interface originally had two active scopes: /24 and /27. was the primary address for the /24 scope, while was the primary address for the /27 scope (because it is listed first). When you issued the ifconfig command to change the prefix of the first address, it deleted it and reinserted it as a secondary address in the /27 scope. So now you should have something like this:

inet brd primary   eth0:11
inet brd secondary eth0:12
inet  brd secondary eth0

It doesn't matter that eth0 should be primary, or that it looks like it should be primary (another reason not to use ifconfig). It was inserted latter in the /27 scope, so it is a secondary address. This also means that outbound packets will be addressed, and that if you bring eth0:11 down using ifconfig, all your addresses will be taken down together. This is how it works.

The only way to fix this is to remove all addresses from the interface and reinsert them in the correct order. Then, the first address added (in the /27 scope) will be the primary address in that scope, and further addresses will all be secondaries.

The addressing was already broken from the beginning, there wasn't much you could do in this situation. Your best solution is to just restart the network service.

One possible workaround is to change the source routing address. This will have almost the same effect of changing the primary address. In your case:

ip route change dev eth0 src

In this case, packets going to will have the source address set to You will need another command if you also want to change the source of packets passing through the gateway.

share|improve this answer
I'm accepting this because it seems to get closest to the heart of the issue, even though I can't test it anymore. Thanks for your details response. – jj33 Jan 8 '10 at 15:48

I had a similar issue (two servers with eth0 and eth1 in the same ethernet segment) and couldn't figure out how to force a source in my case. However, you may try this kind of approach to force the source ip in your case :

ip route add dev eth0 src metric 2

It's about metric again but including the source in the equation. On my home configuration, it allowed me to pick a different ip to connect to my server compared to what the kernel would choose by default.

share|improve this answer
That looks interesting, thanks for the note. – jj33 Dec 24 '09 at 14:13

You did not say what distribution are you using. You should change the configuration files and use some kind of initscript to reload the network settings. (If you skip this step your settings will be lost after reboot.)

The second thing is that nowadays the ip tool is preferred over ifconfig on linux. With ip it you can add and remove ip addresses on-the-fly.

share|improve this answer
CentOS 5.1, though I don't expect the core of this issue to be distribution specific. I did indeed modify the config files such that they will be correct at boot. I fully expect that if I ran 'service network restart' my problem would be fixed, but that would interrupt produciton traffic, which was the entire point of my question. ifconfig can certainly add and remove ip addresses on the fly, but I will look at ip to see if it offers some control over this situation that ifconfig does not. Thanks for the response. – jj33 Dec 23 '09 at 19:07

Have you tried setting the metrics with ifconfig?

metric n Set the routing metric of the interface to n, default 0. The routing metric is used by the routing protocol. Higher metrics have the effect of making a route less favorable; metrics are counted as addition hops to the destination network or host.


Basically you want to prefer using one NIC over another. Try setting links A and B to have a metric of 1.

share|improve this answer
metric in and of itself wouldn't fix this I don't think because all the IPs are using the same routing entry in the table. It's not about choosing the destination, it's about choosing the source IP, and all the source IPs are (now) in the same subnet. Thanks for the response! – jj33 Dec 23 '09 at 21:29
Ah. I got it backwards. – Joseph Kern Dec 24 '09 at 1:03

Not sure if I understand but maybe you can do something like

route add -net dev eth0

This forces all connections to that subnet to go through eth0 which has the IP address you mentioned.

share|improve this answer
I already have a route for that network, via that device, in my routing table as shown in the route -n output above. The "Device" in the routing table seems to be physical device, not logical device, so they all are set to the physical device eth0 regardless of what the logical device of the chosen outbound IP is. Thanks for the response! – jj33 Dec 23 '09 at 21:31

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.