Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let's say you have 2 servers each with 8 CPU cores each.

The servers each run 8 network services that each host an arbitrary number of long-lived TCP/IP client connections.

Clients send messages to the services.

The services do something based on the messages, and potentially notify N>1 of the clients of state changes.

Sure, it sounds like a botnet but it isn't. Consider how IRC works with c2s and s2s connections and s2s message relaying.

  • The servers are in the same data center.
  • The servers can communicate over a private VLAN @1GigE.
  • Messages are < 1KB in size.

How would you coordinate which services on which host should receive and relay messages to connected clients for state change messages?

There's an infinite number of ways to solve this problem efficiently.

  • AMQP (RabbitMQ, ZeroMQ, etc.)
  • Spread Toolkit
  • N^2 connections between allservices (bad)
  • Heck, even run IRC!
  • ...

I'm looking for a solution that:

  • perhaps exploits the fact that there's only a small closed cluster
  • is easy to admin
  • scales well
  • is "dumb" (no weird edge cases)

What are your experiences?

What do you recommend?


share|improve this question


  1. You're opposed to using an existing middleware bus.
  2. Your switch supports IGMP snooping (basically any Cisco switch you can buy will have this turned on by default---I assume Dell/HP switches will also have this feature)
  3. You're all on the same VLAN

Then IPv4 multicast will work out-of-the-box for exactly what you're trying to do. Receivers subscribe to a channel (multicast group), senders send a UDP datagram to the group's multicast address, the switch works out who gets what. It makes your network equipment handle the message routing intelligently.

If your switch doesn't support IGMP snooping, then it will treat the (Ethernet) multicast frames as broadcasts, and send them to all hosts on the VLAN, regardless of whether the host has asked for them or not. So you'll clog the tube between your switch and the hosts, even if the host OS is just dropping the packets before it gets to any application.

You can still use IPv4 Multicast if they aren't all on the same VLAN, but you've got to do more configuration on your gear to make it go, but it will still work just fine.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.