6

I'm caching lots of data in memory to a local Redis server. The server will need to connect to hundreds of other servers in the same data center (as they all need to access the same information hosted by the Redis server).

In order to minimize latency and maximize throughput I plan on connecting all of the servers to the Redis server with a 40 Gigabit Ethernet (40 Gbit/s).

I understand how to connect a couple of servers to one server, but how would I go about connecting hundreds of servers to a single server? I mean, I don't see enough ports to connect so many servers...

Is there a device that will allow me to connect all of these servers together?

Improve this question
12
  • 43
    As kindly as possible : please hire a sysadmin for whatever you're doing. Hire a good one. Spend a lot of money on them.
    – mfinni
    Mar 23, 2022 at 1:49
  • 4
    You have tested this design on plain old 1Gb ethernet and you found that Ethernet bandwidth was a problem, right? Or you calculated it, at least?
    – user253751
    Mar 23, 2022 at 13:49
  • 9
    Redis is apparently single-threaded. Can one thread push 40Gb of bandwidth? I really doubt it - you'll probably be slowed down by Redis. You could use more than one copy of Redis, but if you can use 5 copies of Redis on one server, you can also use one copy on 5 servers (that you already have) and save money by not buying the one big server just for Redis.
    – user253751
    Mar 23, 2022 at 13:50
  • 16
    It's pretty strange to encounter a programmer who's never heard of an Ethernet switch, but you're one of today's lucky 10000
    – user253751
    Mar 23, 2022 at 15:57
  • 5
    "Hundreds of servers in the data-center" - Is the data-center already setup? If yes, your server can just be connected to a switch in the existing network of the data-center. If not, then the question should be "How do I setup a data-center with 100s of servers".
    – RedBaron
    Mar 24, 2022 at 7:43

3 Answers 3

Reset to default
14

I understand how to connect a couple of servers to one server but how would I go about connecting hundreds of servers to a single server? I mean, I don't see enough ports to connect so many servers.

Well... you would need to connect all of the servers to an appropriately sized network switch. They've been in use for decades. The first commercial multiport Ethernet network switch was introduced in 1989.

Improve this answer
0
9

I'm caching lots of data in memory to a local redis server. The server will need to connect to hundreds of other servers in the same data center (as they all need to access the same information hosted by the redis server).

The amount of data you store/cache is rather irrelevant. Relevant is the speed at which you need to transmit data to or from storage.

In order to minimize latency and maximize throughput I plan on connecting all of the servers to redis server with a 40 Gigabit Ethernet (40GbE).

For a large number of high-volume network ports you need a good infrastructure design. You not only require a huge bandwidth inside the redis server - network, storage and processing - but also the means to distribute that bandwidth.

Depending on the exact size, a large chassis switch (up to 800 ports or so) or a hierarchical tree is required. This paper from Cisco should provide a good starting point. A collapsed-core design is likely sufficient for your size.

Improve this answer
2
  • 8
    “hundreds of servers” talking to a single-threaded Redis instance don’t need a developer to buy a million dollar switch. They need to use that money to hire someone who has heard of a switch, and then buy a couple of 48-port unmanaged Gigabit switches.
    – Jacob Krall
    Mar 24, 2022 at 13:40
  • @JacobKrall We don't know how well that Redis server scales or whether it is single-threaded - I was trying to point out the necessity to evaluate the network, storage and processing requirements.
    – Zac67
    Mar 24, 2022 at 14:30
2

Well, it depends on your needs.

You may ask your distributor where you want to buy that hardware. They want to help you to get that deal.

One example:

If the hundreds of servers have only gigabit you might use up to 10x HPE 2930M stacked with stacking modules and stacking cables. Then add a 40 Gbit/s module in at least two of them for redundancy and connect your server with both modules via DAC cables.

The 2930M has up to 48 ports, so 480 servers + the one 40G server. Depending in your needs select different switches.

If you don't need the flexibility, but more throughput you should go with a modular switch.

You can find help on several vendors sites, but here I give you HPE’s solution for finding a switch if you want to do it all alone:

HPE Networking Switch Selector

Again: ask someone who sells that solution to you if he/she can help with the order. Otherwise, you might get in trouble when trying to put everything together and you notice that there are different power supplies, modules, cables that fit into different models. Some 2930Ms, for example, need more power than other 2930M models.

Improve this answer
3
  • 6
    Asking a salesperson for a complex technical solution is a recipe for a disaster.
    – fraxinus
    Mar 23, 2022 at 15:45
  • For that size of course not a salesperson. The Distributors i worked with had technicans/architects to help in cases like that
    – An-dir
    Mar 23, 2022 at 18:04
  • 1
    Asking anyone with a vested interest in upselling, whether that's a salesperson or the distributor's technical advisor, is still a recipe for disaster :-)
    – thkala
    Mar 24, 2022 at 19:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .