10
votes

I have a Java application where scalability is primarily limited by RAM, that I would like to run on one or more servers in a datacenter. Where should I be looking for server hardware that can accommodate 100GB - 512GB or more of RAM? I'm not an expert in such matters so I really don't know where to start.

Is this getting into supercomputer territory (6 figures or more), or could I obtain such a server for low 5-digit dollars?

A few notes based on some questions below:

  • Yes I have tried hard to think of ways to remove this scalability requirement, and no its not really an option. The application fundamentally requires very fast random access to very large amounts of data, storing in a hard disk (via a database perhaps) won't cut it.
  • I'm pretty sure the JVM can, at least in theory, scale up that far. I regularly run my code with 10GB allocated to the Sun 1.6 JVM without noticeable problems.
0

18 Answers 18

6
votes

Unusual requirement sometimes benefit from unusual solutions. Sure you can give 6 figures to Sun, Dell or HP and be done with it, but it is not the only game in town.

For single box solutions, getting up to 128GB is very cheap (32 x 4GB ~ USD 3.000), even with homebrew motherboards that cost less than USD 1.000. (don't mock the makers. If it's good enough for Google ... )

256GB is seriously more expensive (32x8GB ~ USD 18.000), and beyond that ...

Alternatively have you considered Infiniband (10Gbps) interconnected cheap boxes as an alternative?

You could build a 4 node, 16 processor (64 cores), 512GB machine that way and still have change from USD 25.000 .

You would furthermore have the added benefits of gracefull degradation, if your application can run on 3 machines if one of them fails, and possibly get a linear scaling in cost up to 8 nodes (just add 4 more nodes). At that point you are looking at a cool 128 core, 1TB RAM beast for < USD 50.000 .

Before you dismiss the Infiniband proposal as exotic, it isn't for the type of machine you are asking for. e.g. 141 of the top 500 supercomputers are built this way, including 4 out of the top 10 ( http://top500.org/connfam/8 )

2
  • I dont know if Infiniband is the right solution (I have no experience with it), but (in 2011) I would call a system running Java with 100GB+ ram on a single server to be exotic. It is time to consider exotic solutions. Aug 23, 2011 at 16:05
  • -1 for being really, really misleading. Most of the supercomputers in the Top500 are using InfiniBand to provide low-latency networking, not to provide a single coherent image over RDMA -- that usage is really exotic. To make use of that, you need to use a Single-System Image or vSMP product. While you can use something like Kerrighed or OpenSSI for this, these offerings are based on modified kernels and can't split a single process image across nodes. Only ScaleMP, which is a very expensive solution, can deliver a real coherent system image across multiple RDMA-connected servers. Mar 12, 2012 at 19:23
3
votes

Alright, look. You're not going to find a server that has the sort of RAM footprint you're looking for, at least not one that doesn't require its own electrical grid.

Why not take a scalable approach, and use memcached? You can spread the memory around to different machines across the network. The data never has to touch a disk drive, and with the sort of ultra-fast network you can buy with the money you're talking about, latency will hardly be a problem at all.

Here's a memcached client for java: http://www.whalin.com/memcached/

And here's an intro to memcached in case you're not familiar: http://www.danga.com/memcached/

Look into it. It's going to be way more cost effective than building a single monster machine with an insane amount of RAM. Besides, if you're doing something that has that kind of requirement, it's probably mission critical, and you don't need a single point of failure.

3
  • Good idea. I almost like it more than my own idea.
    – phuzion
    Jun 26, 2009 at 2:48
  • That's nonsense. Sandy Bridge, which was launched last week in a server part, can scale up to 768 GB in 1U server package. If you're looking to stick with Westmere parts, you can connect two IBM x3850 or similar servers together via QPI links and power them with less than 4000 watts. (That's the same power footprint as four 2U servers in the same rack space.) Presumably AMD has some competitive offerings in this space as well. Mar 12, 2012 at 19:26
  • 4
    @jgoldschrafe This was asked (and answered) 3 years ago. Mar 13, 2012 at 19:53
2
votes

4 or 8 socket Opteron servers such as the HP DL585 or DL785 or the Sun X4600 can take large amounts of memory in the 128-256GB range. Although they are not cheap, they are certainly not into 6 digit price tags; An 8-way, 32 core Sun X4600 with 256GB of RAM lists at around $35,000 on their web site, and that's about as big as this type of system gets. You will probably find that you can get the system for somewhat less than the list price shown on the web site..

Although 4Gb DIMMs are available, they tend to go at a large price premium, so going up to a system maxed out with these would be considerably more expensive.

If you want to use a system of this type, you will need a 64-bit O/S. Make sure you also get a 64-bit JVM and check that it works well with your application.

1
  • I think you mean a 64-bit JVM, not a 54-bit JVM :P
    – tegbains
    Jun 25, 2009 at 23:34
2
votes

I won't repeat the hardware suggestions (which are sound) but you might want to look at Terracotta to see if it fits for your app.

http://www.terracotta.org/

2
votes

BE absolutely careful which such RAM sizes. We had scaled up a HP machine to 64 GB (HP stated that the machine can take 128 GB), but only after adding an additional riser board, a cooling shaft and so on (after a lot of chatting with HP).
Only because a machine is specified to take up to n GB, it does not mean that it will work without additional changes. In our case not all normal memory modules worked, because they got to hot, only very specific modules worked.

1
vote

The cost of RAM doesn't scale linearly to large sizes. Just because I can buy a 1GB DIMM for $15 doesn't mean I can get a server with 128GB for just $1,920 ... for a start you won't find a motherboard with 128 DIMM slots in it.

Above a certain size (~8 to 16GB) you start to see motherboards requiring fully buffering DIMMs (FB-DIMMs), which will cost you considerably more per GB than standard desktop memory.

We regularly use machines with 128GB of memory in them and the price has come down a long way in recent years, but I don't have any current numbers ... nor any experience of how well the JVM would scale to that size of memory.

1
vote

You actually have lots of options, just from the HP list you have their BL680c blade which can take 128GB, their DL580/585's can take 256GB and their DL785 can take 512GB. Some of IBM's go up to 256GB, as does one Dell too.

0
votes

I think you will start to run into headroom issues at 64gb on traditional hardware. If you can scale out from there you would be ok but my guess is that the far more cost-effective solution would be to question your architecture. Granted I say that with no knowledge of what you are doing but I am just throwing that out there.

1
  • Unfortunately there isn't any easy way to get around the RAM requirement, as the application requires very fast random access to large amounts of data - storing the data on disk just won't cut it :-(
    – sanity
    Jan 31, 2009 at 1:44
0
votes

Would Amazon's EC2 solution be viable for you? It would certainly be the most cost effective solution.

1
  • Afraid not - the maximum amount of RAM an EC2 server can support is 14GB, last time I checked anyway.
    – sanity
    Jan 31, 2009 at 1:45
0
votes

Let's say that you could fit that much memory into a server (if I'm not mistaken, Linux on standard hardware is limited to 64GB, but I'm not sure).

Under most operating systems, the JVM is restricted to a heap space of about 1.4GB-1.6GB, partly because contiguous memory is required and partially because of operating system restrictions.

Hence, an extra RAM wouldn't help you scale up one application, it would only let you run multiple instances of the application. However, you would then need multiple cores and run into various other issues.

What do you need that much RAM for? You may be able to find databases that can be stored in memory or use a RAM drive, but I'm not aware of a JVM that would let you store that much stuff in memory.

5
  • I'm pretty sure the JVM isn't restricted to a heap space of 1.6GB, I regularly run it with 10GB and more with Sun's JVM, of course it has to be on a 64 bit machine.
    – sanity
    Jan 31, 2009 at 1:47
  • I disagree. See: unixville.com/~moazam And several questions here on SO. I am not sure about 64-bit JVMs, AFAIK that is not supported on my 64bit mac at this time, don't know about 64-bit Linux / Win.
    – Uri
    Jan 31, 2009 at 1:58
  • I am using a 64bit JVM, in fact, I'm using one on a Mac. Apple released Java 1.6 for 64bit Macs quite a while ago.
    – sanity
    Jan 31, 2009 at 2:24
  • I wouldn't know, since Eclipse doesn't run on 1.6 for me... But ok, I accept that. What's the max RAM you can put on your machine though?
    – Uri
    Jan 31, 2009 at 2:37
  • i use 64 bit jvms with 16 gb heaps all the time
    – pdeva
    May 30, 2009 at 13:24
0
votes

A typical way to get more system memory is to get more systems. If memory is really the bottleneck, then it's not so much how much memory you've got, but how well connected your data is to your processors. You'll need to scale a lot of things up for that to do you much good.

To clarify, Just adding a couple of zeros to your system memory is probably not going to do what you think it will. What you will find is that now that your whole dataset fits in memory, or even a slightly larger slice of it, you'll run into some other bottleneck, like cache invalidation.

The proper way to scale your system is slowly. If you're currently running, say, on a 4 core system with 8 gigs of ram, first profile the hell out of your app to see where it's really spending time, then try bumping up to 12 or 16 gigs of ram and see how the profiling results have changed.

The real question is, why would you need about 100 times the system memory, relative to other resources, than what is commonly available. If your data access pattern is in some way predictable, what you should be doing is increasing disk bandwidth, several raid controllers with several striped disks will achieve this.

If your data access pattern is really, really random, then there's probably room for a better optimized algorithm.

0
votes

You probably need a specialized server for it.

Try to look at ES7000 from Unisys. The description there is probably a bit out-dated.

It can support up to 512GB of RAM. It's using well-known O/S, like Windows and Linux Enterprise.

It will costs you ~$30K for the standard configuration, but with Itanium and all the bells and whistles, it could go up to ~$600K.

With that much RAM, you can keep lots of hot-data without touching disk space at all.

0
votes

You obviously need 64-Bit Operating Systems, but you're not in Supercomputer Territory. Just as an example, Dell's PowerEdge R900 and R905 are available with 256 GB RAM and use plain standard Intel Xeon/AMD Opteron Processors and run Linux, Solaris or Windows 2003 and 2008.

Of course, buying RAM directly at Dell is not very cost effective (They want ~35,000 US$ for 32 x 8 GB, while you can get it already for about 23,000 US$, possibly less), yet on the other hand you may want to ensure that you have proper support if you're buying a 40,000 US$ Server (you did not expect 256 GB RAM to be cheap, did you? If 128 GB are also OK, you can save ~12,000 US$).

I have no experience on which Operating System to choose though, running 100+ GB Java is usually not something I do :)

0
votes

How about a completely out-of-the-box solution: A database. I know you said it would be too slow but that's based on what's hosting it. How about hosting it on a RAID0 array of enough of These.

$400 for the gadget, Pricewatch lists chips at $55 (I haven't checked compatibility) for 4gb, so that's another $440 for the memory. That gets you 32gb for $840. (The device can in theory take 8gb chips for a total of 64gb but no chips are yet supported.)

RAID0 4 of these and you're in the low end of your range for a bit over $3000 + an ordinary box. 16 of them gets the high end of your range for $14k.

Whether this is useable or not also comes down to the nature of your data--these devices are volatile and will deplete their backup battery in a few hours although they can back up to a CF card.

0
votes

I'm a large fan of the "many cheap servers" approach. Have you looked into maybe running this kind of a process on the Eucalyptus platform, available on Ubuntu 9.04? It's possible that you can run this kind of program over a few computers on their own dedicated gigabit network with multiple servers running 8, 16 or 32GB of RAM, and scale in a linear fashion, adding more cheap servers when you need them.

0
votes

I did read your comment on the nature of your application, but still, you might consider alternative solutions.

FusionIO is one real alternative. Just have a look. At 10K $ it is still much cheaper than high end server. Write bandwidth of 1.0 GB/s - that sounds really crazy.

Another option is SSD, of course. Just in case you have seen the specs for Intel® X25-E Extreme SSD:

Read Latency 75 microseconds
I/O Per Second (IOPS) Random 4 KB reads: >35,000 IOPS
Random 4 KB writes: >3,300 IOPS
Sustained sequential read: up to 250 MB/s
Sustained sequential write: up to 170 MB/s

Putting a bunch of them into raid 10 array can give you enough performance. And with USD 400 per 32 GB, it is so much cheaper then alternative high-end servers.

0
votes

In a similar vein to the FusionIO suggestion, you can get devices that let you hook dynamic RAM to a SATA interface. Something like this (I have no experience of the product or company, it is just the first option that came out of a "Google Shopping" search).

You could use a couple of these as a mounted filesystems to cache data using your app's logic (it is battery backed so should survive boot and other outages) or you could use them as swap space and let the kernel use decide how to use them (though as OS kernels are usually optimised assuming all swap locations are more orders of magnitude slower and more latent than real RAM then this will be, you'll probably have to tweak it significantly to get best use of such an arrangement).

The FusionIO option is going to be better value for money if you really need something that big, this sort of RAM drive may be better as a compromise. Working out how well a server capable of 128Gb RAM on the motherboard and a couple of these with the full 64Gb populated compares price- and performance-wise to a specialist server that supports 256Gb or more directly, I leave as an exercise for the reader!

0
votes

3 years after the questio, things are much easier.

I have been looking up some Siliconmechanics configs.

The cheapest way would be to use AMD platforms with 32 dimms - 512GB - 11.940$.

An alternative, but much more expensive per GB is an Intel platform with 64 dimms - 1TB - 48.769$.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .