Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We are a startup looking at putting our semantic search system for resumes live. The system needs to provide responses to api queries for resume search by running through inverted index data of size around 25 gb (these are not inverted text indexes but rather skill indexes with ranks associated and so require computation and a "from scratch development" rather than using a library like lucene etc). The application is coded in python and the hardware we go for will need to provide the following:

a. 32 GB of RAM so that we can keep the 25gb data in memory to serve requests in real time. b. Ability to handle 3-4 concurrent connections. c. Ability to handle around 1000 - 1500 API calls a day. Each API query causes a search computation to happen over the data in memory and returns results on the basis of certain match and ranking algorithms. d. Uptime requirement is not very stringent. As we will be in the beta phase for some time (say 6 months), we can afford say upto 95% uptime if that can lead us to manage with a cheaper solution.

Wanted inputs on whether we should go for 1. A server system (main drawback - costly, and cash is king for startups) 2. A desktop solution with a board supporting 32 gb (have not been able to locate 1 in India where we are based, max we are getting is 16gb. Main benefit - cost. Main Drawback - Lesser than server consistency) 3. A hosted solution (Main Drawback - Cost for a dedicated 32 GB server).

Thanks in advance.

share|improve this question

migrated from stackoverflow.com Mar 16 '11 at 9:22

This question came from our site for professional and enthusiast programmers.

2 Answers 2

A few rough numbers and comments. Note that values are from North America and I don't know how that compares to anything in India.

  • A dedicated server rented monthly with 32GB will likely be in the 700$/month range.
  • A High-Memory Double Extra Large Instance Amazon EC2 instance with 34GB of memory is the same price, 700$/month, for 100% utilization but not including any storage/bandwidth. You can play around with the Amazon calculator and someone more familiar with Amazon services can tell you what you do and don't need.
  • Buying a server with 32GB of memory will start around 3000$ but this doesn't include any hosting/bandwidth.
  • How sure are you that you need a 25GB index all in RAM? Have you looked at optimizing the index size? Have you benchmarked the index performance with a variety of RAM sizes? You might be able to get away with a much smaller RAM size at only a relatively small cost in performance.
  • Consider scaling out instead of up. Instead of one large machine look at distributing the index across multiple smaller machines. This is especially important when you consider the possible scaling of your system in the future. Your index is 25GB now but if its 100GB in a few years how are you going to scale it effectively?

Without knowing more about your application/business I would start with just a small/basic dedicated server rented monthly or a smaller Amazon EC2 instance, especially if you're worried about spending a few thousand on a larger machine. See how that small server works, or doesn't work, and see if scaling out is an option. If it is start adding more small servers/instances as needed to reach your performance/budget goals.

share|improve this answer

it depends how much cash your start up has.

Why not use Amazon EC2 instance for this? will it meet your spec?

If you want to buy your own server you can get a dell R410 or dell R510 that will meet this requirement.

You also want to think how you will scale out if you become successful

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.