Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Currently, I am using MySQL but I feel it is not keeping up with the stream data that is coming. I have memcached in front of MySQL for caching, but most the data is accessed for single time so the cache is not helping much. The other problem is the ibdata file keeps getting bigger and even deleting data won't claim disk space.

The data is short, average 200 characters. I just want store the data and retrieve it using a key. There is no complex data relationship. There are about 1,000,000 insert and get in hour but getting much bigger. I even tried Amazon RDS service, but the latency between EC2 and RDS within the same region is slow I started getting MySQL too many connection error.

I know that I have to use Nosql type of database, but not sure which one. Disk space is important that I need to claim it when I delete data.

share|improve this question

closed as not constructive by mailq, MDMarra, Holocryptic, jscott, John Gardeniers Oct 6 '11 at 1:23

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

Are you using InnoDB or MyISAM as the default storage engine ??? What version of MySQL are you using ??? – RolandoMySQLDBA Oct 5 '11 at 21:55

It is only you who can choose. There is a perfect starting point at where you can begin your discovery.

Beside seeing all current databases you can compare their features, options and case studies. This should lead you in the right direction.

share|improve this answer

Have you considered using a simpler data store, since you aren't using relationships and only need to query by a key?

I'm thinking something along the lines of Berkeley DB or Redis.

share|improve this answer

First of all - in SQL world MySQL is fastest engine with very good scalability.

  • I think, you can|have to redesign some parts in order to get real performance, 1 million operation per hour isn't impossible number.
  • Maybe you'll want to examine all possible backend-engines and change used at start,fine-tune mysql-settings
  • persistent connections
  • consider running ANALYZE TABLE (or myisamchk --analyze from command line) on a table to help MySQL better optimize queries
  • Use INSERT DELAYED or INSERT LOW_PRIORITY (for MyISAM) to write to your change log table.
  • if you delete data from tables, schedule running OPTIMIZE TABLE query regularly

TBC, if you'll want

share|improve this answer
I think that Oracle might have a thing or two to say about your opening statement. – MDMarra Oct 5 '11 at 22:03
Microsoft may also have something to say about that. – mrdenny Oct 6 '11 at 1:15
If nothing else, a broad and unqualified claim like that cannot be made by ANY SQL server product. Each one shines or fades under different circumstances. – John Gardeniers Oct 6 '11 at 1:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.