I do algorithmic trading as a hobby, and I'm about to step up my game in terms of latency.
I have two questions regarding appropriate hardware and database software.
Before I will purchase my hardware, I would like to know what is in general a good setup for storing many datapoints in multiple tables.
The current CPU and harddrive components I'm contemplating are roughly the following:
- Dual CPU: Dual Intel XEON E5-2660 v2 Deca-Core 2.2GHz (3.0GHz Turbo) 25MB Cache
- Dual Seagate 4TB SSHD
- Kingston ECC 1600MHz (8x8 GB)
Would a double CPU + double SSHD be sufficient to be able to process many datapoints (1.000.000+) per second?
And especially, I am extremely curious to the following:
Is there any advantage in having more harddrives?
What if I split my database tables over multiple harddrives, would that increase speed?
Are there any other hardware (especially harddrive) setups that could increase speed when working with databases?
What would be the best database software I can run to store this many datapoints as fast as possible?
Also, which database software allows for simultaneous reading+writing to the same table? (so no locks)
I know of a database system called 'MemSQL' which seems appropriate. Does anybody have any experience with MemsQL? Do you know any other low-latency database software that is recommendable?
I look forward to your replies. Thanks in advance.