Currently my setup is 1 main server which holds all the mp4 videos and 3 other servers which holds portions of this video. The process is: when a user uploads a video, it is stored into the main server, then synced (rsync) oh one of the 3 servers. So the main server acts as a storage server to sync these videos to their respective servers. The slave servers have 12x HDDs on them with raid 10 configuration. Maximum bandwidth peaks at 2GBps per each server on a dedicated 10GBps line. I believe I believe its the HDD bottleneck because my iowait goes up to 40% at peak hours. When the HDDs are set to 2048 readahead, Im able to maintain below 5% iowait.
I'm planning to replace the HDDs with SSDs but discard the raid setup. Instead, go with a bunch of SSDs for each server. Would this be better than my raid 10 setup? currently the HDDs are 7200 only. Another question, why is that setting the HDD read ahead to 2048 lowers the iowait so much and it doesnt affect the bandwidth throughout? it still averages the same or even better than read ahead at default 128k.
each slave server has similar/the same specs:
- DUAL XEON E5 2603
- 128GB RAM
- 12x3TB HDD 7200 Raid 10
- 10Gbps fiber optic NIC
- 10Gbps dedicated port