We are currently hosting a 3TB library of image galleries on our web server, all searchable by users. With about 400 active users at a certain time on the website, our disk reads are pushing 100% I/O utilization, and this looks like it could become a bottleneck. This is all caused by the actual image reads rather than database(which we are caching in RAM using Redis). We don't think using something like Varnish would help us, as the reads are pretty much random - we have about 7 million images, and no groups of them get significantly more reads then others as far as we can tell. We use Cloudflare's free plan, but it doesn't seem to be helping. We simply can't afford a full fledged CDN, as we push about 10TB of bandwidth a month and this would cost thousands of dollars.
-
1SSD or a spinning disk JBOD with more spindles and higher performance disks.– dmouratiAug 7, 2014 at 16:00
-
Or more servers behind a load balancer, maybe only for the images.– SvenAug 7, 2014 at 16:03
-
1Can you edit your question to provide more specific details regarding the hardware config if your web server?– I say Reinstate MonicaAug 7, 2014 at 16:14
1 Answer
Scale horizontally, not vertically. Instead of having one server do all of the work, setup many to do the same. They may not all host the same data, which allows a smaller set to access. Hadoop may be one way to do this. Another could be just in the way it's coded. putting a lot of little servers behind varnish would definitely be something that could help you out.
Even without fancy software, domain sharding is a way to reduce load on a single node: http://mvark.blogspot.com/2009/06/what-is-domain-sharding_17.html